Compare commits

..
45 Commits
Author SHA1 Message Date
advplyr 1d89e3a58c Merge branch 'master' into v3 2026-11-12 17:11:21 -06:00
advplyr 75d85e971b Update remove book from collection endpoint to require Can Update permission instead of Can Delete 2026-11-12 17:11:08 -06:00
advplyr 41cbcf551d Merge branch 'master' into v3 2026-11-11 19:09:07 -06:00
advplyr 3e0333877b Fix server crash on logout 2026-11-11 19:09:00 -06:00
advplyr 2dde0bf1c7 Validate credentials are string 2026-11-10 19:33:57 -06:00
advplyr 1e60d7af64 Merge branch 'master' into v3 2026-11-09 17:10:29 -06:00
advplyr 83dce682e8 Add library access checks on media progress endpoints 2026-11-09 17:10:22 -06:00
advplyr f88fdf1321 Merge pull request #5715 from mikiher/fix-book-edit-race
Fix book edit crash when a series or book link already exists
2026-11-09 16:47:57 -06:00
advplyr 20fd0cd700 Merge pull request #5714 from mikiher/fix-library-scan-race
Fix library scan crash when a series or book link already exists
2026-11-09 16:40:20 -06:00
mikiher 9b055f80a6 Fix book edit crash when a series or book link already exists 2026-11-09 19:09:36 +02:00
mikiher 04acd53667 Fix library scan crash when a series or book link already exists 2026-11-09 18:47:44 +02:00
advplyr 92bae242f1 Merge branch 'master' into v3 2026-11-08 18:15:39 -06:00
advplyr 49c0ef6c07 Update collection controller to only emit collection socket events to the user making the request 2026-11-08 18:15:32 -06:00
advplyr 34b037087b Merge branch 'master' into v3 2026-11-05 17:50:37 -06:00
advplyr f57ca09f6e Update session middleware to check session userId with current user 2026-11-05 17:50:21 -06:00
advplyr 844b04c6ae Merge branch 'master' into v3 2026-11-04 17:52:01 -06:00
advplyr 149bae7d73 Merge pull request #5694 from mikiher/fix-itunes-parsing
Keep iTunes author credentials attached to the name
2026-11-04 17:51:41 -06:00
advplyr d6d0cf6b8f Merge pull request #5686 from mikiher/fix-upload-auth
Stop unauthenticated multipart uploads from leaving temp files on disk
2026-11-04 17:50:04 -06:00
advplyr c38fe60360 Remove User legacy API token 2026-11-03 17:00:53 -06:00
mikiher 087099faa3 Keep iTunes author credentials attached to the name 2026-11-02 10:04:19 +02:00
advplyr 8c87ae8bad Merge branch 'master' into v3 2026-11-01 17:27:04 -06:00
advplyr ce41ce489c Merge pull request #5691 from mikiher/dedupe-authors
Dedupe repeated book authors
2026-11-01 17:26:38 -06:00
mikiher b31af26837 Name the library item in the server log when a scan fails on it 2026-11-01 09:14:58 +02:00
mikiher 4072054e00 Dedupe book authors before inserting links so a repeated name does not abort the scan 2026-11-01 09:12:50 +02:00
mikiher f1f57d6595 Add one name dedupe function and use it for parsed names and metadata.json 2026-11-01 09:10:45 +02:00
mikiher 7aaa734d3e Parse multipart uploads only after auth and permission checks 2026-10-31 23:19:19 +02:00
mikiher 48d9309eb2 Delete unmoved upload temp files when the response closes 2026-10-31 23:18:01 +02:00
advplyr f85d03ce26 Set 3.0.0-beta package.json version 2026-10-31 12:08:29 -05:00
advplyr 616c8ba152 Github workflow for building docker images with v3 tag 2026-10-31 11:53:20 -05:00
advplyr edb1f7efc9 Version bump v2.37.2 2026-10-23 14:36:19 -05:00
advplyr 92517f473b Update devcontainer.json node variant to 24 2026-10-23 14:19:39 -05:00
advplyr 9715c7a8f7 Merge pull request #5668 from nichwall/fix/issue-5637-copy-fallback
Add EPERM copy fallback
2026-10-22 17:36:02 -05:00
advplyr 95cb5bad27 Remove fs-extra sync updates 2026-10-22 17:34:09 -05:00
Nicholas Wallace 5cb461f9b5 Add EPERM move fallback 2026-10-20 21:55:23 -07:00
Nicholas Wallace 16aa80ed5c Add EPERM copy fallback 2026-10-20 21:39:26 -07:00
advplyr 743d915220 Normalize RouterBasePath env variable 2026-10-05 17:45:14 -05:00
advplyr 34715558df Use transaction and check root on server init 2026-10-04 20:03:56 -05:00
advplyr 2a0507ffc6 Update linuxpackager to only set 0775 on required files #5629 2026-10-02 17:19:38 -05:00
advplyr cff295fdc9 Merge pull request #5418 from SharonZhu0122/fix-5306-feed-cover-cache
Fix RSS feed cover images not updating after cover change (#5306)
2026-10-01 18:13:31 -05:00
advplyr 6b60beff6e Extract out rss feed image URL construction into a separate function 2026-10-01 18:13:09 -05:00
advplyr db433ce076 Merge pull request #5430 from boqiaok/fix/issue-5176-audible-products
Fix Audible search when products are null
2026-10-01 17:56:26 -05:00
advplyr 5edd20dfa3 Merge pull request #5437 from rtainaan/codex/handle-repeated-upload-files
Fix upload crash with repeated file fields
2026-10-01 17:52:59 -05:00
Raymond (Kai) Tainaan 3f9afe57bc Fix upload crash with repeated file fields
The multipart middleware returns an array when a field is repeated, but handleUpload treats every value as one file. Flatten the values before moving them so repeated and unique field names both work.

Signed-off-by: Raymond (Kai) Tainaan <282025710+rtainaan@users.noreply.github.com>
2026-08-06 23:36:07 -07:00
Asher Jin ac63b0c7fc Fix Audible search when products are null 2026-08-04 14:10:32 +12:00
zsh 94400acdcd fix: add cache-busting timestamp to RSS feed cover image URLs
Feed cover image URLs were built only from the item's slug and file
extension, so the URL never changed when the cover image itself
changed. Podcast clients that cache by URL would keep showing the old
cover after an update.

Add a '?ts=' query parameter using the already-computed
entityUpdatedAt value, matching the same cache-busting convention
already used elsewhere in the app (see the updatedAt comment in
LibraryItemController.updateCover).

Applies to all three feed types: single item, collection, and series.

Fixes #5306
2026-07-31 10:28:21 +12:00
46 changed files with 1248 additions and 287 deletions

No files matched your search

+28 -37
View File
@@ -1,40 +1,31 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/javascript-node
{
"name": "Audiobookshelf",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick a Node version: 18, 16, 14.
// Append -bullseye or -buster to pin to an OS version.
// Use -bullseye variants on local arm64/Apple Silicon.
"args": {
"VARIANT": "20"
}
},
"mounts": [
"source=abs-server-node_modules,target=${containerWorkspaceFolder}/node_modules,type=volume",
"source=abs-client-node_modules,target=${containerWorkspaceFolder}/client/node_modules,type=volume"
],
// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
3000,
3333
],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "sh .devcontainer/post-create.sh",
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"dbaeumer.vscode-eslint",
"octref.vetur"
]
}
}
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
"name": "Audiobookshelf",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick a Node version: 18, 16, 14.
// Append -bullseye or -buster to pin to an OS version.
// Use -bullseye variants on local arm64/Apple Silicon.
"args": {
"VARIANT": "24"
}
},
"mounts": ["source=abs-server-node_modules,target=${containerWorkspaceFolder}/node_modules,type=volume", "source=abs-client-node_modules,target=${containerWorkspaceFolder}/client/node_modules,type=volume"],
// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [3000, 3333],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "sh .devcontainer/post-create.sh",
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Add the IDs of extensions you want installed when the container is created.
"extensions": ["dbaeumer.vscode-eslint", "octref.vetur"]
}
}
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
+102
View File
@@ -0,0 +1,102 @@
# Builds the v3 test image
#
# Moves the :v3 tag on Docker Hub and GHCR, the same way master
# moves :edge. Using the server's v3 branch and react-client master.
#
# repository_dispatch (build-v3) is for triggering a build on updates to react-client
name: Build and Push v3 Docker Image
on:
workflow_dispatch:
push:
branches: [v3]
repository_dispatch:
types: [build-v3]
jobs:
build:
if: ${{ github.repository == 'advplyr/audiobookshelf' && (github.event_name != 'push' || !contains(github.event.head_commit.message, 'skip ci')) }}
runs-on: ubuntu-24.04
steps:
- name: Check out server
uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'repository_dispatch' && 'v3' || github.sha }}
- name: Check out React client
uses: actions/checkout@v4
with:
repository: audiobookshelf/audiobookshelf-client-react
ref: master
path: client-react
- name: Server sha
id: server-sha
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Client sha
id: client-sha
working-directory: client-react
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: advplyr/audiobookshelf,ghcr.io/${{ github.repository_owner }}/audiobookshelf
flavor: |
latest=false
tags: |
type=raw,value=v3
labels: |
audiobookshelf.server.sha=${{ steps.server-sha.outputs.sha }}
audiobookshelf.client.sha=${{ steps.client-sha.outputs.sha }}
- name: Setup QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-v3-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-v3-
- name: Login to Dockerhub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Login to ghcr
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GHCR_PASSWORD }}
- name: Build image
uses: docker/build-push-action@v6
with:
file: client-react/Dockerfile
context: .
build-contexts: |
abs-client=client-react
abs-server=.
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
- name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
+6 -1
View File
@@ -35,7 +35,6 @@ echo "--------------------"
mkdir -p dist
rm -rf dist/debian
cp -R build/debian dist/debian
chmod -R 775 dist/debian
controlfile="Package: audiobookshelf
Version: $VERSION
@@ -48,6 +47,12 @@ Description: $DESCRIPTION"
echo "$controlfile" > dist/debian/DEBIAN/control;
# Package contents: directories 0755, files 0644 (including the systemd unit).
# Maintainer scripts must stay executable so dpkg can run them.
find dist/debian -type d -exec chmod 755 {} +
find dist/debian -type f -exec chmod 644 {} +
chmod 755 dist/debian/DEBIAN/preinst dist/debian/DEBIAN/postinst dist/debian/DEBIAN/prerm
# Package debian
pkg --config package.json -t node24-linux-x64 -o dist/debian/usr/share/audiobookshelf/audiobookshelf dist-server/index.js
@@ -42,7 +42,7 @@
<div v-if="userCanUpdate" class="mx-1" :class="isHovering ? '' : 'ml-6'">
<ui-icon-btn icon="edit" borderless @click="clickEdit" />
</div>
<div v-if="userCanDelete" class="mx-1">
<div v-if="userCanUpdate" class="mx-1">
<ui-icon-btn icon="close" borderless @click="removeClick" />
</div>
</div>
@@ -79,8 +79,7 @@ export default {
},
computed: {
translateDistance() {
if (!this.userCanUpdate && !this.userCanDelete) return 'translate-x-0'
else if (!this.userCanUpdate || !this.userCanDelete) return '-translate-x-12'
if (!this.userCanUpdate) return 'translate-x-0'
return '-translate-x-24'
},
media() {
@@ -144,9 +143,6 @@ export default {
},
userCanUpdate() {
return this.$store.getters['user/getUserCanUpdate']
},
userCanDelete() {
return this.$store.getters['user/getUserCanDelete']
}
},
methods: {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "audiobookshelf-client",
"version": "2.37.1",
"version": "3.0.0-beta",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "audiobookshelf-client",
"version": "2.37.1",
"version": "3.0.0-beta",
"license": "ISC",
"dependencies": {
"@nuxtjs/axios": "^5.13.6",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "audiobookshelf-client",
"version": "2.37.1",
"version": "3.0.0-beta",
"buildNumber": 1,
"description": "Self-hosted audiobook and podcast client",
"main": "index.js",
-11
View File
@@ -13,11 +13,6 @@
<widgets-online-indicator :value="!!userOnline" />
<h1 class="text-xl pl-2">{{ username }}</h1>
</div>
<div v-if="legacyToken" class="text-xs space-y-2 mt-4">
<ui-text-input-with-label label="Legacy API Token" :value="legacyToken" readonly show-copy />
<p class="text-warning" v-html="$strings.MessageAuthenticationLegacyTokenWarning" />
</div>
<div class="w-full h-px bg-white/10 my-2" />
<div class="py-2">
<h1 class="text-lg mb-2 text-white/90 px-2 sm:px-0">{{ $strings.HeaderListeningStats }}</h1>
@@ -102,12 +97,6 @@ export default {
}
},
computed: {
legacyToken() {
return this.user.token
},
userToken() {
return this.user.accessToken
},
bookCoverAspectRatio() {
return this.$store.getters['libraries/getBookCoverAspectRatio']
},
+1 -24
View File
@@ -40,15 +40,6 @@
<p v-if="error" class="text-error text-center py-2">{{ error }}</p>
<div v-if="showNewAuthSystemMessage" class="mb-4">
<widgets-alert type="warning">
<div>
<p>{{ $strings.MessageAuthenticationSecurityMessage }}</p>
<a v-if="showNewAuthSystemAdminMessage" href="https://github.com/advplyr/audiobookshelf/discussions/4460" target="_blank" class="underline">{{ $strings.LabelMoreInfo }}</a>
</div>
</widgets-alert>
</div>
<form v-show="login_local" @submit.prevent="submitForm">
<label class="text-xs text-gray-300 uppercase">{{ $strings.LabelUsername }}</label>
<ui-text-input v-model.trim="username" autocomplete="username" :disabled="processing" class="mb-3 w-full" inputName="username" />
@@ -94,10 +85,7 @@ export default {
MetadataPath: '',
login_local: true,
login_openid: false,
authFormData: null,
// New JWT auth system re-login flags
showNewAuthSystemMessage: false,
showNewAuthSystemAdminMessage: false
authFormData: null
}
},
watch: {
@@ -201,8 +189,6 @@ export default {
},
async submitForm() {
this.error = null
this.showNewAuthSystemMessage = false
this.showNewAuthSystemAdminMessage = false
this.processing = true
const payload = {
@@ -238,15 +224,6 @@ export default {
}
})
.then((res) => {
// Force re-login if user is using an old token with no expiration
if (res.user.isOldToken) {
this.username = res.user.username
this.showNewAuthSystemMessage = true
// Admin user sees link to github discussion
this.showNewAuthSystemAdminMessage = res.user.type === 'admin' || res.user.type === 'root'
return false
}
this.setUser(res)
return true
})
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "audiobookshelf",
"version": "2.37.1",
"version": "3.0.0-beta",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "audiobookshelf",
"version": "2.37.1",
"version": "3.0.0-beta",
"license": "GPL-3.0",
"dependencies": {
"axios": "^0.27.2",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "audiobookshelf",
"version": "2.37.1",
"version": "3.0.0-beta",
"buildNumber": 1,
"description": "Self-hosted audiobook and podcast server",
"main": "dist-server/index.js",
+2 -16
View File
@@ -62,18 +62,6 @@ class Auth {
return passport.authenticate('jwt', { session: false })(req, res, next)
}
/**
* Function to generate a jwt token for a given user
* TODO: Old method with no expiration
* @deprecated
*
* @param {{ id:string, username:string }} user
* @returns {string}
*/
generateAccessToken(user) {
return this.tokenManager.generateAccessToken(user)
}
/**
* Invalidate all JWT sessions for a given user
* If user is current user and refresh token is valid, rotate tokens for the current session
@@ -269,10 +257,8 @@ class Auth {
} else {
// UI request -> check if we have a callback url
if (req.cookies.auth_cb) {
let stateQuery = req.cookies.auth_state ? `&state=${req.cookies.auth_state}` : ''
// UI request -> redirect to auth_cb url and send the jwt token as parameter
// TODO: Temporarily continue sending the old token as setToken
res.redirect(302, `${req.cookies.auth_cb}?setToken=${userResponse.user.token}&accessToken=${userResponse.user.accessToken}${stateQuery}`)
const stateQuery = req.cookies.auth_state ? `&state=${req.cookies.auth_state}` : ''
res.redirect(302, `${req.cookies.auth_cb}?accessToken=${userResponse.user.accessToken}${stateQuery}`)
} else {
Logger.error('[Auth] No callback or already expired')
res.redirect(`/login?error=${encodeURIComponent('No callback or already expired')}&autoLaunch=0`)
+16 -5
View File
@@ -415,14 +415,25 @@ class Database {
* Create root user
* @param {string} username
* @param {string} pash
* @param {Auth} auth
* @returns {Promise<boolean>} true if created
*/
async createRootUser(username, pash, auth) {
async createRootUser(username, pash) {
if (!this.sequelize) return false
await this.userModel.createRootUser(username, pash, auth)
this.hasRootUser = true
return true
const transaction = await this.sequelize.transaction()
try {
const rootCount = await this.userModel.count({ where: { type: 'root' }, transaction })
if (rootCount > 0) {
await transaction.rollback()
return false
}
await this.userModel.createRootUser(username, pash, transaction)
await transaction.commit()
this.hasRootUser = true
return true
} catch (error) {
await transaction.rollback()
throw error
}
}
updateServerSettings() {
+24 -13
View File
@@ -4,7 +4,6 @@ const express = require('express')
const http = require('http')
const util = require('util')
const fs = require('./libs/fsExtra')
const fileUpload = require('./libs/expressFileupload')
const cookieParser = require('cookie-parser')
const axios = require('axios')
@@ -13,6 +12,7 @@ const { version } = require('../package.json')
// Utils
const is = require('./libs/requestIp/isJs')
const fileUtils = require('./utils/fileUtils')
const { sweepUploadTempFiles } = require('./utils/fileUploadMiddleware')
const { toNumber } = require('./utils/index')
const { getRequestOrigin } = require('./utils/requestUtils')
const Logger = require('./Logger')
@@ -46,6 +46,22 @@ const passport = require('passport')
const expressSession = require('express-session')
const MemoryStore = require('./libs/memorystore')
/**
* Normalize to '' (root) or '/segment' with no trailing slash.
* Matches audiobookshelf-client-react `src/lib/basePath.ts`.
* @param {string|null|undefined} basePath
* @returns {string}
*/
function normalizeBasePath(basePath) {
if (!basePath) return ''
const trimmed = basePath.trim()
if (!trimmed || trimmed === '/') return ''
const withLeadingSlash = trimmed.startsWith('/') ? trimmed : `/${trimmed}`
return withLeadingSlash.endsWith('/') ? withLeadingSlash.slice(0, -1) : withLeadingSlash
}
/**
* `next()` for a custom server is a NextCustomServer wrapper. After prepare(),
* `.server` is another NextServer wrapper; the Node server with `nextConfig` is
@@ -66,7 +82,7 @@ class Server {
global.isWin = process.platform === 'win32'
global.ConfigPath = fileUtils.filePathToPOSIX(Path.normalize(CONFIG_PATH))
global.MetadataPath = fileUtils.filePathToPOSIX(Path.normalize(METADATA_PATH))
global.RouterBasePath = ROUTER_BASE_PATH
global.RouterBasePath = normalizeBasePath(ROUTER_BASE_PATH)
global.XAccel = process.env.USE_X_ACCEL
global.AllowCors = process.env.ALLOW_CORS === '1'
@@ -183,6 +199,7 @@ class Server {
await this.cleanUserData() // Remove invalid user item progress
await CacheManager.ensureCachePaths()
await sweepUploadTempFiles()
await ShareManager.init()
await this.backupManager.init()
@@ -320,16 +337,6 @@ class Server {
this.server = http.createServer(app)
// Skip file upload parsing for internal-api routes (Next.js proxies read multipart bodies).
router.use(
/^(?!\/internal-api).*/,
fileUpload({
defCharset: 'utf8',
defParamCharset: 'utf8',
useTempFiles: true,
tempFileDir: Path.join(global.MetadataPath, 'tmp')
})
)
router.use(express.urlencoded({ extended: true, limit: '5mb' }))
// Skip JSON parsing for internal-api routes
@@ -472,7 +479,11 @@ class Server {
const rootUsername = newRoot.username || 'root'
const rootPash = newRoot.password ? await this.auth.localAuthStrategy.hashPassword(newRoot.password) : ''
if (!rootPash) Logger.warn(`[Server] Creating root user with no password`)
await Database.createRootUser(rootUsername, rootPash, this.auth)
const created = await Database.createRootUser(rootUsername, rootPash)
if (!created) {
Logger.error(`[Server] attempt to init server when server already has a root user`)
return res.sendStatus(500)
}
res.sendStatus(200)
}
+6
View File
@@ -49,6 +49,12 @@ class LocalAuthStrategy {
* @param {Function} done - Passport callback
*/
async verifyCredentials(req, username, password, done) {
if (typeof username !== 'string' || typeof password !== 'string') {
this.logFailedLoginAttempt(req, typeof username === 'string' ? username : '[non-string]', 'Invalid username or password')
done(null, null)
return
}
// Load the user given it's username
const user = await Database.userModel.getUserByUsername(username.toLowerCase())
+9 -1
View File
@@ -393,7 +393,15 @@ class OidcAuthStrategy {
* @returns {string|null}
*/
getEndSessionUrl(req, idToken, authMethod) {
const client = this.getClient()
// Logout trusts the auth_method cookie. getClient() throws when OpenID is not
// configured, and that throw is outside the request promise, so it exits the process.
let client
try {
client = this.getClient()
} catch (error) {
Logger.error(`[OidcAuth] Cannot build end session URL - ${error.message}`)
return null
}
if (client.issuer.end_session_endpoint && client.issuer.end_session_endpoint.length > 0) {
let postLogoutRedirectUri = null
+3 -34
View File
@@ -80,7 +80,9 @@ class TokenManager {
*/
static isBearerAccessTokenPayload(decoded) {
if (!decoded?.userId) return false
if (decoded.type === 'refresh') return false
// Legacy API tokens were signed with only userId and username, and never expired.
if (decoded.type !== 'access') return false
if (!decoded.exp) return false
return true
}
@@ -100,30 +102,6 @@ class TokenManager {
}
}
/**
* Generate a JWT token for a given user
* TODO: Old method with no expiration
* @deprecated
*
* @param {{ id:string, username:string }} user
* @returns {string}
*/
static generateAccessToken(user) {
return jwt.sign({ userId: user.id, username: user.username }, TokenManager.TokenSecret)
}
/**
* Function to generate a jwt token for a given user
* TODO: Old method with no expiration
* @deprecated
*
* @param {{ id:string, username:string }} user
* @returns {string}
*/
generateAccessToken(user) {
return TokenManager.generateAccessToken(user)
}
/**
* Generate access token for a given user
*
@@ -325,15 +303,6 @@ class TokenManager {
return
}
// TODO: Temporary flag to report old tokens to users
// May be a better place for this but here means we dont have to decode the token again
if (!jwt_payload.exp && !user.isOldToken) {
Logger.debug(`[TokenManager] User ${user.username} is using an access token without an expiration`)
user.isOldToken = true
} else if (jwt_payload.exp && user.isOldToken !== undefined) {
delete user.isOldToken
}
// approve login
done(null, user)
}
+22 -23
View File
@@ -96,12 +96,8 @@ class CollectionController {
return res.status(500).send('Failed to create collection')
}
// Load books expanded
newCollection.books = await newCollection.getBooksExpandedWithLibraryItem()
// Note: The old collection model stores expanded libraryItems in the books property
const jsonExpanded = newCollection.toOldJSONExpanded()
SocketAuthority.emitter('collection_added', jsonExpanded)
const jsonExpanded = await newCollection.getOldJsonExpanded(req.user)
if (jsonExpanded) SocketAuthority.clientEmitter(req.user.id, 'collection_added', jsonExpanded)
res.json(jsonExpanded)
}
@@ -196,9 +192,9 @@ class CollectionController {
}
}
const jsonExpanded = await req.collection.getOldJsonExpanded()
if (wasUpdated) {
SocketAuthority.emitter('collection_updated', jsonExpanded)
const jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
if (wasUpdated && jsonExpanded) {
SocketAuthority.clientEmitter(req.user.id, 'collection_updated', jsonExpanded)
}
res.json(jsonExpanded)
}
@@ -212,14 +208,17 @@ class CollectionController {
* @param {Response} res
*/
async delete(req, res) {
const jsonExpanded = await req.collection.getOldJsonExpanded()
const jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
// Close rss feed - remove from db and emit socket event
await RssFeedManager.closeFeedForEntityId(req.collection.id)
if (jsonExpanded) {
SocketAuthority.clientEmitter(req.user.id, 'collection_removed', jsonExpanded)
}
await req.collection.destroy()
SocketAuthority.emitter('collection_removed', jsonExpanded)
res.sendStatus(200)
}
@@ -254,8 +253,8 @@ class CollectionController {
bookId: libraryItem.mediaId,
order: collectionBooks.length + 1
})
const jsonExpanded = await req.collection.getOldJsonExpanded()
SocketAuthority.emitter('collection_updated', jsonExpanded)
const jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
if (jsonExpanded) SocketAuthority.clientEmitter(req.user.id, 'collection_updated', jsonExpanded)
res.json(jsonExpanded)
}
@@ -299,10 +298,10 @@ class CollectionController {
order++
}
jsonExpanded = await req.collection.getOldJsonExpanded()
SocketAuthority.emitter('collection_updated', jsonExpanded)
jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
if (jsonExpanded) SocketAuthority.clientEmitter(req.user.id, 'collection_updated', jsonExpanded)
} else {
jsonExpanded = await req.collection.getOldJsonExpanded()
jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
}
res.json(jsonExpanded)
}
@@ -361,10 +360,10 @@ class CollectionController {
if (hasUpdated) {
await Database.collectionBookModel.bulkCreate(collectionBooksToAdd)
jsonExpanded = await req.collection.getOldJsonExpanded()
SocketAuthority.emitter('collection_updated', jsonExpanded)
jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
if (jsonExpanded) SocketAuthority.clientEmitter(req.user.id, 'collection_updated', jsonExpanded)
} else {
jsonExpanded = await req.collection.getOldJsonExpanded()
jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
}
res.json(jsonExpanded)
}
@@ -417,9 +416,9 @@ class CollectionController {
order++
}
let jsonExpanded = await req.collection.getOldJsonExpanded()
if (hasUpdated) {
SocketAuthority.emitter('collection_updated', jsonExpanded)
let jsonExpanded = await req.collection.getOldJsonExpanded(req.user)
if (hasUpdated && jsonExpanded) {
SocketAuthority.clientEmitter(req.user.id, 'collection_updated', jsonExpanded)
}
res.json(jsonExpanded)
}
@@ -447,7 +446,7 @@ class CollectionController {
if (req.method == 'DELETE' && !req.params.bookId && !req.user.canDelete) {
Logger.warn(`[CollectionController] User "${req.user.username}" attempted to delete without permission`)
return res.sendStatus(403)
} else if ((req.method == 'PATCH' || req.method == 'POST') && !req.user.canUpdate) {
} else if ((req.method == 'PATCH' || req.method == 'POST' || (req.method == 'DELETE' && req.params.bookId)) && !req.user.canUpdate) {
Logger.warn(`[CollectionController] User "${req.user.username}" attempted to update without permission`)
return res.sendStatus(403)
}
+1 -1
View File
@@ -495,7 +495,7 @@ class MeController {
for (const mediaProgress of mediaProgressesInProgress) {
const oldMediaProgress = mediaProgress.getOldMediaProgress()
const libraryItem = libraryItems.find((li) => li.id === oldMediaProgress.libraryItemId)
if (libraryItem) {
if (libraryItem && req.user.checkCanAccessLibraryItem(libraryItem)) {
if (oldMediaProgress.episodeId && libraryItem.isPodcast) {
const episode = libraryItem.media.podcastEpisodes.find((ep) => ep.id === oldMediaProgress.episodeId)
if (episode) {
+1 -1
View File
@@ -44,7 +44,7 @@ class MiscController {
return res.sendStatus(400)
}
const files = Object.values(req.files)
const files = Object.values(req.files).flat()
let { title, author, series, folder: folderId, library: libraryId } = req.body
// Validate request body
if (!libraryId || !folderId || typeof libraryId !== 'string' || typeof folderId !== 'string' || !title || typeof title !== 'string') {
+5
View File
@@ -361,6 +361,11 @@ class SessionController {
return res.sendStatus(404)
}
if (playbackSession.userId !== req.user.id && !req.user.isAdminOrUp) {
Logger.error(`[SessionController] Non-admin user "${req.user.username}" attempted to access session belonging to another user "${req.params.id}"`)
return res.sendStatus(403)
}
if (req.method == 'DELETE' && !req.user.canDelete) {
Logger.warn(`[SessionController] User "${req.user.username}" attempted to delete without permission`)
return res.sendStatus(403)
+2 -13
View File
@@ -28,13 +28,12 @@ class UserController {
*/
async findAll(req, res) {
if (!req.user.isAdminOrUp) return res.sendStatus(403)
const hideRootToken = !req.user.isRoot
const includes = (req.query.include || '').split(',').map((i) => i.trim())
// Minimal toJSONForBrowser does not include mediaProgress and bookmarks
const allUsers = await Database.userModel.findAll()
const users = allUsers.map((u) => u.toOldJSONForBrowser(hideRootToken, true))
const users = allUsers.map((u) => u.toOldJSONForBrowser(true))
if (includes.includes('latestSession')) {
for (const user of users) {
@@ -97,7 +96,7 @@ class UserController {
return oldMediaProgress
})
const userJson = req.reqUser.toOldJSONForBrowser(!req.user.isRoot)
const userJson = req.reqUser.toOldJSONForBrowser()
userJson.mediaProgress = oldMediaProgresses
@@ -128,7 +127,6 @@ class UserController {
const userId = uuidv4()
const pash = await this.auth.localAuthStrategy.hashPassword(req.body.password)
const token = this.auth.generateAccessToken({ id: userId, username: req.body.username })
const userType = req.body.type || 'user'
// librariesAccessible and itemTagsSelected can be on req.body or req.body.permissions
@@ -173,7 +171,6 @@ class UserController {
username: req.body.username,
email: typeof req.body.email === 'string' ? req.body.email : null,
pash,
token,
isActive: !!req.body.isActive,
permissions,
bookmarks: [],
@@ -236,16 +233,13 @@ class UserController {
}
let hasUpdates = false
let shouldUpdateToken = false
let shouldInvalidateJwtSessions = false
// When changing username create a new API token
if (updatePayload.username && updatePayload.username !== user.username) {
const usernameExists = await Database.userModel.checkUserExistsWithUsername(updatePayload.username)
if (usernameExists) {
return res.status(400).send('Username already taken')
}
user.username = updatePayload.username
shouldUpdateToken = true
shouldInvalidateJwtSessions = true
hasUpdates = true
}
@@ -327,11 +321,6 @@ class UserController {
}
if (hasUpdates) {
if (shouldUpdateToken) {
user.token = this.auth.generateAccessToken(user)
Logger.info(`[UserController] User ${user.username} has generated a new api token`)
}
// Handle JWT session invalidation for username/password changes
if (shouldInvalidateJwtSessions) {
const newTokens = await this.auth.invalidateJwtSessionsForUser(user, req, res)
@@ -26,6 +26,18 @@ const waitFlushProperty = Symbol('wait flush property symbol');
*/
module.exports = (options, req, res, next) => {
req.files = null;
const pendingCleanups = [];
const runPendingCleanups = () => {
while (pendingCleanups.length) {
const cleanupFn = pendingCleanups.pop();
try {
cleanupFn();
} catch (err) {
debugLog(options, `Temp file cleanup failed: ${err}`);
}
}
};
res.once('close', runPendingCleanups);
// Build busboy options and init busboy instance.
const busboyOptions = buildOptions(options, { headers: req.headers });
@@ -62,6 +74,7 @@ module.exports = (options, req, res, next) => {
} = options.useTempFiles
? tempFileHandler(options, field, filename) // Upload into temporary file.
: memHandler(options, field, filename); // Upload into RAM.
pendingCleanups.push(cleanup);
const writePromise = options.useTempFiles
? getWritePromise().catch(err => {
@@ -161,6 +174,7 @@ module.exports = (options, req, res, next) => {
busboy.on('error', (err) => {
debugLog(options, `Busboy error`);
runPendingCleanups();
next(err);
});
@@ -28,6 +28,18 @@ module.exports = (options, fieldname, filename) => {
reject(err);
});
});
// cleanup() can run from an error path and again on response close.
let cleanedUp = false;
// Windows cannot unlink while the write stream still holds the file.
const closeWriteStream = () => new Promise((resolve) => {
if (writeStream.destroyed || writeStream.closed) {
return resolve();
}
writeStream.once('close', resolve);
writeStream.once('error', resolve);
writeStream.end();
});
return {
dataHandler: (data) => {
@@ -51,13 +63,21 @@ module.exports = (options, fieldname, filename) => {
return Buffer.concat([]);
},
cleanup: () => {
if (cleanedUp) return Promise.resolve();
cleanedUp = true;
completed = true;
debugLog(options, `Cleaning up temporary file ${tempFilePath}...`);
writeStream.end();
deleteFile(tempFilePath, err => (err
? debugLog(options, `Cleaning up temporary file ${tempFilePath} failed: ${err}`)
: debugLog(options, `Cleaning up temporary file ${tempFilePath} done.`)
));
return closeWriteStream().then(() => new Promise((resolve) => {
// ENOENT: handler already mv()'d the file; that is success.
deleteFile(tempFilePath, err => {
if (err && err.code !== 'ENOENT') {
debugLog(options, `Cleaning up temporary file ${tempFilePath} failed: ${err}`);
} else {
debugLog(options, `Cleaning up temporary file ${tempFilePath} done.`);
}
resolve();
});
}));
},
getWritePromise: () => writePromise
};
+12 -1
View File
@@ -2,6 +2,7 @@
const fs = require('graceful-fs')
const path = require('path')
const { pipeline } = require('stream')
const mkdirs = require('../mkdirs').mkdirs
const pathExists = require('../path-exists').pathExists
const utimesMillis = require('../util/utimes').utimesMillis
@@ -98,10 +99,20 @@ function mayCopyFile (srcStat, src, dest, opts, cb) {
}
function copyFile (srcStat, src, dest, opts, cb) {
fs.copyFile(src, dest, err => {
const finish = err => {
if (err) return cb(err)
if (opts.preserveTimestamps) return handleTimestampsAndMode(srcStat.mode, src, dest, cb)
return setDestMode(dest, srcStat.mode, cb)
}
fs.copyFile(src, dest, err => {
if (err?.code !== 'EPERM' || !srcStat.isFile()) return finish(err)
// Stream the fallback so database copies do not require a full-file buffer.
pipeline(fs.createReadStream(src, { highWaterMark: 1024 * 1024 }), fs.createWriteStream(dest, { highWaterMark: 1024 * 1024 }), streamError => {
if (streamError) return fs.unlink(dest, () => cb(streamError))
finish()
})
})
}
+1 -1
View File
@@ -56,7 +56,7 @@ function doRename (src, dest, overwrite, isChangingCase, cb) {
function rename (src, dest, overwrite, cb) {
fs.rename(src, dest, err => {
if (!err) return cb()
if (err.code !== 'EXDEV') return cb(err)
if (err.code !== 'EXDEV' && err.code !== 'EPERM') return cb(err)
return moveAcrossDevice(src, dest, overwrite, cb)
})
}
+26 -5
View File
@@ -439,6 +439,7 @@ class Book extends Model {
*/
async updateAuthorsFromRequest(authors, libraryId) {
if (!Array.isArray(authors)) return null
authors = parseNameString.dedupeNames(authors)
if (!this.authors) {
throw new Error(`[Book] Cannot update authors because authors are not loaded for book ${this.id}`)
@@ -466,7 +467,15 @@ class Book extends Model {
const authorsAdded = []
for (const authorName of newAuthorNames) {
const { author, created } = await authorModel.findOrCreateByNameAndLibrary(authorName, libraryId)
await bookAuthorModel.create({ bookId: this.id, authorId: author.id })
// findOrCreate matches case-insensitively, so a second spelling can resolve to an author just linked
if (this.authors.some((au) => au.id === author.id)) continue
try {
await bookAuthorModel.create({ bookId: this.id, authorId: author.id })
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
this.authors.push(author)
continue
}
if (created) {
SocketAuthority.emitter('author_added', author.toOldJSON())
} else {
@@ -522,11 +531,23 @@ class Book extends Model {
}
} else {
const series = await seriesModel.findOrCreateByNameAndLibrary(seriesObj.name, libraryId)
series.bookSeries = await bookSeriesModel.create({ bookId: this.id, seriesId: series.id, sequence: seriesObjSequence })
try {
series.bookSeries = await bookSeriesModel.create({ bookId: this.id, seriesId: series.id, sequence: seriesObjSequence })
seriesAdded.push(series)
hasUpdates = true
Logger.debug(`[Book] "${this.title}" Added series "${series.name}"`)
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
series.bookSeries = await bookSeriesModel.findOne({ where: { bookId: this.id, seriesId: series.id } })
if (!series.bookSeries) throw error
if (series.bookSeries.sequence !== seriesObjSequence) {
series.bookSeries.sequence = seriesObjSequence
await series.bookSeries.save()
hasUpdates = true
Logger.debug(`[Book] "${this.title}" Updated series "${series.name}" sequence ${seriesObjSequence}`)
}
}
this.series.push(series)
seriesAdded.push(series)
hasUpdates = true
Logger.debug(`[Book] "${this.title}" Added series "${series.name}"`)
}
}
+15 -3
View File
@@ -87,6 +87,18 @@ class Feed extends Model {
)
}
/**
* @param {string} slug
* @param {string|null|undefined} coverPath
* @param {Date|null|undefined} entityUpdatedAt
* @returns {string}
*/
static getFeedImageURL(slug, coverPath, entityUpdatedAt) {
if (!coverPath) return '/Logo.png'
const cacheBuster = entityUpdatedAt != null ? `?ts=${entityUpdatedAt.valueOf()}` : ''
return `/feed/${slug}/cover${Path.extname(coverPath)}${cacheBuster}`
}
/**
*
* @param {string} userId
@@ -119,7 +131,7 @@ class Feed extends Model {
entityUpdatedAt,
serverAddress,
feedURL: `/feed/${slug}`,
imageURL: media.coverPath ? `/feed/${slug}/cover${Path.extname(media.coverPath)}` : `/Logo.png`,
imageURL: Feed.getFeedImageURL(slug, media.coverPath, entityUpdatedAt),
siteURL: `/item/${libraryItem.id}`,
title: media.title,
description: media.description,
@@ -212,7 +224,7 @@ class Feed extends Model {
entityUpdatedAt,
serverAddress,
feedURL: `/feed/${slug}`,
imageURL: firstBookWithCover?.coverPath ? `/feed/${slug}/cover${Path.extname(firstBookWithCover.coverPath)}` : `/Logo.png`,
imageURL: Feed.getFeedImageURL(slug, firstBookWithCover?.coverPath, entityUpdatedAt),
siteURL: `/collection/${collectionExpanded.id}`,
title: collectionExpanded.name,
description: collectionExpanded.description || '',
@@ -301,7 +313,7 @@ class Feed extends Model {
entityUpdatedAt,
serverAddress,
feedURL: `/feed/${slug}`,
imageURL: firstBookWithCover?.coverPath ? `/feed/${slug}/cover${Path.extname(firstBookWithCover.coverPath)}` : `/Logo.png`,
imageURL: Feed.getFeedImageURL(slug, firstBookWithCover?.coverPath, entityUpdatedAt),
siteURL: `/library/${booksWithTracks[0].libraryItem.libraryId}/series/${seriesExpanded.id}`,
title: seriesExpanded.name,
description: seriesExpanded.description || '',
+15 -5
View File
@@ -75,11 +75,21 @@ class Series extends Model {
static async findOrCreateByNameAndLibrary(seriesName, libraryId) {
const series = await this.getByNameAndLibrary(seriesName, libraryId)
if (series) return series
return this.create({
name: seriesName,
nameIgnorePrefix: getTitleIgnorePrefix(seriesName),
libraryId
})
try {
return await this.create({
name: seriesName,
nameIgnorePrefix: getTitleIgnorePrefix(seriesName),
libraryId
})
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
// The unique index is on the exact name. lower() can miss that row.
const existingSeries = await this.findOne({
where: { name: seriesName, libraryId }
})
if (!existingSeries) throw error
return existingSeries
}
}
/**
+37 -23
View File
@@ -5,7 +5,6 @@ const { LRUCache } = require('lru-cache')
const Logger = require('../Logger')
const SocketAuthority = require('../SocketAuthority')
const { isNullOrNaN } = require('../utils')
const TokenManager = require('../auth/TokenManager')
class UserCache {
constructor() {
@@ -94,8 +93,6 @@ class User extends Model {
this.pash
/** @type {string} */
this.type
/** @type {string} */
this.token
/** @type {boolean} */
this.isActive
/** @type {boolean} */
@@ -186,20 +183,16 @@ class User extends Model {
* Create root user
* @param {string} username
* @param {string} pash
* @param {import('../Auth')} auth
* @returns {Promise<User>}
*/
static async createRootUser(username, pash, auth) {
static async createRootUser(username, pash, transaction = null) {
const userId = uuidv4()
const token = auth.generateAccessToken({ id: userId, username })
const newUser = {
id: userId,
type: 'root',
username,
pash,
token,
isActive: true,
permissions: this.getDefaultPermissionsForUserType('root'),
bookmarks: [],
@@ -207,7 +200,7 @@ class User extends Model {
seriesHideFromContinueListening: []
}
}
return this.create(newUser)
return this.create(newUser, { transaction })
}
/**
@@ -314,15 +307,12 @@ class User extends Model {
const username = userinfo.preferred_username || userinfo.name || userinfo.sub
const email = userinfo.email && userinfo.email_verified ? userinfo.email : null
const token = TokenManager.generateAccessToken({ id: userId, username })
const newUser = {
id: userId,
type: 'user',
username,
email,
pash: null,
token,
isActive: true,
permissions: this.getDefaultPermissionsForUserType('user'),
bookmarks: [],
@@ -514,6 +504,7 @@ class User extends Model {
email: DataTypes.STRING,
pash: DataTypes.STRING,
type: DataTypes.STRING,
// Column remains so existing databases do not need a migration. New users leave it null.
token: DataTypes.STRING,
isActive: {
type: DataTypes.BOOLEAN,
@@ -596,11 +587,10 @@ class User extends Model {
/**
* User data for browser using old model
*
* @param {boolean} [hideRootToken=false]
* @param {boolean} [minimal=false]
* @returns
*/
toOldJSONForBrowser(hideRootToken = false, minimal = false) {
toOldJSONForBrowser(minimal = false) {
const seriesHideFromContinueListening = this.extraData?.seriesHideFromContinueListening || []
const librariesAccessible = this.permissions?.librariesAccessible || []
const itemTagsSelected = this.permissions?.itemTagsSelected || []
@@ -613,11 +603,6 @@ class User extends Model {
username: this.username,
email: this.email,
type: this.type,
// TODO: Old non-expiring token
token: this.type === 'root' && hideRootToken ? '' : this.token,
// TODO: Temporary flag not saved in db that is set in Auth.js jwtAuthCheck
// Necessary to detect apps using old tokens that no longer match the old token stored on the user
isOldToken: this.isOldToken,
mediaProgress: this.mediaProgresses?.map((mp) => mp.getOldMediaProgress()) || [],
seriesHideFromContinueListening: [...seriesHideFromContinueListening],
bookmarks: this.bookmarks?.map((b) => ({ ...b })) || [],
@@ -743,10 +728,10 @@ class User extends Model {
},
{
model: this.sequelize.models.podcast,
attributes: ['id', 'title'],
attributes: ['id', 'title', 'explicit', 'tags'],
include: {
model: this.sequelize.models.libraryItem,
attributes: ['id']
attributes: ['id', 'libraryId', 'mediaType']
}
}
]
@@ -758,15 +743,38 @@ class User extends Model {
statusCode: 404
}
}
const episodeLibraryItem = podcastEpisode.podcast?.libraryItem
if (!episodeLibraryItem) {
Logger.error(`[User] createUpdateMediaProgress: library item for episode ${progressPayload.episodeId} not found`)
return {
error: 'Library item not found',
statusCode: 404
}
}
if (progressPayload.libraryItemId && episodeLibraryItem.id !== progressPayload.libraryItemId) {
Logger.error(`[User] createUpdateMediaProgress: episode ${progressPayload.episodeId} does not belong to library item ${progressPayload.libraryItemId}`)
return {
error: 'Episode does not belong to library item',
statusCode: 400
}
}
episodeLibraryItem.media = podcastEpisode.podcast
if (!this.checkCanAccessLibraryItem(episodeLibraryItem)) {
Logger.error(`[User] createUpdateMediaProgress: user "${this.username}" attempted to update progress for inaccessible library item "${episodeLibraryItem.id}"`)
return {
error: 'User does not have access to this library item',
statusCode: 403
}
}
mediaItemId = podcastEpisode.id
mediaProgress = podcastEpisode.mediaProgresses?.[0]
podcastId = podcastEpisode.podcastId
} else {
const libraryItem = await this.sequelize.models.libraryItem.findByPk(progressPayload.libraryItemId, {
attributes: ['id', 'mediaId', 'mediaType'],
attributes: ['id', 'libraryId', 'mediaId', 'mediaType'],
include: {
model: this.sequelize.models.book,
attributes: ['id', 'title'],
attributes: ['id', 'title', 'explicit', 'tags'],
required: false,
include: {
model: this.sequelize.models.mediaProgress,
@@ -787,6 +795,12 @@ class User extends Model {
error: 'Library item is not a book',
statusCode: 400
}
} else if (!this.checkCanAccessLibraryItem(libraryItem)) {
Logger.error(`[User] createUpdateMediaProgress: user "${this.username}" attempted to update progress for inaccessible library item "${libraryItem.id}"`)
return {
error: 'User does not have access to this library item',
statusCode: 403
}
}
mediaItemId = libraryItem.media.id
-3
View File
@@ -75,9 +75,6 @@ class Stream extends EventEmitter {
get codecsToForceAAC() {
return ['alac', 'ac3', 'eac3', 'opus']
}
get userToken() {
return this.user.token
}
// Fmp4 does not work on iOS devices: https://github.com/advplyr/audiobookshelf-app/issues/85
// Workaround: Force AAC transcode for FLAC
get hlsSegmentType() {
+1 -1
View File
@@ -160,7 +160,7 @@ class Audible {
timeout
})
.then((res) => {
if (!res?.data?.products) return null
if (!res?.data?.products) return []
return Promise.all(res.data.products.map((result) => this.asinSearch(result.asin, region, timeout)))
})
.catch((error) => {
+50 -5
View File
@@ -27,6 +27,55 @@ const htmlSanitizer = require('../utils/htmlSanitizer')
* @property {boolean} explicit
*/
/**
* Split a list of authors on commas. A piece with no space is a credential and is glued onto the previous name.
*
* @param {string} text
* @returns {string[]}
*/
function parseAuthorPieces(text) {
const authors = []
for (const piece of (text || '').split(',')) {
const name = piece.trim()
if (!name) continue
if (!name.includes(' ') && authors.length) {
authors[authors.length - 1] = `${authors[authors.length - 1]} ${name}`
} else {
authors.push(name)
}
}
return authors
}
function stripCommasFromName(name) {
return (name || '').replace(/,/g, '').replace(/\s+/g, ' ').trim()
}
/**
* Apple lists authors as "Name1, Name2 & Name3", but a comma can also sit inside one name ("Rick Hanson, PhD").
* A comma-separated piece with no space is treated as a credential and glued onto the previous name.
* The text after the last " & " is a single author, so commas there are removed.
* @see https://github.com/advplyr/audiobookshelf/issues/1022
* @see https://github.com/advplyr/audiobookshelf/issues/2661
*
* @param {string} artistName
* @returns {string}
*/
function cleanAuthor(artistName) {
const raw = (artistName || '').trim()
if (!raw) return ''
const lastAmpIndex = raw.lastIndexOf(' & ')
if (lastAmpIndex === -1) {
return stripCommasFromName(raw)
}
const authors = parseAuthorPieces(raw.slice(0, lastAmpIndex))
const lastAuthor = stripCommasFromName(raw.slice(lastAmpIndex + 3))
if (lastAuthor) authors.push(lastAuthor)
return authors.join(', ')
}
class iTunes {
#responseTimeout = 10000
@@ -103,15 +152,11 @@ class iTunes {
}
cleanAudiobook(data) {
// artistName can be "Name1, Name2 & Name3" so we refactor this to "Name1, Name2, Name3"
// see: https://github.com/advplyr/audiobookshelf/issues/1022
const author = (data.artistName || '').split(' & ').join(', ')
return {
id: data.collectionId,
artistId: data.artistId,
title: data.collectionName,
author,
author: cleanAuthor(data.artistName),
description: data.description || null,
publishedYear: data.releaseDate ? data.releaseDate.split('-')[0] : null,
genres: data.primaryGenreName ? [data.primaryGenreName] : null,
+4 -3
View File
@@ -11,6 +11,7 @@ const date = require('../libs/dateAndTime')
const CacheManager = require('../managers/CacheManager')
const RssFeedManager = require('../managers/RssFeedManager')
const { fileUploadParser, requireUploadPermission } = require('../utils/fileUploadMiddleware')
const LibraryController = require('../controllers/LibraryController')
const UserController = require('../controllers/UserController')
@@ -110,7 +111,7 @@ class ApiRouter {
this.router.get('/items/:id/download', LibraryItemController.middleware.bind(this), LibraryItemController.download.bind(this))
this.router.patch('/items/:id/media', LibraryItemController.middleware.bind(this), LibraryItemController.updateMedia.bind(this))
this.router.get('/items/:id/cover', LibraryItemController.getCover.bind(this))
this.router.post('/items/:id/cover', LibraryItemController.middleware.bind(this), LibraryItemController.uploadCover.bind(this))
this.router.post('/items/:id/cover', LibraryItemController.middleware.bind(this), requireUploadPermission, fileUploadParser, LibraryItemController.uploadCover.bind(this))
this.router.patch('/items/:id/cover', LibraryItemController.middleware.bind(this), LibraryItemController.updateCover.bind(this))
this.router.delete('/items/:id/cover', LibraryItemController.middleware.bind(this), LibraryItemController.removeCover.bind(this))
this.router.post('/items/:id/match', LibraryItemController.middleware.bind(this), LibraryItemController.match.bind(this))
@@ -202,7 +203,7 @@ class ApiRouter {
this.router.delete('/backups/:id', BackupController.middleware.bind(this), BackupController.delete.bind(this))
this.router.get('/backups/:id/download', BackupController.middleware.bind(this), BackupController.download.bind(this))
this.router.get('/backups/:id/apply', BackupController.middleware.bind(this), BackupController.apply.bind(this))
this.router.post('/backups/upload', BackupController.middleware.bind(this), BackupController.upload.bind(this))
this.router.post('/backups/upload', BackupController.middleware.bind(this), fileUploadParser, BackupController.upload.bind(this))
this.router.patch('/backups/path', BackupController.middleware.bind(this), BackupController.updatePath.bind(this))
//
@@ -343,7 +344,7 @@ class ApiRouter {
//
// Misc Routes
//
this.router.post('/upload', MiscController.handleUpload.bind(this))
this.router.post('/upload', requireUploadPermission, fileUploadParser, MiscController.handleUpload.bind(this))
this.router.get('/tasks', MiscController.getTasks.bind(this))
this.router.patch('/settings', MiscController.updateServerSettings.bind(this))
this.router.patch('/sorting-prefixes', MiscController.updateSortingPrefixes.bind(this))
+88 -32
View File
@@ -224,12 +224,16 @@ class BookScanner {
if (!media.authors.some((au) => au.name === authorName)) {
const existingAuthorId = await Database.getAuthorIdByName(libraryItemData.libraryId, authorName)
if (existingAuthorId) {
await Database.bookAuthorModel.create({
bookId: media.id,
authorId: existingAuthorId
})
libraryScan.authorsNumBooksChangedIds.add(existingAuthorId)
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added author "${authorName}"`)
try {
await Database.bookAuthorModel.create({
bookId: media.id,
authorId: existingAuthorId
})
libraryScan.authorsNumBooksChangedIds.add(existingAuthorId)
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added author "${authorName}"`)
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
}
authorsUpdated = true
} else {
const newAuthor = await Database.authorModel.create({
@@ -262,22 +266,51 @@ class BookScanner {
if (!existingBookSeries) {
const existingSeriesId = await Database.getSeriesIdByName(libraryItemData.libraryId, seriesObj.name)
if (existingSeriesId) {
await Database.bookSeriesModel.create({
bookId: media.id,
seriesId: existingSeriesId,
sequence: seriesObj.sequence
})
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added series "${seriesObj.name}"${seriesObj.sequence ? ` with sequence "${seriesObj.sequence}"` : ''}`)
try {
await Database.bookSeriesModel.create({
bookId: media.id,
seriesId: existingSeriesId,
sequence: seriesObj.sequence
})
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added series "${seriesObj.name}"${seriesObj.sequence ? ` with sequence "${seriesObj.sequence}"` : ''}`)
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
}
seriesUpdated = true
} else {
const newSeries = await Database.seriesModel.create({
name: seriesObj.name,
nameIgnorePrefix: getTitleIgnorePrefix(seriesObj.name),
libraryId: libraryItemData.libraryId
})
await media.addSeries(newSeries, { through: { sequence: seriesObj.sequence } })
Database.addSeriesToFilterData(libraryItemData.libraryId, newSeries.name, newSeries.id)
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added new series "${seriesObj.name}"${seriesObj.sequence ? ` with sequence "${seriesObj.sequence}"` : ''}`)
let newSeries
let seriesCreated = false
try {
newSeries = await Database.seriesModel.create({
name: seriesObj.name,
nameIgnorePrefix: getTitleIgnorePrefix(seriesObj.name),
libraryId: libraryItemData.libraryId
})
seriesCreated = true
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
newSeries = await Database.seriesModel.findOne({
where: { name: seriesObj.name, libraryId: libraryItemData.libraryId }
})
if (!newSeries) throw error
}
if (seriesCreated) {
Database.addSeriesToFilterData(libraryItemData.libraryId, newSeries.name, newSeries.id)
}
try {
await Database.bookSeriesModel.create({
bookId: media.id,
seriesId: newSeries.id,
sequence: seriesObj.sequence
})
if (seriesCreated) {
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added new series "${seriesObj.name}"${seriesObj.sequence ? ` with sequence "${seriesObj.sequence}"` : ''}`)
} else {
libraryScan.addLog(LogLevel.DEBUG, `Updating book "${bookMetadata.title}" added series "${seriesObj.name}"${seriesObj.sequence ? ` with sequence "${seriesObj.sequence}"` : ''}`)
}
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
}
seriesUpdated = true
}
} else if (seriesObj.sequence && existingBookSeries.bookSeries.sequence !== seriesObj.sequence) {
@@ -507,24 +540,38 @@ class BookScanner {
}
}
if (bookMetadata.series.length) {
const linkedSeriesIds = new Set()
for (const seriesObj of bookMetadata.series) {
if (!seriesObj.name) continue
const matchingSeriesId = await Database.getSeriesIdByName(libraryItemData.libraryId, seriesObj.name)
if (matchingSeriesId) {
bookObject.bookSeries.push({
seriesId: matchingSeriesId,
sequence: seriesObj.sequence
})
} else {
bookObject.bookSeries.push({
sequence: seriesObj.sequence,
series: {
let seriesId = await Database.getSeriesIdByName(libraryItemData.libraryId, seriesObj.name)
if (!seriesId) {
let series
let seriesCreated = false
try {
series = await Database.seriesModel.create({
name: seriesObj.name,
nameIgnorePrefix: getTitleIgnorePrefix(seriesObj.name),
libraryId: libraryItemData.libraryId
}
})
})
seriesCreated = true
} catch (error) {
if (error.name !== 'SequelizeUniqueConstraintError') throw error
series = await Database.seriesModel.findOne({
where: { name: seriesObj.name, libraryId: libraryItemData.libraryId }
})
if (!series) throw error
}
seriesId = series.id
if (seriesCreated) {
Database.addSeriesToFilterData(libraryItemData.libraryId, series.name, series.id)
}
}
if (linkedSeriesIds.has(seriesId)) continue
linkedSeriesIds.add(seriesId)
bookObject.bookSeries.push({
seriesId,
sequence: seriesObj.sequence
})
}
}
@@ -703,6 +750,15 @@ class BookScanner {
bookMetadata.description = htmlSanitizer.sanitize(bookMetadata.description)
}
// One dedupe after every metadata source. A repeated name inserts the same bookAuthor or bookSeries twice and aborts the scan.
bookMetadata.authors = parseNameString.dedupeNames(bookMetadata.authors)
const seenSeriesNames = new Set()
bookMetadata.series = bookMetadata.series.filter((series) => {
if (!series?.name || seenSeriesNames.has(series.name)) return false
seenSeriesNames.add(series.name)
return true
})
return bookMetadata
}
+3
View File
@@ -29,6 +29,9 @@ class LibraryScan {
/** @type {string[]} */
this.seriesRemovedFromBooks = []
/** @type {string} item label while a library item is being scanned, so a thrown error can name it */
this.failureItem = null
this.logs = []
}
+30 -3
View File
@@ -116,7 +116,8 @@ class LibraryScanner {
} catch (err) {
libraryScan.setComplete()
Logger.error(`[LibraryScanner] Library scan ${libraryScan.id} failed after ${libraryScan.elapsedTimestamp} | ${libraryScan.resultStats}.`, err)
const failedItem = libraryScan.failureItem ? ` while scanning ${libraryScan.failureItem}` : ''
Logger.error(`[LibraryScanner] Library scan ${libraryScan.id} failed after ${libraryScan.elapsedTimestamp} | ${libraryScan.resultStats}${failedItem}.`, err)
task.data.scanResults = libraryScan.scanResults
const taskFailedString = {
@@ -169,6 +170,7 @@ class LibraryScanner {
const libraryItemIdsMissing = []
let libraryItemsUpdated = []
for (const existingLibraryItem of existingLibraryItems) {
libraryScan.failureItem = scanItemFailureLabel(existingLibraryItem.title, existingLibraryItem.path)
// First try to find matching library item with exact file path
let libraryItemData = libraryItemDataFound.find((lid) => lid.path === existingLibraryItem.path)
if (!libraryItemData) {
@@ -221,6 +223,8 @@ class LibraryScanner {
}
}
libraryScan.failureItem = null
// Emit item updates in chunks of 10 to client
if (libraryItemsUpdated.length === 10) {
SocketAuthority.libraryItemsEmitter('items_updated', libraryItemsUpdated)
@@ -259,12 +263,14 @@ class LibraryScanner {
if (libraryItemDataFound.length) {
let newLibraryItems = []
for (const libraryItemData of libraryItemDataFound) {
libraryScan.failureItem = scanItemFailureLabel(libraryItemData.mediaMetadata?.title, libraryItemData.path)
const newLibraryItem = await LibraryItemScanner.scanNewLibraryItem(libraryItemData, libraryScan.library.settings, libraryScan)
if (newLibraryItem) {
newLibraryItems.push(newLibraryItem)
libraryScan.resultsAdded++
}
libraryScan.failureItem = null
// Emit new items in chunks of 10 to client
if (newLibraryItems.length === 10) {
@@ -597,7 +603,12 @@ class LibraryScanner {
}
// Scan library item for updates
Logger.debug(`[LibraryScanner] Folder update for relative path "${itemDir}" is in library item "${existingLibraryItem.media.title}" with id "${existingLibraryItem.id}" - scan for updates`)
itemGroupingResults[itemDir] = await LibraryItemScanner.scanLibraryItem(existingLibraryItem.id, updatedLibraryItemDetails)
try {
itemGroupingResults[itemDir] = await LibraryItemScanner.scanLibraryItem(existingLibraryItem.id, updatedLibraryItemDetails)
} catch (err) {
Logger.error(`[LibraryScanner] Failed while scanning ${scanItemFailureLabel(existingLibraryItem.media?.title, existingLibraryItem.path)}`)
throw err
}
continue
} else if (library.settings.audiobooksOnly && !hasAudioFiles(fileUpdateGroup, itemDir)) {
Logger.debug(`[LibraryScanner] Folder update for relative path "${itemDir}" has no audio files`)
@@ -626,7 +637,13 @@ class LibraryScanner {
Logger.debug(`[LibraryScanner] Folder update group must be a new item "${itemDir}" in library "${library.name}"`)
const isSingleMediaItem = isSingleMediaFile(fileUpdateGroup, itemDir)
const newLibraryItem = await LibraryItemScanner.scanPotentialNewLibraryItem(fullPath, library, folder, isSingleMediaItem)
let newLibraryItem
try {
newLibraryItem = await LibraryItemScanner.scanPotentialNewLibraryItem(fullPath, library, folder, isSingleMediaItem)
} catch (err) {
Logger.error(`[LibraryScanner] Failed while scanning ${scanItemFailureLabel(null, fullPath)}`)
throw err
}
if (newLibraryItem) {
SocketAuthority.libraryItemEmitter('item_added', newLibraryItem)
}
@@ -638,6 +655,16 @@ class LibraryScanner {
}
module.exports = new LibraryScanner()
/**
* @param {string} title
* @param {string} path
* @returns {string}
*/
function scanItemFailureLabel(title, path) {
if (title && path) return `"${title}" at "${path}"`
return `"${path || title || 'unknown item'}"`
}
function ItemToFileInoMatch(libraryItem1, libraryItem2) {
return libraryItem1.isFile && libraryItem2.libraryFiles.some((lf) => lf.ino === libraryItem1.ino)
}
+76
View File
@@ -0,0 +1,76 @@
const Path = require('path')
const fs = require('../libs/fsExtra')
const fileUpload = require('../libs/expressFileupload')
const Logger = require('../Logger')
/**
* Multipart parser for the three routes that read req.files.
* Reads MetadataPath per request so route registration does not depend on it.
*
* @param {import('express').Request} req
* @param {import('express').Response} res
* @param {import('express').NextFunction} next
*/
function fileUploadParser(req, res, next) {
fileUpload({
defCharset: 'utf8',
defParamCharset: 'utf8',
useTempFiles: true,
tempFileDir: Path.join(global.MetadataPath, 'tmp')
})(req, res, next)
}
module.exports.fileUploadParser = fileUploadParser
/**
* Reject users without upload permission before the multipart parser writes temp files.
*
* @param {import('express').Request} req
* @param {import('express').Response} res
* @param {import('express').NextFunction} next
*/
function requireUploadPermission(req, res, next) {
if (!req.user?.canUpload) {
Logger.warn(`User "${req.user?.username || 'unknown'}" attempted to upload without permission`)
return res.sendStatus(403)
}
next()
}
module.exports.requireUploadPermission = requireUploadPermission
/**
* Remove leftover upload temp files (tmp-*) from metadata/tmp.
* Comic extraction uses the same directory with other names; those stay.
*
* @returns {Promise<void>}
*/
async function sweepUploadTempFiles() {
const tmpDir = Path.join(global.MetadataPath, 'tmp')
let names
try {
names = await fs.readdir(tmpDir)
} catch (error) {
if (error.code === 'ENOENT') return
Logger.warn(`[FileUpload] Failed to read upload temp directory "${tmpDir}": ${error.message}`)
return
}
let removed = 0
for (const name of names) {
if (!name.startsWith('tmp-')) continue
const fullPath = Path.join(tmpDir, name)
try {
const stat = await fs.lstat(fullPath)
if (!stat.isFile()) continue
await fs.unlink(fullPath)
removed++
} catch (error) {
if (error.code === 'ENOENT') continue
Logger.warn(`[FileUpload] Failed to remove leftover upload temp file "${fullPath}": ${error.message}`)
}
}
if (removed) {
Logger.info(`[FileUpload] Removed ${removed} leftover upload temp file(s) from "${tmpDir}"`)
}
}
module.exports.sweepUploadTempFiles = sweepUploadTempFiles
@@ -1,5 +1,6 @@
const Logger = require('../../Logger')
const parseSeriesString = require('../parsers/parseSeriesString')
const { dedupeNames } = require('../parsers/parseNameString')
const mediaTypeKeys = {
book: {
@@ -140,8 +141,7 @@ function validateMetadataValue(key, value, expectedType) {
return undefined
}
const cleanedArray = value.filter((t) => typeof t === 'string')
return [...new Set(cleanedArray.map((t) => t.trim()).filter((t) => t))]
return dedupeNames(value)
}
Logger.warn(`[abmetadataGenerator] Unknown expected type "${expectedType}" for key "${key}"`)
+24 -2
View File
@@ -35,6 +35,28 @@ module.exports.nameToLastFirst = (firstLast) => {
return `${nameObj.last_name}, ${nameObj.first_name}`
}
/**
* Trim names, drop blanks, and drop exact repeats. The first spelling is kept.
* Callers that build bookAuthors rows should use this instead of deduping per metadata source.
*
* @param {string[]} names
* @returns {string[]}
*/
function dedupeNames(names) {
if (!Array.isArray(names)) return []
const seen = new Set()
const deduped = []
for (const name of names) {
if (typeof name !== 'string') continue
const trimmed = name.trim()
if (!trimmed || seen.has(trimmed)) continue
seen.add(trimmed)
deduped.push(trimmed)
}
return deduped
}
module.exports.dedupeNames = dedupeNames
/**
* Parses a name string into an array of names
*
@@ -62,7 +84,7 @@ module.exports.parse = (nameString) => {
// If names are in Chinese,Japanese and Korean languages, return as is.
if (/[\u4e00-\u9fff\u3040-\u30ff\u31f0-\u31ff]/.test(splitNames[0])) {
return {
names: splitNames
names: dedupeNames(splitNames)
}
}
@@ -102,7 +124,7 @@ module.exports.parse = (nameString) => {
names = names.filter((n) => n.first_name || n.last_name)
// Set name strings and remove duplicates
const namesArray = [...new Set(names.map((a) => (a.first_name ? `${a.first_name} ${a.last_name}` : a.last_name)))]
const namesArray = dedupeNames(names.map((a) => (a.first_name ? `${a.first_name} ${a.last_name}` : a.last_name)))
return {
names: namesArray // Array of first last
+10
View File
@@ -39,6 +39,16 @@ describe('TokenManager', () => {
describe('jwtAuthCheck', () => {
const user = { id: userId, username: 'testuser', isActive: true }
it('rejects legacy API tokens that have no type or expiration', async () => {
const legacyToken = jwt.sign({ userId, username: 'testuser' }, secret)
const decoded = jwt.verify(legacyToken, secret)
const done = sinon.spy()
await tokenManager.jwtAuthCheck(decoded, done)
expect(done.calledWith(null, null)).to.be.true
})
it('rejects refresh tokens for API auth', async () => {
const refreshToken = tokenManager.generateRefreshToken(user)
const decoded = jwt.verify(refreshToken, secret)
+96
View File
@@ -0,0 +1,96 @@
const { expect } = require('chai')
const sinon = require('sinon')
const fs = require('graceful-fs')
const os = require('os')
const path = require('path')
const fsExtra = require('../../server/libs/fsExtra')
describe('fsExtra copy fallback', () => {
let dir
let source
let dest
let contents
beforeEach(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'abs-copy-'))
source = path.join(dir, 'source')
dest = path.join(dir, 'dest')
contents = Buffer.alloc(2 * 1024 * 1024 + 17, 0x61)
fs.writeFileSync(source, contents)
})
afterEach(() => {
sinon.restore()
fs.rmSync(dir, { recursive: true, force: true })
})
it('copies through streams when asynchronous copyFile is denied', async () => {
sinon.stub(fs, 'copyFile').callsFake((src, target, callback) => {
callback(Object.assign(new Error('copyfile denied'), { code: 'EPERM' }))
})
await fsExtra.copy(source, dest)
expect(fs.readFileSync(dest).equals(contents)).to.be.true
})
it('moves across devices when asynchronous copyFile is denied', async () => {
sinon.stub(fs, 'rename').callsFake((src, target, callback) => {
callback(Object.assign(new Error('cross-device link'), { code: 'EXDEV' }))
})
sinon.stub(fs, 'copyFile').callsFake((src, target, callback) => {
callback(Object.assign(new Error('copyfile denied'), { code: 'EPERM' }))
})
await fsExtra.move(source, dest)
expect(fs.existsSync(source)).to.be.false
expect(fs.readFileSync(dest).equals(contents)).to.be.true
})
it('moves over an existing file when asynchronous rename and copyFile are denied', async () => {
fs.writeFileSync(dest, 'old database')
sinon.stub(fs, 'rename').callsFake((src, target, callback) => {
callback(Object.assign(new Error('rename denied'), { code: 'EPERM' }))
})
sinon.stub(fs, 'copyFile').callsFake((src, target, callback) => {
callback(Object.assign(new Error('copyfile denied'), { code: 'EPERM' }))
})
await fsExtra.move(source, dest, { overwrite: true })
expect(fs.existsSync(source)).to.be.false
expect(fs.readFileSync(dest).equals(contents)).to.be.true
})
it('keeps the source when a move cannot copy it', async () => {
sinon.stub(fs, 'rename').callsFake((src, target, callback) => {
callback(Object.assign(new Error('rename denied'), { code: 'EPERM' }))
})
sinon.stub(fs, 'copyFile').callsFake((src, target, callback) => {
callback(Object.assign(new Error('read failed'), { code: 'EIO' }))
})
try {
await fsExtra.move(source, dest)
expect.fail('Expected move to fail')
} catch (error) {
expect(error.code).to.equal('EIO')
}
expect(fs.readFileSync(source).equals(contents)).to.be.true
})
it('propagates unrelated copy errors', async () => {
sinon.stub(fs, 'copyFile').callsFake((src, target, callback) => {
callback(Object.assign(new Error('file missing'), { code: 'ENOENT' }))
})
try {
await fsExtra.copy(source, dest)
expect.fail('Expected copy to fail')
} catch (error) {
expect(error.code).to.equal('ENOENT')
}
expect(fs.existsSync(dest)).to.be.false
})
})
+71
View File
@@ -0,0 +1,71 @@
const { expect } = require('chai')
const { Sequelize } = require('sequelize')
const sinon = require('sinon')
const Database = require('../../../server/Database')
const Logger = require('../../../server/Logger')
describe('Book author and series updates', () => {
/** @type {import('../../../server/models/Library')} */
let library
/** @type {import('../../../server/models/Book')} */
let book
beforeEach(async () => {
global.ServerSettings = {}
Database.sequelize = new Sequelize({ dialect: 'sqlite', storage: ':memory:', logging: false })
Database.sequelize.uppercaseFirst = (str) => (str ? `${str[0].toUpperCase()}${str.substr(1)}` : '')
await Database.buildModels()
sinon.stub(Logger, 'info')
sinon.stub(Logger, 'debug')
sinon.stub(Logger, 'warn')
sinon.stub(Logger, 'error')
library = await Database.libraryModel.create({ name: 'Test Library', mediaType: 'book' })
book = await Database.bookModel.create({ title: 'Test Book', audioFiles: [], tags: [], narrators: [], genres: [], chapters: [] })
book.authors = []
book.series = []
})
afterEach(async () => {
sinon.restore()
await Database.sequelize.sync({ force: true })
})
it('does not throw when the bookSeries link already exists', async () => {
const series = await Database.seriesModel.create({ name: 'Crescent City', libraryId: library.id })
await Database.bookSeriesModel.create({ bookId: book.id, seriesId: series.id, sequence: '3' })
const result = await book.updateSeriesFromRequest([{ name: 'Crescent City', sequence: '3' }], library.id)
expect(result.hasUpdates).to.be.false
const links = await Database.bookSeriesModel.findAll({ where: { bookId: book.id, seriesId: series.id } })
expect(links).to.have.lengthOf(1)
})
it('reuses an existing series when create hits the name unique constraint', async () => {
const series = await Database.seriesModel.create({ name: 'Crescent City', libraryId: library.id })
sinon.stub(Database.seriesModel, 'getByNameAndLibrary').resolves(null)
const result = await book.updateSeriesFromRequest([{ name: 'Crescent City', sequence: '3' }], library.id)
expect(result.seriesAdded).to.have.lengthOf(1)
expect(result.seriesAdded[0].id).to.equal(series.id)
const seriesRows = await Database.seriesModel.findAll({ where: { libraryId: library.id, name: 'Crescent City' } })
expect(seriesRows).to.have.lengthOf(1)
const links = await Database.bookSeriesModel.findAll({ where: { bookId: book.id, seriesId: series.id } })
expect(links).to.have.lengthOf(1)
})
it('does not throw when the bookAuthor link already exists', async () => {
const author = await Database.authorModel.create({ name: 'Sarah Maas', libraryId: library.id })
await Database.bookAuthorModel.create({ bookId: book.id, authorId: author.id })
const result = await book.updateAuthorsFromRequest(['Sarah Maas'], library.id)
expect(result.authorsAdded).to.have.lengthOf(0)
const links = await Database.bookAuthorModel.findAll({ where: { bookId: book.id, authorId: author.id } })
expect(links).to.have.lengthOf(1)
})
})
+53
View File
@@ -0,0 +1,53 @@
const iTunes = require('../../../server/providers/iTunes')
const { expect } = require('chai')
describe('iTunes', () => {
let itunes
beforeEach(() => {
itunes = new iTunes()
})
describe('cleanAudiobook', () => {
function authorFrom(artistName) {
return itunes.cleanAudiobook({ artistName }).author
}
it('should return an empty string if artistName is missing', () => {
expect(authorFrom()).to.equal('')
expect(authorFrom('')).to.equal('')
})
it('should keep a single author without commas', () => {
expect(authorFrom('Joy Fielding')).to.equal('Joy Fielding')
})
it('should strip commas from a single author with a credential', () => {
expect(authorFrom('Rick Hanson, PhD')).to.equal('Rick Hanson PhD')
})
it('should split two authors joined by an ampersand', () => {
expect(authorFrom('Name1 & Name2')).to.equal('Name1, Name2')
})
it('should split three authors using commas and an ampersand', () => {
expect(authorFrom('Keltie Knight, Becca Tobin & Jac Vanek')).to.equal('Keltie Knight, Becca Tobin, Jac Vanek')
})
it('should glue a one-word credential onto the previous author before the ampersand', () => {
expect(authorFrom('Rick Hanson, PhD & Jane Doe')).to.equal('Rick Hanson PhD, Jane Doe')
})
it('should glue several one-word credentials in a row', () => {
expect(authorFrom('Rick Hanson, PhD, MD & Jane Doe')).to.equal('Rick Hanson PhD MD, Jane Doe')
})
it('should strip commas from the last author', () => {
expect(authorFrom('Rick Hanson & Jane Doe, PhD')).to.equal('Rick Hanson, Jane Doe PhD')
})
it('should glue credentials before the ampersand and strip commas from the last author', () => {
expect(authorFrom('Rick Hanson, PhD & Jane Doe, PhD')).to.equal('Rick Hanson PhD, Jane Doe PhD')
})
})
})
+123
View File
@@ -0,0 +1,123 @@
const { expect } = require('chai')
const { Sequelize } = require('sequelize')
const sinon = require('sinon')
const Database = require('../../../server/Database')
const Logger = require('../../../server/Logger')
const BookScanner = require('../../../server/scanner/BookScanner')
const LibraryScan = require('../../../server/scanner/LibraryScan')
describe('BookScanner.rescanExistingBookLibraryItem', () => {
/** @type {import('../../../server/models/Library')} */
let library
/** @type {import('../../../server/models/LibraryItem')} */
let libraryItem
/** @type {import('../../../server/models/Book')} */
let book
/** @type {LibraryScan} */
let libraryScan
const librarySettings = { audiobooksOnly: true }
beforeEach(async () => {
global.ServerSettings = {}
Database.sequelize = new Sequelize({ dialect: 'sqlite', storage: ':memory:', logging: false })
Database.sequelize.uppercaseFirst = (str) => (str ? `${str[0].toUpperCase()}${str.substr(1)}` : '')
await Database.buildModels()
sinon.stub(Logger, 'info')
sinon.stub(Logger, 'debug')
sinon.stub(Logger, 'warn')
sinon.stub(Logger, 'error')
library = await Database.libraryModel.create({ name: 'Test Library', mediaType: 'book' })
const libraryFolder = await Database.libraryFolderModel.create({ path: '/test', libraryId: library.id })
book = await Database.bookModel.create({ title: 'Test Book', audioFiles: [], tags: [], narrators: [], genres: [], chapters: [], coverPath: '/cover.jpg' })
libraryItem = await Database.libraryItemModel.create({
libraryFiles: [],
mediaId: book.id,
mediaType: 'book',
libraryId: library.id,
libraryFolderId: libraryFolder.id
})
libraryItem.isMissing = true
libraryItem.isFile = true
// Stale associations: the rows exist in the database, but this scan's loaded book does not see them
sinon.stub(libraryItem, 'getMedia').callsFake(async () => {
book.authors = []
book.series = []
book.audioFiles = []
book.coverPath = '/cover.jpg'
return book
})
libraryScan = new LibraryScan()
libraryScan.library = library
})
afterEach(async () => {
sinon.restore()
await Database.sequelize.sync({ force: true })
})
function scanData() {
return {
hasAudioFileChanges: false,
audioLibraryFiles: [],
imageLibraryFiles: [],
imageLibraryFilesRemoved: [],
imageLibraryFilesModified: [],
ebookLibraryFiles: [],
ebookLibraryFilesModified: [],
libraryId: library.id
}
}
it('does not throw when the bookSeries link already exists', async () => {
const series = await Database.seriesModel.create({ name: 'Crescent City', libraryId: library.id })
await Database.bookSeriesModel.create({ bookId: book.id, seriesId: series.id, sequence: '3' })
sinon.stub(BookScanner, 'getBookMetadataFromScanData').resolves({
title: 'Test Book',
authors: [],
series: [{ name: 'Crescent City', sequence: '3' }]
})
const result = await BookScanner.rescanExistingBookLibraryItem(libraryItem, scanData(), librarySettings, libraryScan)
expect(result.wasUpdated).to.be.true
const links = await Database.bookSeriesModel.findAll({ where: { bookId: book.id, seriesId: series.id } })
expect(links).to.have.lengthOf(1)
})
it('reuses an existing series when create hits the name unique constraint', async () => {
const series = await Database.seriesModel.create({ name: 'Crescent City', libraryId: library.id })
sinon.stub(Database, 'getSeriesIdByName').resolves(null)
sinon.stub(BookScanner, 'getBookMetadataFromScanData').resolves({
title: 'Test Book',
authors: [],
series: [{ name: 'Crescent City', sequence: '3' }]
})
await BookScanner.rescanExistingBookLibraryItem(libraryItem, scanData(), librarySettings, libraryScan)
const seriesRows = await Database.seriesModel.findAll({ where: { libraryId: library.id, name: 'Crescent City' } })
expect(seriesRows).to.have.lengthOf(1)
const links = await Database.bookSeriesModel.findAll({ where: { bookId: book.id, seriesId: series.id } })
expect(links).to.have.lengthOf(1)
})
it('does not throw when the bookAuthor link already exists', async () => {
const author = await Database.authorModel.create({ name: 'Sarah Maas', libraryId: library.id })
await Database.bookAuthorModel.create({ bookId: book.id, authorId: author.id })
sinon.stub(BookScanner, 'getBookMetadataFromScanData').resolves({
title: 'Test Book',
authors: ['Sarah Maas'],
series: []
})
await BookScanner.rescanExistingBookLibraryItem(libraryItem, scanData(), librarySettings, libraryScan)
const links = await Database.bookAuthorModel.findAll({ where: { bookId: book.id, authorId: author.id } })
expect(links).to.have.lengthOf(1)
})
})
@@ -0,0 +1,221 @@
const { expect } = require('chai')
const express = require('express')
const fs = require('fs')
const http = require('http')
const os = require('os')
const path = require('path')
const sinon = require('sinon')
const Logger = require('../../../server/Logger')
const { fileUploadParser, requireUploadPermission, sweepUploadTempFiles } = require('../../../server/utils/fileUploadMiddleware')
function listTmpFiles(dir) {
if (!fs.existsSync(dir)) return []
return fs.readdirSync(dir).filter((name) => name.startsWith('tmp-'))
}
function sendMultipart(port, urlPath, extraHeaders = {}) {
return new Promise((resolve, reject) => {
const boundary = '----AbsUploadTestBoundary'
const payload = [
`--${boundary}`,
'Content-Disposition: form-data; name="f"; filename="probe.txt"',
'Content-Type: text/plain',
'',
'probe',
`--${boundary}--`,
''
].join('\r\n')
const req = http.request(
{
hostname: '127.0.0.1',
port,
path: urlPath,
method: 'POST',
headers: {
'Content-Type': `multipart/form-data; boundary=${boundary}`,
'Content-Length': Buffer.byteLength(payload),
...extraHeaders
}
},
(res) => {
res.resume()
res.on('end', () => resolve(res.statusCode))
}
)
req.on('error', reject)
req.write(payload)
req.end()
})
}
function listen(app) {
const server = http.createServer(app)
return new Promise((resolve) => {
server.listen(0, '127.0.0.1', () => {
resolve({ server, port: server.address().port })
})
})
}
function closeServer(server) {
return new Promise((resolve, reject) => {
server.close((err) => (err ? reject(err) : resolve()))
})
}
async function waitFor(predicate, timeoutMs = 2000) {
const started = Date.now()
while (Date.now() - started < timeoutMs) {
if (predicate()) return
await new Promise((resolve) => setTimeout(resolve, 25))
}
throw new Error('Timed out waiting for condition')
}
describe('fileUploadMiddleware', () => {
let metadataDir
let tmpDir
let previousMetadataPath
beforeEach(() => {
previousMetadataPath = global.MetadataPath
metadataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'abs-upload-meta-'))
tmpDir = path.join(metadataDir, 'tmp')
global.MetadataPath = metadataDir
sinon.stub(Logger, 'info')
sinon.stub(Logger, 'warn')
})
afterEach(() => {
sinon.restore()
global.MetadataPath = previousMetadataPath
fs.rmSync(metadataDir, { recursive: true, force: true })
})
describe('requireUploadPermission', () => {
it('returns 403 when the user cannot upload', () => {
const req = { user: { canUpload: false, username: 'guest' } }
const res = { sendStatus: sinon.stub() }
const next = sinon.stub()
requireUploadPermission(req, res, next)
expect(res.sendStatus.calledOnceWithExactly(403)).to.equal(true)
expect(next.called).to.equal(false)
})
it('calls next when the user can upload', () => {
const req = { user: { canUpload: true, username: 'uploader' } }
const res = { sendStatus: sinon.stub() }
const next = sinon.stub()
requireUploadPermission(req, res, next)
expect(next.calledOnce).to.equal(true)
expect(res.sendStatus.called).to.equal(false)
})
})
describe('sweepUploadTempFiles', () => {
it('removes leftover tmp-* files and leaves other names', async () => {
fs.mkdirSync(tmpDir, { recursive: true })
fs.writeFileSync(path.join(tmpDir, 'tmp-1-123'), 'stale')
fs.writeFileSync(path.join(tmpDir, 'tmp-2-456'), 'stale')
fs.writeFileSync(path.join(tmpDir, 'comic-extract-keep'), 'keep')
fs.mkdirSync(path.join(tmpDir, 'tmp-not-a-file'))
await sweepUploadTempFiles()
expect(fs.existsSync(path.join(tmpDir, 'tmp-1-123'))).to.equal(false)
expect(fs.existsSync(path.join(tmpDir, 'tmp-2-456'))).to.equal(false)
expect(fs.existsSync(path.join(tmpDir, 'comic-extract-keep'))).to.equal(true)
expect(fs.existsSync(path.join(tmpDir, 'tmp-not-a-file'))).to.equal(true)
})
it('returns when the temp directory does not exist', async () => {
await sweepUploadTempFiles()
expect(fs.existsSync(tmpDir)).to.equal(false)
})
})
describe('upload parser mounting', () => {
it('does not write tmp-* files for a route without the parser', async () => {
const app = express()
app.use((req, res) => res.sendStatus(404))
const { server, port } = await listen(app)
try {
const status = await sendMultipart(port, '/not-a-route')
expect(status).to.equal(404)
expect(listTmpFiles(tmpDir)).to.deep.equal([])
} finally {
await closeServer(server)
}
})
it('does not write tmp-* files when permission is rejected before parsing', async () => {
const app = express()
app.use((req, res, next) => {
req.user = { canUpload: false, username: 'guest' }
next()
})
app.post('/upload', requireUploadPermission, fileUploadParser, (req, res) => {
res.sendStatus(200)
})
const { server, port } = await listen(app)
try {
const status = await sendMultipart(port, '/upload')
expect(status).to.equal(403)
expect(listTmpFiles(tmpDir)).to.deep.equal([])
} finally {
await closeServer(server)
}
})
it('removes the temp file when the handler rejects without moving it', async () => {
const app = express()
app.use((req, res, next) => {
req.user = { canUpload: true, username: 'uploader' }
next()
})
app.post('/upload', requireUploadPermission, fileUploadParser, (req, res) => {
res.status(400).send('Invalid request no file or url')
})
const { server, port } = await listen(app)
try {
const status = await sendMultipart(port, '/upload')
expect(status).to.equal(400)
await waitFor(() => listTmpFiles(tmpDir).length === 0)
} finally {
await closeServer(server)
}
})
it('keeps a moved file and leaves no tmp-* leftover', async () => {
const dest = path.join(metadataDir, 'moved.txt')
const app = express()
app.use((req, res, next) => {
req.user = { canUpload: true, username: 'uploader' }
next()
})
app.post('/upload', requireUploadPermission, fileUploadParser, async (req, res) => {
await req.files.f.mv(dest)
res.sendStatus(200)
})
const { server, port } = await listen(app)
try {
const status = await sendMultipart(port, '/upload')
expect(status).to.equal(200)
expect(fs.readFileSync(dest, 'utf8')).to.equal('probe')
await waitFor(() => listTmpFiles(tmpDir).length === 0)
} finally {
await closeServer(server)
}
})
})
})
@@ -1,6 +1,6 @@
const chai = require('chai')
const expect = chai.expect
const { parse, nameToLastFirst } = require('../../../../server/utils/parsers/parseNameString')
const { parse, nameToLastFirst, dedupeNames } = require('../../../../server/utils/parsers/parseNameString')
describe('parseNameString', () => {
describe('parse', () => {
@@ -80,6 +80,20 @@ describe('parseNameString', () => {
})
})
describe('dedupeNames', () => {
it('trims, drops blanks, and keeps the first spelling of an exact repeat', () => {
expect(dedupeNames([' Stephen King ', 'Stephen King', '', ' ', 'stephen king'])).to.deep.equal(['Stephen King', 'stephen king'])
})
it('preserves order and ignores non-strings', () => {
expect(dedupeNames(['Amy Yao', null, 'Amy Yao', 3, 'Raymond Chandler'])).to.deep.equal(['Amy Yao', 'Raymond Chandler'])
})
it('returns an empty array for non-arrays', () => {
expect(dedupeNames(null)).to.deep.equal([])
})
})
describe('nameToLastFirst', () => {
it('converts First Last to Last, First format', () => {
const result = nameToLastFirst('John Smith')