mirror of
https://github.com/advplyr/audiobookshelf.git
synced 2026-08-29 14:47:20 +02:00
[PR #3952] [MERGED] Improve book library page query performance on title, titleIgnorePrefix, and addedAt sort orders. #4129
Closed
opened 2026-04-25 00:18:26 +02:00 by adam
·
0 comments
No Branch/Tag Specified
master
auth_sessions_enhancements
account_sessions_table
logout_all_devices
pw_change_invalidates_sessions
book_tags_genres_dedupe
episode_download_fallback
Issue-4540-SortBy-StartedDate-and-FinishedDate
episode_meta_tagging
fix_authorize_race_condition
redirect_transcode_requests
progress_updated_sort
fix_ereader_socket_event
fix_change_empty_root_password
fix_podcast_session_track_index
fix_set_token
session_modal_user
localize_durations
fix_oidc_create_user
jwt_auth_refactor
fix_scanner_deleting_single_file_books
fix_mediaprogress_updatedat_2
experimental_next_client
podcast_episode_duration
episode-timestamps-clickable
book_author_secondary_sort_title
podcast_useragents
pathexists_user_access
fix_pathexists_join
book_author_secondary_sort
clean_duplicate_mediaprogress
sanitize_html_description
trix_prevent_attachments
check_path_api_fix
fix_mediaprogress_updatedat
increase_express_json_limit
fix_dockerfile_nunicode
search_episodes
audiobook_tools_update
episode_secondary_sorts
hls_stream_url_update
new_session_track_endpoint
audiobook_tools_enhancements
watcher_rescans_update
player_track_tooltip
fix_exclude_prefixes_crash
socket_item_events
fix_podcast_episode_scanner_promise
new_stats_controller
count_cache_for_userpermissions
parsing-opf-v3
validate_migration_files
fix-quick-match-all-crash
fix-chapter-end-sleep-timer
stringify_sequelize_query
remove-col-ambiguity
fix_next_prev_edit_description
details_trim_whitespace
fix_content_url_basepath
fix_logger_fatal
progress_bar_visibility
batch-edit-populate-map-details
feed_generator_updates
bookmark-modal-updates
migrate-library-item-in-scanner
migrate-new-library-items
migrate-podcasts-new-library-item-2
migrate-podcasts-new-library-item
fix-remove-episode-from-playlist
playback-session-use-new-library-item
refactor-library-item
fix-heatmap-caption
feed-episodes-upsert
share-media-player-media-session-api
remove-old-playlist
remove_old_collection_object
plugin-implementation-demo
feed_migration
refactor-feeds-from-item
fix_remove_authors_no_books
v2.17.3-fk-constraints-migration
migrations-first-upgrade
sqlite_2
feature/nuxt-target-server
waveform
sqlite
playlists
video
v2.36.0
v2.35.1
v2.35.0
v2.34.0
v2.33.2
v2.33.1
v2.33.0
v2.32.1
v2.32.0
v2.31.0
v2.30.0
v2.29.0
v2.28.0
v2.27.0
v2.26.3
v2.26.2
v2.26.1
v2.26.0
v2.25.1
v2.25.0
v2.24.0
v2.23.0
v2.22.0
v2.21.0
v2.20.0
v2.19.5
v2.19.4
v2.19.3
v2.19.2
v2.19.1
v2.19.0
v2.18.1
v2.18.0
v2.17.7
v2.17.6
v2.17.5
v2.17.4
v2.17.3
v2.17.2
v2.17.1
v2.17.0
v2.16.2
v2.16.1
v2.16.0
v2.15.1
v2.15.0
v2.14.0
v2.13.4
v2.13.3
v2.13.2
v2.13.1
v2.13.0
v2.12.3
v2.12.2
v2.12.1
v2.12.0
v2.11.0
v2.10.1
v2.10.0
v2.9.0
v2.8.1
v2.8.0
v2.7.2
v2.7.1
v2.7.0
v2.6.0
v2.5.0
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.23
v2.2.22
v2.2.21
v2.2.20
v2.2.19
v2.2.18
v2.2.17
v2.2.16
v2.2.15
v2.2.14
v2.2.13
v2.2.12
v2.2.11
v2.2.10
v2.2.9
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.5
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.24
v2.0.23
v2.0.22
v2.0.21
v2.0.20
v2.0.19
v2.0.18
v2.0.17
v2.0.16
v2.0.15
v2.0.14
v2.0.13
v2.0.12
v2.0.11
v2.0.10
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v1.7.2
v1.7.1
v1.7.0
v1.6.0
v1.5.5
v1.5.0
v1.4.11
v1.4.9
v1.4.7
v1.4.6
v1.4.4
v1.4.2
v1.4.0
v1.4.1
v1.3.4
v1.3.3
v1.3.1
v1.2.8
v1.2.6
v1.2.5
v1.2.4
v1.2.1
v1.1.15
v1.1.14
v1.1.13
v1.1.12
v1.1.11
v1.1.10
v1.1.9
v1.1.8
v1.0.0
0.9.61-beta.0
0.9.61-beta
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
Mirrored from GitHub Pull Request
No labels
pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
adam (Adam Melkus)
Clear assignees
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/audiobookshelf#4129
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/advplyr/audiobookshelf/pull/3952
Author: @mikiher
Created: 2/8/2025
Status: ✅ Merged
Merged: 2/11/2025
Merged by: @advplyr
Base:
master← Head:query-performance📝 Commits (6)
52bb286Add a profile utility functiona131432Improve page load queries on title, titleIgnorePrefix, and addedAt sort order3d08a35Add index on (libraryId, mediaType, createdAt)9a26119Update server/models/Book.jsef45f84Update upwards migration to be idempotent4cc300dUpdate changelog with v2.19.1 migration📊 Changes
8 files changed (+416 additions, -9 deletions)
View changed files
📝
server/migrations/changelog.md(+1 -0)➕
server/migrations/v2.19.1-copy-title-to-library-items.js(+164 -0)📝
server/models/Book.js(+9 -0)📝
server/models/LibraryItem.js(+16 -1)📝
server/scanner/BookScanner.js(+2 -0)➕
server/utils/profiler.js(+41 -0)📝
server/utils/queries/libraryItemsBookFilters.js(+35 -8)➕
test/server/migrations/v2.19.1-copy-title-to-library-items.test.js(+148 -0)📄 Description
Brief summary
Significantly improves book library Sequelize page queries for the following sort orders:
Which issue is fixed?
This partially fixes #2073 (resolving the book library load times, but not the podcast library load times)
In-depth Description
After digging more into the details of the issues people were complaining about in #2073 and doing additional performance analysis in Sequelize query to bring the page here, I made the following observations:
1 is by far the most serious problem, and also causes significant degradation in query performance as the offset becomes larger.
When the main query is sorting by title:
it's evident in the query plan that the query engine cannot use the existing book.title index, and needs to build a temporary tree for sorting.
Even when you remove the
feedstable join from the query:The query plan still doesn't make use of the book.title index:
The significant boost in performance can come only if the title column is put in the libraryItems table, and an index on
(libraryId, mediaType, title)is built. This way, filtering and sorting happens at the same time, and the index can be traveresed very quickly to reach the required offset without needing to look at the tables themselves.So with a query like this:
We get the following query plan:
Which is optimal! (or, to be more precise, optimal given the current architecture)
Resolution
The following changes were made:
titleandtitleIgnorePrefixcolumns were added tolibraryItemsbookschange.libraryItems:(libraryId, mediaType, title)(libraryId, mediaType, titleIgnorePrefix)(libraryId, mediaType, createdAt)findAllis called instead offindAndCountAllHow have you tested this?
I tested on loading 72 consecutive page of 35 books each, on each of the above sorting orders, on an ABS docker container running on a Synology 920+ NAS (I wanted to test on a hardware that was much weaker than my dev machine).
Results
All measurements are in ms.
Summary
Overall, we see a 94-95% drop (!) in mean and median query time.
Standard deviaion also reduces drastically from ~400 to ~10.
Note how the steady rise in query time (as the the requested offset grows) which is quite visible before, is not noticable after.
Sorting by title - before:
Sorting by title - after:
Sorting by titleIgnorePrefix - before:
Sorting by titleIgnorePrefix - after:
Sorting by addedAt - before:
Sorting by addedAt - after:
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.