mirror of
https://github.com/advplyr/audiobookshelf.git
synced 2026-05-30 23:40:40 +02:00
[Bug]: FATAL: [Server] Unhandled rejection: SequelizeUniqueConstraintError: Validation error #2081
Closed
opened 2026-04-25 00:03:15 +02:00 by adam
·
30 comments
No Branch/Tag Specified
master
book_tags_genres_dedupe
episode_download_fallback
Issue-4540-SortBy-StartedDate-and-FinishedDate
episode_meta_tagging
fix_authorize_race_condition
redirect_transcode_requests
progress_updated_sort
fix_ereader_socket_event
fix_change_empty_root_password
fix_podcast_session_track_index
fix_set_token
session_modal_user
localize_durations
fix_oidc_create_user
jwt_auth_refactor
fix_scanner_deleting_single_file_books
fix_mediaprogress_updatedat_2
experimental_next_client
podcast_episode_duration
episode-timestamps-clickable
book_author_secondary_sort_title
podcast_useragents
pathexists_user_access
fix_pathexists_join
book_author_secondary_sort
clean_duplicate_mediaprogress
sanitize_html_description
trix_prevent_attachments
check_path_api_fix
fix_mediaprogress_updatedat
increase_express_json_limit
fix_dockerfile_nunicode
search_episodes
audiobook_tools_update
episode_secondary_sorts
hls_stream_url_update
new_session_track_endpoint
audiobook_tools_enhancements
watcher_rescans_update
player_track_tooltip
fix_exclude_prefixes_crash
socket_item_events
fix_podcast_episode_scanner_promise
new_stats_controller
count_cache_for_userpermissions
parsing-opf-v3
validate_migration_files
fix-quick-match-all-crash
fix-chapter-end-sleep-timer
stringify_sequelize_query
remove-col-ambiguity
fix_next_prev_edit_description
details_trim_whitespace
fix_content_url_basepath
fix_logger_fatal
progress_bar_visibility
batch-edit-populate-map-details
feed_generator_updates
bookmark-modal-updates
migrate-library-item-in-scanner
migrate-new-library-items
migrate-podcasts-new-library-item-2
migrate-podcasts-new-library-item
fix-remove-episode-from-playlist
playback-session-use-new-library-item
refactor-library-item
fix-heatmap-caption
feed-episodes-upsert
share-media-player-media-session-api
remove-old-playlist
remove_old_collection_object
plugin-implementation-demo
feed_migration
refactor-feeds-from-item
fix_remove_authors_no_books
v2.17.3-fk-constraints-migration
migrations-first-upgrade
sqlite_2
feature/nuxt-target-server
waveform
sqlite
playlists
video
v2.35.1
v2.35.0
v2.34.0
v2.33.2
v2.33.1
v2.33.0
v2.32.1
v2.32.0
v2.31.0
v2.30.0
v2.29.0
v2.28.0
v2.27.0
v2.26.3
v2.26.2
v2.26.1
v2.26.0
v2.25.1
v2.25.0
v2.24.0
v2.23.0
v2.22.0
v2.21.0
v2.20.0
v2.19.5
v2.19.4
v2.19.3
v2.19.2
v2.19.1
v2.19.0
v2.18.1
v2.18.0
v2.17.7
v2.17.6
v2.17.5
v2.17.4
v2.17.3
v2.17.2
v2.17.1
v2.17.0
v2.16.2
v2.16.1
v2.16.0
v2.15.1
v2.15.0
v2.14.0
v2.13.4
v2.13.3
v2.13.2
v2.13.1
v2.13.0
v2.12.3
v2.12.2
v2.12.1
v2.12.0
v2.11.0
v2.10.1
v2.10.0
v2.9.0
v2.8.1
v2.8.0
v2.7.2
v2.7.1
v2.7.0
v2.6.0
v2.5.0
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.23
v2.2.22
v2.2.21
v2.2.20
v2.2.19
v2.2.18
v2.2.17
v2.2.16
v2.2.15
v2.2.14
v2.2.13
v2.2.12
v2.2.11
v2.2.10
v2.2.9
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.5
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.24
v2.0.23
v2.0.22
v2.0.21
v2.0.20
v2.0.19
v2.0.18
v2.0.17
v2.0.16
v2.0.15
v2.0.14
v2.0.13
v2.0.12
v2.0.11
v2.0.10
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v1.7.2
v1.7.1
v1.7.0
v1.6.0
v1.5.5
v1.5.0
v1.4.11
v1.4.9
v1.4.7
v1.4.6
v1.4.4
v1.4.2
v1.4.0
v1.4.1
v1.3.4
v1.3.3
v1.3.1
v1.2.8
v1.2.6
v1.2.5
v1.2.4
v1.2.1
v1.1.15
v1.1.14
v1.1.13
v1.1.12
v1.1.11
v1.1.10
v1.1.9
v1.1.8
v1.0.0
0.9.61-beta.0
0.9.61-beta
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
Mirrored from GitHub Pull Request
No Label
bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
adam (Adam Melkus)
Clear assignees
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/audiobookshelf#2081
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hardwareadictos on GitHub (Jul 5, 2024).
What happened?
When adding a large amount of ebooks to a library (like 30000) container crashes with error ERROR: [Watcher] Error: ENOSPC: System limit for number of file watchers reached on library scan
What did you expect to happen?
Scaning taking long but not crashing
Steps to reproduce the issue
Just adding large amount of ebooks and scanning
Audiobookshelf version
2.10.1
How are you running audiobookshelf?
Docker
What OS is your Audiobookshelf server hosted from?
Linux
If the issue is being seen in the UI, what browsers are you seeing the problem on?
None
Logs
Additional Notes
No response
@advplyr commented on GitHub (Jul 5, 2024):
Is it crashing the container or just showing an error?
You need to increase the number of file watchers on your system or you can disable the watcher.
@hardwareadictos commented on GitHub (Jul 6, 2024):
It just hungs and reports 502 error on browser.
Just applied this solution, which consists on a little kernel parameter: https://stackoverflow.com/questions/55763428/react-native-error-enospc-system-limit-for-number-of-file-watchers-reached
Will check during a week and report back.
@hardwareadictos commented on GitHub (Jul 6, 2024):
Well, no more limit errors, but now i can reproduce this on every scan on this library:
And this does't hung the container, this directly makes it crash
@advplyr commented on GitHub (Jul 8, 2024):
Can you share your docker config and if you are using remote file systems for the mounts
@hardwareadictos commented on GitHub (Jul 8, 2024):
Sure!
And yes, audiobooks, podcasts and books are stored on a remote NFS. It has full permissions for the UID/GID provided on the docker-compose file and over a 1G network.
Do you need any more info?
@nichwall commented on GitHub (Jul 8, 2024):
Are there supposed to be 2 slashes in the audio book mount? Not sure if that's a typo.
@hardwareadictos commented on GitHub (Jul 8, 2024):
Yes its a typo, just edited the real mountpoints before posting them here, edited and corrected it-
@mikiher commented on GitHub (Jul 10, 2024):
From what I read above, it looks like you have watcher enabled and are running (automated?) library scans.
First of all, since your libraries are NFS mounted, I suspect that the watcher is not working for you (which is likely why you need to run library scans). Watcher only works reliably when watched files and directories are on the local file system - when they're mounted on a remote NFS, watcher will appear to be running but will not get notified of file change events.
Therefore, I'd advise disabling watcher (uncheck Settings -> Enable watcher).
Once you do that, please let us know if your library scans are still crashing.
@hardwareadictos commented on GitHub (Jul 10, 2024):
Good morning!!
Will try and report soon.
Something to take into account: I have all my libraries using NFS shared storage (podcasts and audiobooks) with watchers enabled working fine. I also have a Jellyfin instance on the same server working with something similar to watchers (if not the same) enabled on my media directories with NFS shared storage also working fine for more than 4 years. The only difference between my other libraries is that ebook ones have more than 130000 files (which maybe is a lot for a sqlite database).
Let me test and report back. Thanks a lot for the support :)
@hardwareadictos commented on GitHub (Jul 10, 2024):
Well, same issue as before, watchers disabled for ebooks library and scanned manually (logs on debug mode):
It scans like 135 books of +- 130000 and it just crashes on this constrain.
@mikiher commented on GitHub (Jul 10, 2024):
If your watchers are working fine, why would you need an automated library scan? the Audiobookshelf watcher is supposed to detect any file changes in any of the directories of your library, and add books automatically when they're detectedm without requiring a library scan. Is it doing that?
@hardwareadictos commented on GitHub (Jul 10, 2024):
Maybe i didn't explain myself correctly: Watchers are disabled on that library scan, automated scans are also disabled, i'm just triggering a manual scan. Scan is what's crashing.
The thing ends with that constrain error and reboots de container with an exit 0
@mikiher commented on GitHub (Jul 10, 2024):
OK, undetstood. Thanks for turning log level to Debug. I'm looking at the logs.
@mikiher commented on GitHub (Jul 10, 2024):
As with the other bug, I'd appreciate if you attach the whole server side log (including debug printings from the whole last scan until the crash). This kind of failure might hint at some race condition accessing the database, so I want to understand all that happened (and all that didn't happen) prior to the failure.
@hardwareadictos commented on GitHub (Jul 10, 2024):
No problem! Will grab them in an hour.
@mikiher commented on GitHub (Jul 10, 2024):
Just the last day (which contains the last scan you ran manually).
On Wed, Jul 10, 2024 at 8:04 PM hardwareadictos @.***>
wrote:
@hardwareadictos commented on GitHub (Jul 10, 2024):
Here you have it:
2024-07-10.txt
thanks! :D
@mikiher commented on GitHub (Jul 11, 2024):
One more question: has the library "Libros" been successfully scanned before, or were you never able to fully scan it?
@hardwareadictos commented on GitHub (Jul 11, 2024):
Never as it's really big, other libraries like podcasts and Audiolibros are being scanned fine.
I even deleted and recreated Libros library to avoid issues creating it the first time without success
@mikiher commented on GitHub (Jul 11, 2024):
OK, I believe I understand what is happening here.
I think this is a specific issue with the book
/libros/Yvonne Whittal/Cadenas de oro (65639)/Cadenas de oro - Yvonne Whittal.epubthat crashes Audiobookshelf - I think it has a duplicate series metadata inside its opf file (somehwhat similar to #2650)Can you possibly attach this specific epub here so I can examine it?
@mikiher commented on GitHub (Jul 11, 2024):
Yeah, I was able to repro the crash by duplicating the series metadata lines in a random epub's opf file:
Then, scanning this epub I get the same crash:
@hardwareadictos commented on GitHub (Jul 12, 2024):
Nice found!! Let me check it later, I will exclude this ebook from the library folder later and will reescan to confirm. Thanks for the effort.
@hardwareadictos commented on GitHub (Jul 12, 2024):
Okay i found the duplicates also and i deleted them with calibre editor:
And boila, now it's scanning beyond the book n135 which was the culprit. Now i have another 129750 books to scan and "correct" if it founds another one broken.
From my ignorance: is it posible on the app code side to skip repeated metadata tags and only keep the first one found? Don't know if it's posible, i would say yes because my ereaders can read this book with those repeated tags, but i don't know if that would compromise the app code :)
Edit: I found another one (which was posible):
@mikiher commented on GitHub (Jul 12, 2024):
Of course, something like this should never cause a crash. I'm working on a fix.
@hardwareadictos commented on GitHub (Jul 12, 2024):
Nice!! Then im goint to stop "fixing" the books jeje.
Let me know when you find something, will gladly test :)
@mikiher commented on GitHub (Jul 12, 2024):
The fix #3152 is in review.
@hardwareadictos commented on GitHub (Jul 12, 2024):
Nice!! Thanks a lot for the support, quick and effective, loving the app, waiting anxiously for the next release 👏👏👏
@alxgarci commented on GitHub (Aug 3, 2024):
I can confirm i had the same issue and using the 'edge' (last) docker build this issue is now solved! Ty for the fix!
@github-actions[bot] commented on GitHub (Aug 5, 2024):
Fixed in v2.12.0.
@hardwareadictos commented on GitHub (Aug 9, 2024):
I can confirm that this is working fine now!! Now i found another issue regarding big ebook libraries management, will open another issue