starred/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2026-05-30 23:40:40 +02:00
Code Issues 1.0k Packages Projects Releases 10 Wiki Activity

BUG: Large library timing out #92

New Issue
Closed
opened 2026-04-24 22:58:11 +02:00 by adam · 15 comments
adam commented 2026-04-24 22:58:11 +02:00
Owner
Copy Link
Copy Source

Originally created by @itschrisonline on GitHub (Nov 15, 2021).

Hi All
When ever trying do anything including play and audiobook (after sometime), browse or search, I get
[2021-11-15T19:57:14.835Z] INFO: === Starting Server ===
[2021-11-15T19:57:14.835Z] INFO: [Server] Init v1.6.18
[2021-11-15T19:57:14.837Z] INFO: Removing Orphan Stream kvz34ibn
[2021-11-15T19:57:15.482Z] INFO: [DB] 0 Collections Loaded
[2021-11-15T19:57:15.542Z] INFO: [DB] 2 Users Loaded
[2021-11-15T19:57:15.630Z] INFO: [DB] 1 Libraries Loaded
[2021-11-15T19:57:17.053Z] INFO: [DB] 3993 Audiobooks Loaded
[2021-11-15T19:57:17.053Z] INFO: [Db] Root user exists, pw: true
[2021-11-15T19:57:17.056Z] INFO: [BackupManager] 0 Backups Found
[2021-11-15T19:57:17.056Z] INFO: [BackupManager] Auto Backups are disabled
[2021-11-15T19:57:17.075Z] INFO: [LogManager] Init current daily log filename: 2021-11-16.txt
[2021-11-15T19:57:17.214Z] INFO: [Watcher] Initializing watcher for "Main"..
[2021-11-15T19:57:17.246Z] INFO: Running on http://0.0.0.0:80
[2021-11-15T19:57:21.661Z] INFO: [SOCKET] Socket Connected TwcWI_tbGNyMRn_XAAAB
[2021-11-15T19:57:39.837Z] INFO: [Server] Socket TwcWI_tbGNyMRn_XAAAB disconnected from client "root" after 18176ms
[2021-11-15T19:57:41.019Z] INFO: [SOCKET] Socket Connected CsHLb83lzraXGX4cAAAD

[2021-11-15T20:02:14.208Z] INFO: [Server] Socket CsHLb83lzraXGX4cAAAD disconnected from client "root" after 273189ms
[2021-11-15T20:02:54.939Z] INFO: [SOCKET] Socket Connected x44Kx06Cgtlujm5NAAAF
[2021-11-15T20:02:58.433Z] INFO: [SOCKET] Socket Connected WPFANlX88wvsH0SoAAAH
[2021-11-15T20:03:00.229Z] INFO: User has stream open already kvz34ibn
[2021-11-15T20:03:00.230Z] ERROR: Invalid user stream id kvz34ibn

[2021-11-15T20:03:18.954Z] INFO: [Server] Socket x44Kx06Cgtlujm5NAAAF disconnected from client "root" after 24015ms
[2021-11-15T20:04:36.027Z] INFO: [Server] Socket WPFANlX88wvsH0SoAAAH disconnected from client "chris" after 97594ms
[2021-11-15T20:04:36.027Z] INFO: [Server] Socket WPFANlX88wvsH0SoAAAH disconnected from client "chris" after 97594ms
[2021-11-15T20:06:25.875Z] INFO: [SOCKET] Socket Connected UyWDMU6yWtWWkpbWAAAJ

[2021-11-15T20:07:37.338Z] INFO: [Server] Socket UyWDMU6yWtWWkpbWAAAJ disconnected from client "chris" after 71463ms
[2021-11-15T20:08:04.411Z] INFO: [SOCKET] Socket Connected 86NVwqv5y_6eZtPVAAAL

[2021-11-15T20:09:54.069Z] ERROR: Api called without a token /s/book/kvytw2xn/Blindside%20FBI%20Thriller%20

[2021-11-15T20:09:54.069Z] ERROR: Api called without a token /s/book/kvytw2xn/Blindside%20FBI%20Thriller%20

[2021-11-15T20:09:55.734Z] ERROR: Api called without a token /s/book/kvytvdpw/Eleventh%20Hour%20FBI%20Thriller%20

[2021-11-15T20:10:00.008Z] ERROR: Api called without a token /s/book/kvytt0ju/The%20Target%20FBI%20Thriller%20

[2021-11-15T20:10:00.008Z] ERROR: Api called without a token /s/book/kvytt0ju/The%20Target%20FBI%20Thriller%20

[2021-11-15T20:10:09.960Z] ERROR: Api called without a token /s/book/kvytu35b/Riptide%20FBI%20Thriller%20

[2021-11-15T20:10:09.982Z] ERROR: Api called without a token /s/book/kvytsacf/The%20Maze%20FBI%20Thriller%20

[2021-11-15T20:10:15.732Z] INFO: [Server] Socket 86NVwqv5y_6eZtPVAAAL disconnected from client "chris" after 131321ms
[2021-11-15T20:11:15.669Z] INFO: [SOCKET] Socket Connected n1adSY3wl7DGzH4PAAAN
[2021-11-15T20:11:29.953Z] INFO: Recieved ping
[2021-11-15T20:11:30.922Z] INFO: [SOCKET] Socket Connected UaXiXrNGEfHf1ZirAAAP

Which is making the server/app unusable

Originally created by @itschrisonline on GitHub (Nov 15, 2021). Hi All When ever trying do anything including play and audiobook (after sometime), browse or search, I get [2021-11-15T19:57:14.835Z] INFO: === Starting Server === [2021-11-15T19:57:14.835Z] INFO: [Server] Init v1.6.18 [2021-11-15T19:57:14.837Z] INFO: Removing Orphan Stream kvz34ibn [2021-11-15T19:57:15.482Z] INFO: [DB] 0 Collections Loaded [2021-11-15T19:57:15.542Z] INFO: [DB] 2 Users Loaded [2021-11-15T19:57:15.630Z] INFO: [DB] 1 Libraries Loaded [2021-11-15T19:57:17.053Z] INFO: [DB] 3993 Audiobooks Loaded [2021-11-15T19:57:17.053Z] INFO: [Db] Root user exists, pw: true [2021-11-15T19:57:17.056Z] INFO: [BackupManager] 0 Backups Found [2021-11-15T19:57:17.056Z] INFO: [BackupManager] Auto Backups are disabled [2021-11-15T19:57:17.075Z] INFO: [LogManager] Init current daily log filename: 2021-11-16.txt [2021-11-15T19:57:17.214Z] INFO: [Watcher] Initializing watcher for "Main".. [2021-11-15T19:57:17.246Z] INFO: Running on http://0.0.0.0:80 [2021-11-15T19:57:21.661Z] INFO: [SOCKET] Socket Connected TwcWI_tbGNyMRn_XAAAB [2021-11-15T19:57:39.837Z] INFO: [Server] Socket TwcWI_tbGNyMRn_XAAAB disconnected from client "root" after 18176ms [2021-11-15T19:57:41.019Z] INFO: [SOCKET] Socket Connected CsHLb83lzraXGX4cAAAD [2021-11-15T20:02:14.208Z] INFO: [Server] Socket CsHLb83lzraXGX4cAAAD disconnected from client "root" after 273189ms [2021-11-15T20:02:54.939Z] INFO: [SOCKET] Socket Connected x44Kx06Cgtlujm5NAAAF [2021-11-15T20:02:58.433Z] INFO: [SOCKET] Socket Connected WPFANlX88wvsH0SoAAAH [2021-11-15T20:03:00.229Z] INFO: User has stream open already kvz34ibn [2021-11-15T20:03:00.230Z] ERROR: Invalid user stream id kvz34ibn [2021-11-15T20:03:18.954Z] INFO: [Server] Socket x44Kx06Cgtlujm5NAAAF disconnected from client "root" after 24015ms [2021-11-15T20:04:36.027Z] INFO: [Server] Socket WPFANlX88wvsH0SoAAAH disconnected from client "chris" after 97594ms [2021-11-15T20:04:36.027Z] INFO: [Server] Socket WPFANlX88wvsH0SoAAAH disconnected from client "chris" after 97594ms [2021-11-15T20:06:25.875Z] INFO: [SOCKET] Socket Connected UyWDMU6yWtWWkpbWAAAJ [2021-11-15T20:07:37.338Z] INFO: [Server] Socket UyWDMU6yWtWWkpbWAAAJ disconnected from client "chris" after 71463ms [2021-11-15T20:08:04.411Z] INFO: [SOCKET] Socket Connected 86NVwqv5y_6eZtPVAAAL [2021-11-15T20:09:54.069Z] ERROR: Api called without a token /s/book/kvytw2xn/Blindside%20FBI%20Thriller%20 [2021-11-15T20:09:54.069Z] ERROR: Api called without a token /s/book/kvytw2xn/Blindside%20FBI%20Thriller%20 [2021-11-15T20:09:55.734Z] ERROR: Api called without a token /s/book/kvytvdpw/Eleventh%20Hour%20FBI%20Thriller%20 [2021-11-15T20:10:00.008Z] ERROR: Api called without a token /s/book/kvytt0ju/The%20Target%20FBI%20Thriller%20 [2021-11-15T20:10:00.008Z] ERROR: Api called without a token /s/book/kvytt0ju/The%20Target%20FBI%20Thriller%20 [2021-11-15T20:10:09.960Z] ERROR: Api called without a token /s/book/kvytu35b/Riptide%20FBI%20Thriller%20 [2021-11-15T20:10:09.982Z] ERROR: Api called without a token /s/book/kvytsacf/The%20Maze%20FBI%20Thriller%20 [2021-11-15T20:10:15.732Z] INFO: [Server] Socket 86NVwqv5y_6eZtPVAAAL disconnected from client "chris" after 131321ms [2021-11-15T20:11:15.669Z] INFO: [SOCKET] Socket Connected n1adSY3wl7DGzH4PAAAN [2021-11-15T20:11:29.953Z] INFO: Recieved ping [2021-11-15T20:11:30.922Z] INFO: [SOCKET] Socket Connected UaXiXrNGEfHf1ZirAAAP Which is making the server/app unusable
adam closed this issue 2026-04-24 22:58:11 +02:00
adam commented 2026-04-24 22:58:12 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 15, 2021):

Those "Api called without a token" errors do you know if they were requests from the mobile app?

Can you create a library with a smaller number of books so we can see if this issue is coming from your library size?

@advplyr commented on GitHub (Nov 15, 2021): Those "Api called without a token" errors do you know if they were requests from the mobile app? Can you create a library with a smaller number of books so we can see if this issue is coming from your library size?
adam commented 2026-04-24 22:58:12 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 16, 2021):

I released v1.6.19 that only loads the covers in view, which improved performance for a 1k book library.
I will be testing with a 2k library next and making improvements.

@advplyr commented on GitHub (Nov 16, 2021): I released `v1.6.19` that only loads the covers in view, which improved performance for a 1k book library. I will be testing with a 2k library next and making improvements.
adam commented 2026-04-24 22:58:13 +02:00
Author
Owner
Copy Link
Copy Source

@itschrisonline commented on GitHub (Nov 16, 2021):

Hi Thanks, Yes they were from the mobile app

@itschrisonline commented on GitHub (Nov 16, 2021): Hi Thanks, Yes they were from the mobile app
adam commented 2026-04-24 22:58:13 +02:00
Author
Owner
Copy Link
Copy Source

@seanap commented on GitHub (Nov 22, 2021):

The latest version fixed the Page Unresponsive in Chrome. It still takes a good 15s - 20s to load the library but it's a lot better. My issue now is with adding books. All my books are on an SMB mount, so detecting new file changes can be challenging. In plex I have to manually scan to pick up new books.

Last night in AudiobookShelf I rescanned my library and let it run over night. 12hrs later it was still scanning, but my Issues section was growing with "Audiobook directory is missing!". I think the connection to my files got disconnected from abs for some reason. The only way I found for me to add books is to completely wipe all config data out and start from scratch with a fresh scan on a fresh install.

@seanap commented on GitHub (Nov 22, 2021): The latest version fixed the Page Unresponsive in Chrome. It still takes a good 15s - 20s to load the library but it's a lot better. My issue now is with adding books. All my books are on an SMB mount, so detecting new file changes can be challenging. In plex I have to manually scan to pick up new books. Last night in AudiobookShelf I rescanned my library and let it run over night. 12hrs later it was still scanning, but my Issues section was growing with "Audiobook directory is missing!". I think the connection to my files got disconnected from abs for some reason. The only way I found for me to add books is to completely wipe all config data out and start from scratch with a fresh scan on a fresh install.
adam commented 2026-04-24 22:58:14 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 22, 2021):

Thanks for that update.
Audiobookshelf relies on either the folder path or the folders inode value. My guess is the inode value will be changing in your case, so do you know if the file path was updated at all? Maybe a root folder name changed?

This could be a bug that I'm yet to discover because I think someone posted the same thing recently.

@advplyr commented on GitHub (Nov 22, 2021): Thanks for that update. Audiobookshelf relies on either the folder path or the folders inode value. My guess is the inode value will be changing in your case, so do you know if the file path was updated at all? Maybe a root folder name changed? This could be a bug that I'm yet to discover because I think someone posted the same thing recently.
adam commented 2026-04-24 22:58:14 +02:00
Author
Owner
Copy Link
Copy Source

@seanap commented on GitHub (Nov 22, 2021):

Root folders have not changed, I did add Authors/books to the Audiobook folder though. All other services, including other dockers in the same compose file, have no issues with the SMB storage.

Just as a point of comparison, with the same smb connected audiobook library, Plex takes 20min to do a full scan.

@seanap commented on GitHub (Nov 22, 2021): Root folders have not changed, I did add Authors/books to the Audiobook folder though. All other services, including other dockers in the same compose file, have no issues with the SMB storage. Just as a point of comparison, with the same smb connected audiobook library, Plex takes 20min to do a full scan.
adam commented 2026-04-24 22:58:14 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 22, 2021):

I really appreciate the comparison. I know the scanner is still slow, but 12 hours seems crazy.

Right now the scanner is going through each audio file 1 at a time and probing, this is the time consuming task. I'm going to start on what I will call Scanner v5 soon which will probe audio files in parallel. But still, 12 hours?? I wonder if this is related to the SMB.
When you say Plex took 20 minutes to scan are you talking about a fresh empty Plex library? Audiobookshelf only probes audio files that it hasn't seen before, so subsequent scans should be very fast.

@advplyr commented on GitHub (Nov 22, 2021): I really appreciate the comparison. I know the scanner is still slow, but 12 hours seems crazy. Right now the scanner is going through each audio file 1 at a time and probing, this is the time consuming task. I'm going to start on what I will call Scanner v5 soon which will probe audio files in parallel. But still, 12 hours?? I wonder if this is related to the SMB. When you say Plex took 20 minutes to scan are you talking about a fresh empty Plex library? Audiobookshelf only probes audio files that it hasn't seen before, so subsequent scans should be very fast.
adam commented 2026-04-24 22:58:14 +02:00
Author
Owner
Copy Link
Copy Source

@seanap commented on GitHub (Nov 22, 2021):

The Plex scan is a full manual scan that touches every file, it's been a long time since starting fresh so I don't know how long that would take, but I can create a new test library and let it go.

The 12hr scan should have been a subsequent scan since I was just adding a small amount of books ~100 to the main library, but after abs lost connection to my root folder it attempted to go through every file 1 by 1. I would estimate from how many books I have and how many Issues I started to get that I was maybe only 20% through the library after 12hrs. Subsequent scans have never worked for me, but I can try to put a smaller group of files in there and test to see if SMB is an issue or if it's to do with the larger file count.

@seanap commented on GitHub (Nov 22, 2021): The Plex scan is a full manual scan that touches every file, it's been a long time since starting fresh so I don't know how long that would take, but I can create a new test library and let it go. The 12hr scan should have been a subsequent scan since I was just adding a small amount of books ~100 to the main library, but after abs lost connection to my root folder it attempted to go through every file 1 by 1. I would estimate from how many books I have and how many Issues I started to get that I was maybe only 20% through the library after 12hrs. Subsequent scans have never worked for me, but I can try to put a smaller group of files in there and test to see if SMB is an issue or if it's to do with the larger file count.
adam commented 2026-04-24 22:58:15 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 22, 2021):

The Plex scan time isn't that important, just curious. I think it will be nice to get to that level of performance, but still a ways away from that.

Testing the SMB a bit more would be great. Since I can't reproduce this issue on my machine right now I just want to get as much details as I can. Thanks for your help!

@advplyr commented on GitHub (Nov 22, 2021): The Plex scan time isn't that important, just curious. I think it will be nice to get to that level of performance, but still a ways away from that. Testing the SMB a bit more would be great. Since I can't reproduce this issue on my machine right now I just want to get as much details as I can. Thanks for your help!
adam commented 2026-04-24 22:58:15 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 23, 2021):

I caught a mistake in the scanner while setting up the new one. When looking for removed audiobooks it was only matching on the inode value, which most likely explains your issue.
I'm pushing a fix for that in release v1.6.26

@advplyr commented on GitHub (Nov 23, 2021): I caught a mistake in the scanner while setting up the new one. When looking for removed audiobooks it was only matching on the inode value, which most likely explains your issue. I'm pushing a fix for that in release `v1.6.26`
adam commented 2026-04-24 22:58:16 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 26, 2021):

I released a new build of the scanner in v1.6.28.
I have a test library with just over 2000 audio files and an initial scan runs in less than a minute. That same scan takes 2 - 3 minutes using the old scanner.
I'm still not sure how your scans could be taking so long, but this should be at least twice as fast. SMB shares are going to be slower, but not that slow.

@advplyr commented on GitHub (Nov 26, 2021): I released a new build of the scanner in `v1.6.28`. I have a test library with just over 2000 audio files and an initial scan runs in less than a minute. That same scan takes 2 - 3 minutes using the old scanner. I'm still not sure how your scans could be taking so long, but this should be at least twice as fast. SMB shares are going to be slower, but not *that* slow.
adam commented 2026-04-24 22:58:16 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Dec 2, 2021):

Just released v1.6.31 that is for performance on large libraries.

Instead of loading all the audiobooks upfront and handling sorting/filtering on the client side, audiobooks are fetched as needed.
The same thing will be done for the mobile app next.

@advplyr commented on GitHub (Dec 2, 2021): Just released `v1.6.31` that is for performance on large libraries. Instead of loading all the audiobooks upfront and handling sorting/filtering on the client side, audiobooks are fetched as needed. The same thing will be done for the mobile app next.
adam commented 2026-04-24 22:58:17 +02:00
Author
Owner
Copy Link
Copy Source

@itschrisonline commented on GitHub (Dec 6, 2021):

I don't know if this is the same issue but on the app I get "Bookshelf is empty for library Main". This is using local lan IP and remote using domain
Works using a browser.

@itschrisonline commented on GitHub (Dec 6, 2021): I don't know if this is the same issue but on the app I get "Bookshelf is empty for library Main". This is using local lan IP and remote using domain Works using a browser.
adam commented 2026-04-24 22:58:17 +02:00
Author
Owner
Copy Link
Copy Source

@itschrisonline commented on GitHub (Dec 6, 2021):

Made new librray and put small amout of books in. Works now

@itschrisonline commented on GitHub (Dec 6, 2021): Made new librray and put small amout of books in. Works now
adam commented 2026-04-24 22:58:18 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Dec 7, 2021):

The app was just updated to use the new bookshelf that will only load books needed. So the app should work just as well on large libraries.

@advplyr commented on GitHub (Dec 7, 2021): The app was just updated to use the new bookshelf that will only load books needed. So the app should work just as well on large libraries.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
Mirrored from GitHub Pull Request
 sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam (Adam Melkus)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#92
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?