starred/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2026-05-30 23:40:40 +02:00
Code Issues 1.0k Packages Projects Releases 10 Wiki Activity

[Bug]: Majority of Items show up as missing in full library scan #2587

New Issue
Closed
opened 2026-04-25 00:08:42 +02:00 by adam · 8 comments
adam commented 2026-04-25 00:08:42 +02:00
Owner
Copy Link
Copy Source

Originally created by @kymahi on GitHub (Feb 19, 2025).

What happened?

I had a few books not show up after my latest batch of uploads. Went to the settings and ran a full library scan. It completed in under a second and marked over 500 of our 7XX books missing. Clicking each of the books' three-dot menu and clicking Re-scan immediately resolves the issue, but this is obviously not a fix for over 500 books.

All subsequent scans produce the same issue.

What did you expect to happen?

Scan the library and add newly uploaded books.

Steps to reproduce the issue

  1. Go to settings
  2. Libraries
  3. Click Scan on the library

Audiobookshelf version

v2.19.3

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

macOS

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

2025-02-18 22:52:12.767

INFO

[LibraryScanner] Starting library scan ebe5f8be-95e1-45e6-9531-e7636dd31b75 for Audiobooks

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/Robert Bevan/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Robert Bevan/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Robert Bevan/._.DS_Store' }

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/Luke Smitherd/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Luke Smitherd/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Luke Smitherd/._.DS_Store' }

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/Lemony Snicket/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Lemony Snicket/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Lemony Snicket/._.DS_Store' }

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/Lauren Graham/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Lauren Graham/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Lauren Graham/._.DS_Store' }

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/David Wong/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/David Wong/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/David Wong/._.DS_Store' }

2025-02-18 22:52:12.884

ERROR

[fileUtils] Recurse files file "/audiobooks/Craig Alanson/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Craig Alanson/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Craig Alanson/._.DS_Store' }

2025-02-18 22:52:12.968

INFO

[LibraryScanner] Library scan "ebe5f8be-95e1-45e6-9531-e7636dd31b75" completed in 0:00 | 0 Added | 0 Updated | 528 Missing

2025-02-18 22:52:12.975

INFO

[LibraryScan] Scan log saved "/metadata/logs/scans/2025-02-18_ebe5f8be-95e1-45e6-9531-e7636dd31b75.txt"

2025-02-18 22:52:12.975

INFO

[LibraryController] Scan complete

Additional Notes

NOTE 1: Library is hosted on a NAS drive on my network mounted on my host server.

NOTE 2: This also occurred on v2.18.x and also after updating to v2.19.3

NOTE 3: In the logs there are errors about some files, but those files don't exist and are just hidden, Mac-only files that I purged after finding them on the server.

Originally created by @kymahi on GitHub (Feb 19, 2025). ### What happened? I had a few books not show up after my latest batch of uploads. Went to the settings and ran a full library scan. It completed in under a second and marked over 500 of our 7XX books missing. Clicking each of the books' three-dot menu and clicking Re-scan immediately resolves the issue, but this is obviously not a fix for over 500 books. All subsequent scans produce the same issue. ### What did you expect to happen? Scan the library and add newly uploaded books. ### Steps to reproduce the issue 1. Go to settings 2. Libraries 3. Click Scan on the library ### Audiobookshelf version v2.19.3 ### How are you running audiobookshelf? Docker ### What OS is your Audiobookshelf server hosted from? macOS ### If the issue is being seen in the UI, what browsers are you seeing the problem on? None ### Logs ```shell 2025-02-18 22:52:12.767 INFO [LibraryScanner] Starting library scan ebe5f8be-95e1-45e6-9531-e7636dd31b75 for Audiobooks 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/Robert Bevan/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Robert Bevan/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Robert Bevan/._.DS_Store' } 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/Luke Smitherd/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Luke Smitherd/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Luke Smitherd/._.DS_Store' } 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/Lemony Snicket/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Lemony Snicket/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Lemony Snicket/._.DS_Store' } 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/Lauren Graham/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Lauren Graham/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Lauren Graham/._.DS_Store' } 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/David Wong/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/David Wong/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/David Wong/._.DS_Store' } 2025-02-18 22:52:12.884 ERROR [fileUtils] Recurse files file "/audiobooks/Craig Alanson/._.DS_Store" has error [Error: EPERM: operation not permitted, stat '/audiobooks/Craig Alanson/._.DS_Store'] { errno: -1, code: 'EPERM', syscall: 'stat', path: '/audiobooks/Craig Alanson/._.DS_Store' } 2025-02-18 22:52:12.968 INFO [LibraryScanner] Library scan "ebe5f8be-95e1-45e6-9531-e7636dd31b75" completed in 0:00 | 0 Added | 0 Updated | 528 Missing 2025-02-18 22:52:12.975 INFO [LibraryScan] Scan log saved "/metadata/logs/scans/2025-02-18_ebe5f8be-95e1-45e6-9531-e7636dd31b75.txt" 2025-02-18 22:52:12.975 INFO [LibraryController] Scan complete ``` ### Additional Notes NOTE 1: Library is hosted on a NAS drive on my network mounted on my host server. NOTE 2: This also occurred on v2.18.x and also after updating to v2.19.3 NOTE 3: In the logs there are errors about some files, but those files don't exist and are just hidden, Mac-only files that I purged after finding them on the server.
adam added the bug label 2026-04-25 00:08:42 +02:00
adam closed this issue 2026-04-25 00:08:42 +02:00
adam commented 2026-04-25 00:08:42 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Feb 19, 2025):

Books being marked as missing with a full library scan and showing back up is almost always caused by an incorrect directory structure for your library.

Every book must be in its own folder (unless in the root of the library, so no subfolders at all). For example, you can't have a media file just in the author folder and then other folders there, everything will be grouped under that media file not in a subfolder. When you scan the full library, the scanner will find the first file, group everything together, and then mark the other books as missing because it has already been added to an existing book.

When you scan the individual items, the server is not traversing the entire structure and just going to that individual book and is able to mark it as not missing.

If you have the watcher enabled and temporarily had a media file not in a folder, things could still have gotten grouped while you were moving things around.

@nichwall commented on GitHub (Feb 19, 2025): Books being marked as missing with a full library scan and showing back up is almost always caused by an incorrect directory structure for your library. Every book must be in its own folder (unless in the root of the library, so no subfolders at all). For example, you can't have a media file just in the author folder and then other folders there, everything will be grouped under that media file not in a subfolder. When you scan the full library, the scanner will find the first file, group everything together, and then mark the other books as missing because it has already been added to an existing book. When you scan the individual items, the server is not traversing the entire structure and just going to that individual book and is able to mark it as not missing. If you have the watcher enabled and temporarily had a media file not in a folder, things could still have gotten grouped while you were moving things around.
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 19, 2025):

Image

I mostly let ABS do its thing with the filesystem and don't mess with the actual files for this fear exactly, but this is not the case here. This book I just uploaded (via the webpage's uploader) and then ran the scan.

@kymahi commented on GitHub (Feb 19, 2025): <img width="1255" alt="Image" src="https://github.com/user-attachments/assets/74bdb934-b1bc-4eed-9d5d-d13e5941764f" /> I mostly let ABS do its thing with the filesystem and don't mess with the actual files for this fear exactly, but this is not the case here. This book I just uploaded (via the webpage's uploader) and then ran the scan.
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Feb 19, 2025):

Can you share a screenshot of the file system of Peter Clines and Threshold Universe folders?

@nichwall commented on GitHub (Feb 19, 2025): Can you share a screenshot of the file system of `Peter Clines` and `Threshold Universe` folders?
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 19, 2025):

Image Image Image

is this what you mean?

I will say some of the earlier uploaded books are not in their own folders and are just hanging out in the root folder (since we just started the folder with a folder of books and not uploading them), but it's pretty hit-or-miss whether those are found.

Like this one:

Image

just a file in root, but not missing.

@kymahi commented on GitHub (Feb 19, 2025): <img width="1032" alt="Image" src="https://github.com/user-attachments/assets/f98c6db0-73dc-4a72-ae2c-3ea1914217bd" /> <img width="1032" alt="Image" src="https://github.com/user-attachments/assets/287b95f3-9af2-45f4-8bf9-5ed5954055ac" /> <img width="1032" alt="Image" src="https://github.com/user-attachments/assets/87867914-baf6-4598-bb07-cb0f59cc06ea" /> is this what you mean? I will say some of the earlier uploaded books are not in their own folders and are just hanging out in the root folder (since we just started the folder with a folder of books and not uploading them), but it's pretty hit-or-miss whether those are found. Like this one: <img width="1229" alt="Image" src="https://github.com/user-attachments/assets/d1206631-fb67-4399-875a-97d3ca7e2583" /> just a file in root, but not missing.
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 19, 2025):

2025-02-19_d3f24e50-b91e-4bb4-b969-04e27b7788a3.txt

I tried to restart the whole server computer and ran the scan again. This one took longer, but still is missing 528 files. It seems like it's the same books every time too. I manually re-scanned several books, and those same books are missing again.

@kymahi commented on GitHub (Feb 19, 2025): [2025-02-19_d3f24e50-b91e-4bb4-b969-04e27b7788a3.txt](https://github.com/user-attachments/files/18856430/2025-02-19_d3f24e50-b91e-4bb4-b969-04e27b7788a3.txt) I tried to restart the whole server computer and ran the scan again. This one took longer, but still is missing 528 files. It seems like it's the same books every time too. I manually re-scanned several books, and those same books are missing again.
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 19, 2025):

Dunno if this will help, but here is what the logs print when doing the manual "Re-scan" option from individual books' three-dot menu:

2025-02-19 02:17:33.578

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" key "ino" changed from "3765" to "1459"

2025-02-19 02:17:33.579

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" was missing but now found

2025-02-19 02:17:33.579

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" changed: [ino,isMissing,size,lastScan]

2025-02-19 02:17:33.581

DEBUG

[ApiCacheManager] libraryItem.afterUpdate: Clearing cache

2025-02-19 02:17:33.827

DEBUG

[AudioFileScanner] Smart track order for "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" using track key trackNumFromFilename

2025-02-19 02:17:33.828

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": "The Tomb of the Chatelaine" Getting metadata with precedence [folderStructure, audioMetatags, txtFiles, opfFile, absMetadata]

2025-02-19 02:17:33.828

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": setChapters: Using embedded chapters in first audio file /audiobooks/Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine/The_Tomb_of_the_Chatelaine_A_1920s_Country_House_Murder_Mystery_(Heathcliff_Lennox_Book_6)-LC_128_44100_stereo.m4b

2025-02-19 02:17:33.829

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Found metadata file "/metadata/items/bac52ac5-4f31-43e9-a880-ce3f46100282/metadata.json"

2025-02-19 02:17:33.832

DEBUG

[ApiCacheManager] book.afterUpdate: Clearing cache

2025-02-19 02:17:33.833

DEBUG

[Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Success saving abmetadata to "/metadata/items/bac52ac5-4f31-43e9-a880-ce3f46100282/metadata.json"
@kymahi commented on GitHub (Feb 19, 2025): Dunno if this will help, but here is what the logs print when doing the manual "Re-scan" option from individual books' three-dot menu: ``` 2025-02-19 02:17:33.578 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" key "ino" changed from "3765" to "1459" 2025-02-19 02:17:33.579 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" was missing but now found 2025-02-19 02:17:33.579 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Library item "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" changed: [ino,isMissing,size,lastScan] 2025-02-19 02:17:33.581 DEBUG [ApiCacheManager] libraryItem.afterUpdate: Clearing cache 2025-02-19 02:17:33.827 DEBUG [AudioFileScanner] Smart track order for "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine" using track key trackNumFromFilename 2025-02-19 02:17:33.828 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": "The Tomb of the Chatelaine" Getting metadata with precedence [folderStructure, audioMetatags, txtFiles, opfFile, absMetadata] 2025-02-19 02:17:33.828 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": setChapters: Using embedded chapters in first audio file /audiobooks/Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine/The_Tomb_of_the_Chatelaine_A_1920s_Country_House_Murder_Mystery_(Heathcliff_Lennox_Book_6)-LC_128_44100_stereo.m4b 2025-02-19 02:17:33.829 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Found metadata file "/metadata/items/bac52ac5-4f31-43e9-a880-ce3f46100282/metadata.json" 2025-02-19 02:17:33.832 DEBUG [ApiCacheManager] book.afterUpdate: Clearing cache 2025-02-19 02:17:33.833 DEBUG [Scan] "Karen Baugh Menuhin/Heathcliff Lennox/The Tomb of the Chatelaine": Success saving abmetadata to "/metadata/items/bac52ac5-4f31-43e9-a880-ce3f46100282/metadata.json" ```
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 21, 2025):

@advplyr Is there a limit on the number of directories ABS will scan?

I've done numerous things to fix this, including ensuring, by hand, every book is in the correct directory and format (e.g $AUTHOR/$SERIES/$TITLE/$FILE.m4b), as well as starting a whole new library from scratch.

When I was adding the directory to the library, I noticed there were several directories just not showing up in the list of author directories.

For example, A.G. Riddle and Agatha Christie were not showing up in the list when adding my /audiobooks source. I figured, hey, must be a permissions issue. So on the ABS host server, i ran sudo mv A.G.\ Riddle A.\ G.\ Riddle (space between the initials), and then it showed up in the list! But then when I did sudo mv Agatha\ Christie Agatha\ Christe\ 1 then Agatha Christe 1 showed up and A. G. Riddle disappeared.

Every time I renamed an author folder, the newly renamed one would show up, and then previously renamed one would disappear. The only thing I can think of is that there's some kind of limit on the number of author folders, but I can't imagine that's the case.

If anyone else has ever experienced this issue, or has an idea about the issue, I could use any help I can get.

@kymahi commented on GitHub (Feb 21, 2025): @advplyr Is there a limit on the number of directories ABS will scan? I've done numerous things to fix this, including ensuring, by hand, every book is in the correct directory and format (e.g `$AUTHOR/$SERIES/$TITLE/$FILE.m4b`), as well as starting a whole new library from scratch. When I was adding the directory to the library, I noticed there were several directories just not showing up in the list of author directories. For example, `A.G. Riddle` and `Agatha Christie` were not showing up in the list when adding my /audiobooks source. I figured, hey, must be a permissions issue. So on the ABS host server, i ran `sudo mv A.G.\ Riddle A.\ G.\ Riddle` (space between the initials), and then it showed up in the list! But then when I did `sudo mv Agatha\ Christie Agatha\ Christe\ 1` then `Agatha Christe 1` showed up and `A. G. Riddle` disappeared. Every time I renamed an author folder, the newly renamed one would show up, and then previously renamed one would disappear. The only thing I can think of is that there's some kind of limit on the number of author folders, but I can't imagine that's the case. If anyone else has ever experienced this issue, or has an idea about the issue, I could use any help I can get.
adam commented 2026-04-25 00:08:43 +02:00
Author
Owner
Copy Link
Copy Source

@kymahi commented on GitHub (Feb 21, 2025):

Solved. It was SMB. I don't know why, but it was. I used NFS to mount the drive, and everything shows up and scans.

@kymahi commented on GitHub (Feb 21, 2025): Solved. It was SMB. I don't know why, but it was. I used NFS to mount the drive, and everything shows up and scans.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
Mirrored from GitHub Pull Request
 sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam (Adam Melkus)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#2587
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?