mirror of
https://github.com/advplyr/audiobookshelf.git
synced 2026-05-30 23:40:40 +02:00
[Bug]: Books being "overwritten" during scans due to mistaken inode identity #919
Assignees
adam (Adam Melkus)
Clear assignees
adam (Adam Melkus)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/audiobookshelf#919
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @wtanksleyjr on GitHub (Jan 27, 2023).
Describe the issue
Docker-compose running ABS 2.2.12 on Ubuntu Linux 22.04; all folders mapped to the same 8TB HDD (not SDD). Prefer OPF data, and ODMs as well. Only one Library configured (I've reported this in the past when I had two libraries, but I wasn't able to capture any real information at the time).
I've noticed this many times, but always too late to get useful logs: I'd remember loading a book, but when I search for it, it isn't there. So I re-download. Later, I'd find a book that has the same cover as the one I re-downloaded but some other book's title. When I look at its folder and files, I find that the title matches the files, but the cover is from the newly missing book.
Interestingly, it turns out the "missing" book is right there on the drive where I'd put it, and absolutely nothing is ever suspicious about either the original book or the one that replaced it leaving only its cover - nothing's mixed up on the disk. So this is not a data loss problem, only metadata confusion.
So I've finally noticed this happening in time to actually capture logs -- the problem was that this is apparently random, so it usually gets books I don't notice because I loaded them a long time ago, or for which my logs are long gone.
The smoking gun is a line saying something like
[2023-01-27 03:12:49] DEBUG: [Scanner] scanFolderUpdates: Library item found by inode value "olderbook => newerbook" (Scanner.js:584). (Those two are the names of the folders directly containing the books.) I can see this line is intended to handle hard links, but every time it appears in my repo it's mistaken, there's no hard link there (if you want I'll deep-scan for duplicate inode numbers, but I've already confirmed none of the three books in this example share any common inode numbers for any of their files).My first example involves two "=>" lines that I actually WATCHED swap the same cover's title twice - it was random that I was looking in the right place at the right time. The original book is "The Skull of the World"; it had been loaded and Matched on the 22nd, and today the 26th I happened to be looking at newly loaded books, and I saw the cover for Skull of the World appear with the title "American Resistance", which I'd just gotten. Within a few second, the title changed again to "Unfree Speech" (still keeping the same "Skull" cover). Needless to say, both of those books had just been moved into the repo - but now I only have one book with the wrong cover and a jamble of metadata left over because the nonfiction books didn't have a subtitle or series so the resulting book had both of those from the original fiction book.
How's that for an intro? Any questions? Do I need to provide logs? Can I trim them a little? I've got them saved in full.
Steps to reproduce the issue
Audiobookshelf version
2.2.12
How are you running audiobookshelf?
Docker
@advplyr commented on GitHub (Feb 12, 2023):
I'm not sure what is going on here but it would be helpful to see what the actual inode value is so I added it in the logs for the latest release.
If you have this happen again that same line of the log you shared will include the inode value.
@wtanksleyjr commented on GitHub (Feb 12, 2023):
100% will check for that.
-Wm
On Sat, Feb 11, 2023 at 3:55 PM advplyr @.***> wrote:
@ceramicwhite commented on GitHub (Feb 18, 2023):
@advplyr
This happens on my server sometimes after I do scan.
A couple things I've noticed:
metadata/cache/coversfixes the problemI have
v2.2.15running and the cover problem just happened again but I don't see the Inode values and debug is on. Here's the log if it helps:2023-02-18.txt
Book with incorrect cover is Trackers (
li_o1kwryisuc929ch8ix)Cover being shown is Polar Shift (
li_83e4qfl4ce94b1ibtq)Hope this helps
@wtanksleyjr commented on GitHub (Mar 22, 2023):
Wow, it happened ... I didn't expect to ever see it again, but this time I got something recent enough that I even have the logs!
163 [2023-03-22 15:31:58] DEBUG: [Scanner] scanFolderUpdates: Library item found by inode value=63328265. "Solo Leveling, Vol. 2 (Novel) - B09GHJTKSJ => 9524067" (Scanner.js:585)Here's a lightly edited log (I searched for the two books concerned and kept everything between the two islands of log entries that mention them). I have the full logs as well, if you want.
narrowed-log.txt
(The second item is in a folder with a numeric name, in case you're wondering - it's a unique ID. The other one is how I have Libation configured to save my Audible books.)
The sequence of events here was that I'd returned and deleted "Solo Leveling", so the item is marked missing; but in the meantime I checked out "The Staff Engineer's Path". Apparently the latter item is somehow seen to have the same inode as the former (you can now see its value right above).
One question ... what file is being examined for an inode there?
@advplyr commented on GitHub (Mar 22, 2023):
Ah that could be it. Just to clarify, you first deleted the book folder (which would have freed up that inode value), then you added the new book which got incorrectly matched?
The inode being used is the folder
@wtanksleyjr commented on GitHub (Mar 22, 2023):
That's correct in this case, although not in any of the previous cases
(those all involved both books still existing).
Additionally, when you compare inodes, are you guarding against comparing
them across filesystems (inodes are not unique across filesystems)? Can you
even tell from inside Docker? Perhaps this is the cause of the other guy
who posted here involving both an SSD and HDD. In my case I know they're
the same filesystem and even the same Docker mount point so they're sure to
be unique.
On Wed, Mar 22, 2023 at 1:52 PM advplyr @.***> wrote:
@wtanksleyjr commented on GitHub (Mar 22, 2023):
My gut feeling is that this matching is there for only one special case: if the "newly discovered" folder is actually an already-scanned folder. The implication is you're detecting renames, which makes sense and I don't know why it's generating false positives except for this new case.
But it seems to me the current method isn't the greatest way to handle renames, because it creates both an orphaned book object and a new book object for the rename even when it's correct. And the new book object contains two layers of metadata, the result of the initial scan on the newer folder with the nonblank items from the older item applied over it (which, by the way, easily produces very bad results). It would seem better to process the rename detection by changing the folder name on the old book's object, instead of by creating a new book and then overwriting its metadata.
I'm trying to think of ways to handle this correctly, and it's hard to do. One thing that makes sense is to never treat a discovered inode as a rename when the book that has the old inode still exists -- that's a quick test that would completely stop all of the cases I reported above even though we still don't know why they happen.
That wouldn't stop my new problem, of course. But that's going to be a very rare problem, since it only happens if I leave a deleted item just sitting there while I load new items. It's rather a bad problem, mind you, because it can result in completely overwriting a new book's info with an old book. But I can't think of how to prevent it apart from keeping it rare ... maybe when a book's marked as completely missing, remove its inode record after finishing the scans as completely as possible? That way rename detections can still be done the old way, but inode reuse won't kill us (unless the system reuses the inode really quickly).
@advplyr commented on GitHub (Jun 14, 2023):
This was how it was originally built but there is a bug somewhere. I opened an issue for this a while ago https://github.com/advplyr/audiobookshelf/issues/1161
This issue came up again and has me looking at why some filesystems have such small inode values. This comment had some insight https://github.com/advplyr/audiobookshelf/issues/1733#issuecomment-1527178564
I don't think there is a way to detect the actual filesystem from inside docker but if there is that would be the best solution.
@advplyr commented on GitHub (Jun 14, 2023):
This is a good idea.
For reference the inode value on my XFS filesystems are this long
649362779914890946. So even if I had my audiobooks spread across 100 drives the likelihood of a collision would be very small.@zoredache commented on GitHub (Jul 14, 2023):
I was finally got around to importing a bunch of audio books I picked up from old Humble Bundles in the past to m4b and adding them to my library. I added like ~60 books at once, and this trashed the metadata for like 50 books got trashed.
My system has is using zfs, and I have separate datasets I use for my audible backups, and the directory I use for books I purchased outside audible.
As an example. One of the books was the Big Finish book Phantasmagoria which I added to purchased_misc doing this trashed the metadata for the book 'The Happiness Hypothesis'.
Stat output
Audiobookshelf logs
Anyway my first suggestion is if you see an 'File Added' that where the inode matches an existing book. Add a check so that you don't update the old book, if the files still exist for the old book.
@advplyr commented on GitHub (Sep 28, 2023):
As of 2.4.3 the scanner is no longer checking for inode first. It is looking for exact matching file path first then falling back to inode. There is more to be done here because you could still have this collision for renames.
@jaxley commented on GitHub (Jul 3, 2025):
This is happening to me as well. It appears that it is incorrectly matching inodes across separate filesystems. My uploaded books are store in a separate mounted filesystem from my former calibre library (a bind mount of a different filesystem entirely)
Perhaps if the fully qualified paths aren't rooted to a common directory, you shouldn't trust any inode matches...
The library has clearly different rooted paths for the two directories.
Logs showing this corruption happening
@jaxley commented on GitHub (Jul 3, 2025):
There is a stat.dev Id that is available via the fs API https://nodejs.org/api/fs.html#class-fsstats. That should be used to disambiguate inodes that can be the same across devices.
The code probably could double check several other things to ensure its logic is robust to avoid corruption.
@Armelline commented on GitHub (Jan 15, 2026):
I'm having this issue drive me nuts. When ABS feels like it it will just assign an added book's filepath to another, completely different book in a different author folder. Then when I fix that one the issue passes to another book. Eventually I've had to remove a dozen books from my library and anything I add gets fubar'd. 2.32.1, Docker, macOS.
"As of 2.4.3 the scanner is no longer checking for inode first. It is looking for exact matching file path first then falling back to inode." I'm unfortunately seeing no evidence of this. It's happily swapping around books and I can't seem to do anything to stop it.
Folder path:
Author/Series/Book/file.m4b
sometimes:
Author/Series/Series2/Book/file.m4b
@Armelline commented on GitHub (Jan 15, 2026):
Here's some logs that show (iirc, I might not be remembering exactly):
I'm now "untangled" (these was the last in a long line of such faffings).
Hopefully the logs can help explain what on earth is going on here as it's driving me pretty nuts having to do this dance every time ABS loses a book to another books' listing.
I didn't check the logs for API key but I assume it wouldn't be written there and it'll be expired by the time I finish posting this anyway.
logs.txt
@jaxley commented on GitHub (Jan 15, 2026):
I fixed this in the linked PR. Response I got from the developers so far is they are busy working on various rewrites and features so there's no ETA on when the PR will get reviewed and merged.
I agree it's really annoying!
I'm running a local build of my PR since August.
@Armelline commented on GitHub (Jan 15, 2026):
Thanks @jaxley. I'm pretty new to Docker but I'll wrap my head around how to do the same. Hopefully your PR fixes things for me too! This issue made me really regret swapping over from Plex but it's too late to go back now so I'll stick with it!