[Bug]: Database lock file not being deleted (Windows) #719

Closed
opened 2026-04-24 23:18:04 +02:00 by adam · 17 comments
Owner

Originally created by @mikehoyle on GitHub (Oct 26, 2022).

Describe the issue

Setup:
Running Audiobookshelf from a Windows 10 Machine via a docker instance. A single library, imported, with no changes after import.

Listening modes:
Web & Android app

Problem:
Screenshot 2022-10-26 123411
When listening on web, I get continuous popups indicating that my listening progress is not being synced. It appears every few seconds while listening. Following the advice to restart playback does nothing. It seems to sometimes tell the truth -- sometimes my progress actually is synced (despite the message), while other times it is forgotten.

On the Android app, an error also appears in the form of a red exclamation point over the book image. I know this isn't the place to file mobile app bugs, but this likely reinforces that it's a server problem, experienced by both web & app.

Recent logs show a re-occurring error that seems to indicate a file lock is being held preventing updates. I'm not sure what would be holding that lock (or what file is the culprit), as I don't have any files explicitly open.

Steps to reproduce the issue

  1. For me, the issue appears any time I play an audiobook in the web player. I know these aren't helpful repro steps, but I've done little else. I'm happy to provide more information as needed.

Audiobookshelf version

v2.2.1

How are you running audiobookshelf?

Docker

Originally created by @mikehoyle on GitHub (Oct 26, 2022). ### Describe the issue Setup: Running Audiobookshelf from a Windows 10 Machine via a docker instance. A single library, imported, with no changes after import. Listening modes: Web & Android app Problem: ![Screenshot 2022-10-26 123411](https://user-images.githubusercontent.com/74795643/198120678-7ff1d2ac-62d0-4597-aa66-72d0086baabb.png) When listening on web, I get continuous popups indicating that my listening progress is not being synced. It appears every few seconds while listening. Following the advice to restart playback does nothing. It seems to sometimes tell the truth -- sometimes my progress actually is synced (despite the message), while other times it is forgotten. On the Android app, an error also appears in the form of a red exclamation point over the book image. I know this isn't the place to file mobile app bugs, but this likely reinforces that it's a server problem, experienced by both web & app. [Recent logs](https://github.com/advplyr/audiobookshelf/files/9873275/recent_logs.txt) show a re-occurring error that seems to indicate a file lock is being held preventing updates. I'm not sure what would be holding that lock (or what file is the culprit), as I don't have any files explicitly open. ### Steps to reproduce the issue 1. For me, the issue appears any time I play an audiobook in the web player. I know these aren't helpful repro steps, but I've done little else. I'm happy to provide more information as needed. ### Audiobookshelf version v2.2.1 ### How are you running audiobookshelf? Docker
adam added the bug label 2026-04-24 23:18:04 +02:00
adam closed this issue 2026-04-24 23:18:04 +02:00
Author
Owner

@louisjg commented on GitHub (Oct 26, 2022):

Just to chime in, I experience the exact same thing, albeit running in Docker on Linux. It keeps popping up, and sometimes it syncs, sometimes it doesn't. I haven't checked the log files, but will try to remember to do so and post back.

v2.2.1 running in a Linux (Ubuntu) docker.
Browser: Chrome Version 106.0.5249.119

@louisjg commented on GitHub (Oct 26, 2022): Just to chime in, I experience the exact same thing, albeit running in Docker on Linux. It keeps popping up, and sometimes it syncs, sometimes it doesn't. I haven't checked the log files, but will try to remember to do so and post back. v2.2.1 running in a Linux (Ubuntu) docker. Browser: Chrome Version 106.0.5249.119
Author
Owner

@advplyr commented on GitHub (Oct 26, 2022):

The lock file issue has to do with the database and seems specific to Windows file systems. I develop primarily on Windows and haven't had this happen to me since I fixed some issues about 6 - 9 months ago.

When I could reproduce this error the reason it was happening was because the lock file was not being deleted. The lock files exist in /config/ and get created then deleted to prevent simultaneous writes to a db file.

The lock file wasn't being deleted because Windows still had a file handler open for that file. Any software that scans your file system (i.e. antivirus or another media server) could potentially be reading that file preventing them from being deleted by abs.

Most likely the database will be replaced with Sqlite so we won't have to deal with this anymore but in the meantime you can check if anything else could be accessing files inside your /config folder.

@louisjg yours is most likely a separate issue not relating to the lock files. I haven't seen the lockfile issue reported for anything but Windows users.

@advplyr commented on GitHub (Oct 26, 2022): The lock file issue has to do with the database and seems specific to Windows file systems. I develop primarily on Windows and haven't had this happen to me since I fixed some issues about 6 - 9 months ago. When I could reproduce this error the reason it was happening was because the lock file was not being deleted. The lock files exist in `/config/` and get created then deleted to prevent simultaneous writes to a db file. The lock file wasn't being deleted because Windows still had a file handler open for that file. Any software that scans your file system (i.e. antivirus or another media server) could potentially be reading that file preventing them from being deleted by abs. Most likely the database will be replaced with Sqlite so we won't have to deal with this anymore but in the meantime you can check if anything else could be accessing files inside your `/config` folder. @louisjg yours is most likely a separate issue not relating to the lock files. I haven't seen the lockfile issue reported for anything but Windows users.
Author
Owner

@mikehoyle commented on GitHub (Oct 27, 2022):

Interesting. To me in the windows folks system the offending lock shows as
an empty directory. Can I just delete it manually?

On Wed, Oct 26, 2022, 2:18 PM advplyr @.***> wrote:

The lock file issue has to do with the database and seems specific to
Windows file systems. I develop primarily on Windows and haven't had this
happen to me since I fixed some issues about 6 - 9 months ago.

When I could reproduce this error the reason it was happening was because
the lock file was not being deleted. The lock files exist in /config/ and
get created then deleted to prevent simultaneous writes to a db file.

The lock file wasn't being deleted because Windows still had a file
handler open for that file. Any software that scans your file system (i.e.
antivirus or another media server) could potentially be reading that file
preventing them from being deleted by abs.

Most likely the database will be replaced with Sqlite so we won't have to
deal with this anymore but in the meantime you can check if anything else
could be accessing files inside your /config folder.

@louisjg https://github.com/louisjg yours is most likely a separate
issue not relating to the lock files. I haven't seen the lockfile issue
reported for anything but Windows users.


Reply to this email directly, view it on GitHub
https://github.com/advplyr/audiobookshelf/issues/1095#issuecomment-1292663827,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR2UU66RTEKJKCOEK4ZVNKDWFGN4DANCNFSM6AAAAAARPKYOFY
.
You are receiving this because you authored the thread.Message ID:
@.***>

@mikehoyle commented on GitHub (Oct 27, 2022): Interesting. To me in the windows folks system the offending lock shows as an empty directory. Can I just delete it manually? On Wed, Oct 26, 2022, 2:18 PM advplyr ***@***.***> wrote: > The lock file issue has to do with the database and seems specific to > Windows file systems. I develop primarily on Windows and haven't had this > happen to me since I fixed some issues about 6 - 9 months ago. > > When I could reproduce this error the reason it was happening was because > the lock file was not being deleted. The lock files exist in /config/ and > get created then deleted to prevent simultaneous writes to a db file. > > The lock file wasn't being deleted because Windows still had a file > handler open for that file. Any software that scans your file system (i.e. > antivirus or another media server) could potentially be reading that file > preventing them from being deleted by abs. > > Most likely the database will be replaced with Sqlite so we won't have to > deal with this anymore but in the meantime you can check if anything else > could be accessing files inside your /config folder. > > @louisjg <https://github.com/louisjg> yours is most likely a separate > issue not relating to the lock files. I haven't seen the lockfile issue > reported for anything but Windows users. > > — > Reply to this email directly, view it on GitHub > <https://github.com/advplyr/audiobookshelf/issues/1095#issuecomment-1292663827>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AR2UU66RTEKJKCOEK4ZVNKDWFGN4DANCNFSM6AAAAAARPKYOFY> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
Author
Owner

@advplyr commented on GitHub (Oct 27, 2022):

Yes you can delete it. The lock file is an empty directory

@advplyr commented on GitHub (Oct 27, 2022): Yes you can delete it. The lock file is an empty directory
Author
Owner

@mikehoyle commented on GitHub (Oct 27, 2022):

Update on this:

Deleting the lock file does seem to work... for a time. However, coming back to the session later, the exact same problem occurred. It seems to only ever happen with the config/user lockfile, so I don't anticipate it's some spurious issue such as a antivirus holding the lock.

@mikehoyle commented on GitHub (Oct 27, 2022): Update on this: Deleting the lock file does seem to work... for a time. However, coming back to the session later, the exact same problem occurred. It seems to only ever happen with the config/user lockfile, so I don't anticipate it's some spurious issue such as a antivirus holding the lock.
Author
Owner

@louisjg commented on GitHub (Oct 27, 2022):

@louisjg yours is most likely a separate issue not relating to the lock files. I haven't seen the lockfile issue reported for anything but Windows users.

OK, I will poke around and see if I can find out what is causing it; if I figure it out, I will make a separate ticket. Thanks!

@louisjg commented on GitHub (Oct 27, 2022): > @louisjg yours is most likely a separate issue not relating to the lock files. I haven't seen the lockfile issue reported for anything but Windows users. OK, I will poke around and see if I can find out what is causing it; if I figure it out, I will make a separate ticket. Thanks!
Author
Owner

@louisjg commented on GitHub (Oct 27, 2022):

@mikehoyle You can try to use resmon to see what process has the lock file open. There are other ways, but this is the easiest when it works.

https://www.winhelponline.com/blog/resource-monitor-find-process-locked-file-windows-7/

If you have any of your directories on a windows share, oplocks can cause this as well. SMB doesn't do so well with them sometimes.

@louisjg commented on GitHub (Oct 27, 2022): @mikehoyle You can try to use resmon to see what process has the lock file open. There are other ways, but this is the easiest when it works. [https://www.winhelponline.com/blog/resource-monitor-find-process-locked-file-windows-7/](url) If you have any of your directories on a windows share, oplocks can cause this as well. SMB doesn't do so well with them sometimes.
Author
Owner

@mikehoyle commented on GitHub (Oct 29, 2022):

Alright, I had to wait a while before I got another repro, but this time it's happening with a different lock, the sessions lock.

I can see through Resource Monitor and Process Explorer that nothing is holding a lock on the file. This may well be a repro of #845 and #653 before it, although -- while closed -- those don't seem to offer a concrete solution IMO.

I can also confirm through windows that the file itself is days old, so assuming that fs.mtime is translating properly to docker, it should be considered "stale" by any measure I see in the lockfile code. The default, afaict, is 10,000 millis and I don't see it changed elsewhere. Based on my reading of the code, this error is occurring because it's determined the lockfile isn't stale, when it most definitely is (source).

This is based on first reading and is in vendored code so I don't know that anyone else would have insight there, but based on my reading the lock definitely should be deleted in this scenario, even if it were missed in a past run. My only guess is that for some reason the file stat is misleading due to the Docker context, but I don't know why it would only occur randomly rather than every time.

@mikehoyle commented on GitHub (Oct 29, 2022): Alright, I had to wait a while before I got another repro, but this time it's happening with a different lock, the sessions lock. I can see through Resource Monitor and Process Explorer that nothing is holding a lock on the file. This may well be a repro of #845 and #653 before it, although -- while closed -- those don't seem to offer a concrete solution IMO. I can also confirm through windows that the file itself is days old, so assuming that fs.mtime is translating properly to docker, it should be considered "stale" by any measure I see in the [lockfile code](https://github.com/advplyr/audiobookshelf/blob/089d4b5cee3cf688976ce0483758f6d83d457ba1/server/libs/properLockfile/lib/lockfile.js). The default, afaict, is 10,000 millis and I don't see it changed elsewhere. Based on my reading of the code, this error is occurring because it's determined the lockfile *isn't* stale, when it most definitely is ([source](https://github.com/advplyr/audiobookshelf/blob/089d4b5cee3cf688976ce0483758f6d83d457ba1/server/libs/properLockfile/lib/lockfile.js#L67)). This is based on first reading and is in vendored code so I don't know that anyone else would have insight there, but based on my reading the lock definitely should be deleted in this scenario, even if it were missed in a past run. My only guess is that for some reason the file stat is misleading due to the Docker context, but I don't know why it would only occur randomly rather than every time.
Author
Owner

@advplyr commented on GitHub (Oct 29, 2022):

@mikehoyle that's a good observation, I hadn't even considered that. I haven't looked into it yet but are you passing in a timezone to Docker?

Are you running the server on something other then Windows then mapping in Windows folders?

@advplyr commented on GitHub (Oct 29, 2022): @mikehoyle that's a good observation, I hadn't even considered that. I haven't looked into it yet but are you passing in a timezone to Docker? Are you running the server on something other then Windows then mapping in Windows folders?
Author
Owner

@mikehoyle commented on GitHub (Oct 30, 2022):

I am running in Docker Desktop via the standard docker steps at https://www.audiobookshelf.org/install#docker. All mounted directories are standard Windows directories with no linking.

Running $ date in the docker terminal does display the current time zone as UTC (whereas I'm US/West Coast), but I would hope that isn't the problem given it's all stored as epoch time on the backend anyways. I tried running a few $ stat commands in the Docker shell for the container and all seems about right. Unfortunately I currently don't have any offending files, because I keep deleting them after doing a little debugging to keep the project working properly. Next time I get a repro I'll think to check that.

When I check Docker logs I do get a little bit of new information that in the audiobookshelf log display:

Error: Stale lock or attempt to update it after release
    at process.<anonymous> (/server/libs/njodb/index.js:101:37)
    at process.emit (node:events:513:28)
    at processEmit [as emit] (/server/libs/signalExit/index.js:204:34)
    at process._fatalException (node:internal/process/execution:167:25) {
  code: 'ECOMPROMISED'
}

This is thrown by njodb any time an ECOMPROMISED error bubbles up, and as far as I can tell any lockfile error of that time sources back to this block of code being called. That could be due to a few things (njodb unfortunately black-boxes the source stack-trace of the error), like it being too long between updates, but that still doesn't add any clarity to why this would just be a problem sometimes, for some users. It could be just as you said, a temporary hold due to antivirus or the like causing problems and things get in a bad state from there.

Regardless of how it happens, once it's set as compromised then it's doomed, because it's removed from the global locks list but not from the file system, so we're destined to always think the lock is being held but not ours, leading to the constant stream of errors that never resolve themselves.

Neither proper-lockfile nor njodb github repos have promising-looking bugs filed. I do see another repo addressed a similar issue by significantly increasing the staleness requirements (see here). If you're okay with diverging from njodb upstream, I think this is where you'd make the update. Even better, to not diverge the njodb lib, you could add it as a userproperties parameter in the Database instantiations here. It seems to me like it couldn't hurt to increase the default stale time significantly from 5 seconds (The example above used 5 minutes), that could go a long way to prevent errors like this if, for example, new files get scanned and locked for up to five seconds by Windows and then are stale by the time they're accessible.

@mikehoyle commented on GitHub (Oct 30, 2022): I am running in Docker Desktop via the standard docker steps at https://www.audiobookshelf.org/install#docker. All mounted directories are standard Windows directories with no linking. Running `$ date` in the docker terminal does display the current time zone as UTC (whereas I'm US/West Coast), but I would hope that isn't the problem given it's all stored as epoch time on the backend anyways. I tried running a few `$ stat` commands in the Docker shell for the container and all seems about right. Unfortunately I currently don't have any offending files, because I keep deleting them after doing a little debugging to keep the project working properly. Next time I get a repro I'll think to check that. When I check Docker logs I do get a little bit of new information that in the audiobookshelf log display: ``` Error: Stale lock or attempt to update it after release at process.<anonymous> (/server/libs/njodb/index.js:101:37) at process.emit (node:events:513:28) at processEmit [as emit] (/server/libs/signalExit/index.js:204:34) at process._fatalException (node:internal/process/execution:167:25) { code: 'ECOMPROMISED' } ``` This is thrown by njodb any time an ECOMPROMISED error bubbles up, and as far as I can tell any lockfile error of that time sources back to [this](https://github.com/advplyr/audiobookshelf/blob/089d4b5cee3cf688976ce0483758f6d83d457ba1/server/libs/properLockfile/lib/lockfile.js#L185) block of code being called. That could be due to a few things (njodb unfortunately black-boxes the source stack-trace of the error), like it being [too long between updates](https://github.com/advplyr/audiobookshelf/blob/089d4b5cee3cf688976ce0483758f6d83d457ba1/server/libs/properLockfile/lib/lockfile.js#L120), but that still doesn't add any clarity to why this would just be a problem sometimes, for some users. It could be just as you said, a temporary hold due to antivirus or the like causing problems and things get in a bad state from there. Regardless of how it happens, once it's set as compromised then it's doomed, because it's removed from the global locks list but not from the file system, so we're destined to always think the lock is being held but not ours, leading to the constant stream of errors that never resolve themselves. Neither proper-lockfile nor njodb github repos have promising-looking bugs filed. I do see another repo addressed a similar issue by significantly increasing the staleness requirements (see [here](https://github.com/yarnpkg/yarn/pull/8260/commits/a7ccca4d5a144f7dc4cf757f9b7af02802270d98)). If you're okay with diverging from njodb upstream, I think [this](https://github.com/advplyr/audiobookshelf/blob/089d4b5cee3cf688976ce0483758f6d83d457ba1/server/libs/njodb/index.js#L58) is where you'd make the update. Even better, to not diverge the njodb lib, you could add it as a userproperties parameter in the Database instantiations [here](https://github.com/advplyr/audiobookshelf/blob/ff04eb8d5e3df09b39a38fb2cef7d0a0b1c4e6b0/server/Db.js#L27). It seems to me like it couldn't hurt to increase the default stale time significantly from 5 seconds (The example above used 5 minutes), that could go a long way to prevent errors like this if, for example, new files get scanned and locked for up to five seconds by Windows and then are stale by the time they're accessible.
Author
Owner

@advplyr commented on GitHub (Oct 30, 2022):

When this was happening before with a file handler being left on the lockfile it was throwing a different error message.

I thought 5 minutes felt like a long time so I set it to 2 minutes to try it out. Hopefully this can hold us over until we migrate to sqlite. If you are up for using the edge docker release then you can test this out before the official release.

@advplyr commented on GitHub (Oct 30, 2022): When this was happening before with a file handler being left on the lockfile it was throwing a different error message. I thought 5 minutes felt like a long time so I set it to 2 minutes to try it out. Hopefully this can hold us over until we migrate to sqlite. If you are up for using the edge docker release then you can test this out before the official release.
Author
Owner

@mikehoyle commented on GitHub (Oct 30, 2022):

Great, Running on edge now, I'll update with results

@mikehoyle commented on GitHub (Oct 30, 2022): Great, Running on edge now, I'll update with results
Author
Owner

@mikehoyle commented on GitHub (Oct 31, 2022):

Unfortunately, this didn't solve the problem :(

I'm about out of ideas for now.

@mikehoyle commented on GitHub (Oct 31, 2022): Unfortunately, this didn't solve the problem :( I'm about out of ideas for now.
Author
Owner

@mikehoyle commented on GitHub (Oct 31, 2022):

Something that just occurred to me is the volumes I am mounting are all on an external drive. I'm not sure how or if that may be contributing, but all my drives including the audiobook one are NTFS file systems.

@mikehoyle commented on GitHub (Oct 31, 2022): Something that just occurred to me is the volumes I am mounting are all on an external drive. I'm not sure how or if that may be contributing, but all my drives including the audiobook one are NTFS file systems.
Author
Owner

@advplyr commented on GitHub (Nov 2, 2022):

That is probably it. Network drives have been behind a lot of the weird issues we've had with Windows file systems but I don't know enough about it to explain what is going on.
In order to test that you only need to move the /config volume to a local drive.

@advplyr commented on GitHub (Nov 2, 2022): That is probably it. Network drives have been behind a lot of the weird issues we've had with Windows file systems but I don't know enough about it to explain what is going on. In order to test that you only need to move the `/config` volume to a local drive.
Author
Owner

@mikehoyle commented on GitHub (Nov 5, 2022):

Just to clarify, the drive in question is local and external (as in usb-connected), not a network drive.

However I've moved my config to an internal drive and haven't experienced an issue in days, this seems like it was the problem. Closing for now.

@mikehoyle commented on GitHub (Nov 5, 2022): Just to clarify, the drive in question is local and external (as in usb-connected), not a network drive. However I've moved my config to an internal drive and haven't experienced an issue in days, this seems like it was the problem. Closing for now.
Author
Owner

@Kuro96 commented on GitHub (Mar 1, 2023):

same issue when running on linux with nfs

@Kuro96 commented on GitHub (Mar 1, 2023): same issue when running on linux with nfs
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#719