[Bug]: Watcher not working #1546

Closed
opened 2026-04-24 23:49:40 +02:00 by adam · 15 comments
Owner

Originally created by @Hallo951 on GitHub (Nov 25, 2023).

Describe the issue

I have the problem that after updating from version 2.3.3 directly to version 2.5.0, the watcher no longer works properly.

The problem is that the watcher does not recognise any new audiobooks. I don't get an error message either. It simply does not recognise them. If I scan the same folder manually (partial scan), it recognises all unrecorded audiobooks without any problems. Before the update, everything was fine with the Watcher.

I use Abs under Linux (Debian) without Docker. To test the functionality of the watcher with a new library, I created a new folder directly on the computer on which Abs is running. I then created a new library in Abs with active folder watcher and linked it to the newly created folder.

To test the watcher, I copied a new audiobook into the newly created folder and waited. Unfortunately, the error was the same and Abs did not recognise the audiobook in the newly created folder on its own (automatically). It only recognised it during a manual scan.

Why does this no longer work? The manual scan of a folder works and recognises all audiobooks not yet in the database! Could something have gone wrong during the direct update from version 2.3.3. to version 2.5.0?

Steps to reproduce the issue

  1. Copy a new audiobook in the audiobook folder.

Audiobookshelf version

2.5.0

How are you running audiobookshelf?

Debian/PPA

Originally created by @Hallo951 on GitHub (Nov 25, 2023). ### Describe the issue I have the problem that after updating from version 2.3.3 directly to version 2.5.0, the watcher no longer works properly. The problem is that the watcher does not recognise any new audiobooks. I don't get an error message either. It simply does not recognise them. If I scan the same folder manually (partial scan), it recognises all unrecorded audiobooks without any problems. Before the update, everything was fine with the Watcher. I use Abs under Linux (Debian) without Docker. To test the functionality of the watcher with a new library, I created a new folder directly on the computer on which Abs is running. I then created a new library in Abs with active folder watcher and linked it to the newly created folder. To test the watcher, I copied a new audiobook into the newly created folder and waited. Unfortunately, the error was the same and Abs did not recognise the audiobook in the newly created folder on its own (automatically). It only recognised it during a manual scan. Why does this no longer work? The manual scan of a folder works and recognises all audiobooks not yet in the database! Could something have gone wrong during the direct update from version 2.3.3. to version 2.5.0? ### Steps to reproduce the issue 1. Copy a new audiobook in the audiobook folder. ### Audiobookshelf version 2.5.0 ### How are you running audiobookshelf? Debian/PPA
adam added the bug label 2026-04-24 23:49:40 +02:00
adam closed this issue 2026-04-24 23:49:40 +02:00
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

Nothing changed with the watcher so I'm not sure. You could try setting up a separate install of v2.3.3 to see if it works there again. Maybe something changed with your file system.

@advplyr commented on GitHub (Nov 25, 2023): Nothing changed with the watcher so I'm not sure. You could try setting up a separate install of v2.3.3 to see if it works there again. Maybe something changed with your file system.
Author
Owner

@Hallo951 commented on GitHub (Nov 25, 2023):

No, everything has remained the same on my file system. Didn't you once write that you had built something into the watcher that checks whether a copy operation has been completed and only then starts the watcher? This was not yet available in version 2.3.3. I could well imagine that it has something to do with that.

As I said, the file system is completely the same. I just did an update. It doesn't work with a new folder and a new library either.

@Hallo951 commented on GitHub (Nov 25, 2023): No, everything has remained the same on my file system. Didn't you once write that you had built something into the watcher that checks whether a copy operation has been completed and only then starts the watcher? This was not yet available in version 2.3.3. I could well imagine that it has something to do with that. As I said, the file system is completely the same. I just did an update. It doesn't work with a new folder and a new library either.
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

Yeah there was an update with that but if you have debug logs on you would still be able to see watcher logs like this:

Logger.debug('[Watcher] File Added', path)
Logger.debug(`[Watcher] Rename ${pathFrom} => ${pathTo}`)

Also you would see the watcher ready logs when starting the server for each library

Logger.info(`[Watcher] "${library.name}" Ready`)
@advplyr commented on GitHub (Nov 25, 2023): Yeah there was an update with that but if you have debug logs on you would still be able to see watcher logs like this: ```js Logger.debug('[Watcher] File Added', path) Logger.debug(`[Watcher] Rename ${pathFrom} => ${pathTo}`) ``` Also you would see the watcher ready logs when starting the server for each library ```js Logger.info(`[Watcher] "${library.name}" Ready`) ```
Author
Owner

@Hallo951 commented on GitHub (Nov 25, 2023):

The watcher is also initialised when I start it (see screenshot), but I don't get any further messages like:

Logger.debug('[Watcher] File Added', path) Logger.debug([Watcher] Rename ${pathFrom} => ${pathTo})

This simply does not exist for me. Could it be that the watcher under Linux does not recognise when a file has been copied and therefore the watcher does not start? As I understand it, the watcher only starts when it receives the signal that the copying process has been completed or not?

Screenshot_20231125_173955_com brave browser

@Hallo951 commented on GitHub (Nov 25, 2023): The watcher is also initialised when I start it (see screenshot), but I don't get any further messages like: `Logger.debug('[Watcher] File Added', path) Logger.debug(`[Watcher] Rename ${pathFrom} => ${pathTo}`)` This simply does not exist for me. Could it be that the watcher under Linux does not recognise when a file has been copied and therefore the watcher does not start? As I understand it, the watcher only starts when it receives the signal that the copying process has been completed or not? ![Screenshot_20231125_173955_com brave browser](https://github.com/advplyr/audiobookshelf/assets/40667862/35eb2f52-bffe-440e-b754-5ff3f96b6d8a)
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

The watcher logs I shared are emitted as soon as the watcher detects the change, so if you aren't seeing it then it means the watcher is not detecting file changes in your file system.
Abs uses https://github.com/fabiospampinato/watcher which is an improvement over the basic nodejs file watcher. Network file systems very often do not work with file watchers. If you test creating a library not using an NFS drive then I would guess that the watcher will work for you.

@advplyr commented on GitHub (Nov 25, 2023): The watcher logs I shared are emitted as soon as the watcher detects the change, so if you aren't seeing it then it means the watcher is not detecting file changes in your file system. Abs uses https://github.com/fabiospampinato/watcher which is an improvement over the basic nodejs file watcher. Network file systems very often do not work with file watchers. If you test creating a library not using an NFS drive then I would guess that the watcher will work for you.
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

The watcher not being usable for many setups because of NFS drives is the reason that I added the scheduled scans btw.

@advplyr commented on GitHub (Nov 25, 2023): The watcher not being usable for many setups because of NFS drives is the reason that I added the scheduled scans btw.
Author
Owner

@Hallo951 commented on GitHub (Nov 25, 2023):

Ok, I'll test it again, even though I've already done so. I created a new folder in my mount folder locally on my server on which Abs is also running and then entered it in a newly created Abs library. Then I tested the watcher with a new audio book which I copied into the folder. Unfortunately without success.

If I create the folder in a different location in the system, I get the following error message:

ERROR [Watcher] Error: EACCES: permission denied, watch '/Hoerbuch_Test/A. Lee Martinez/Diner of horrors'

Does this mean that the watcher generally does not work with mounted folders from a synology nas?

@Hallo951 commented on GitHub (Nov 25, 2023): Ok, I'll test it again, even though I've already done so. I created a new folder in my mount folder locally on my server on which Abs is also running and then entered it in a newly created Abs library. Then I tested the watcher with a new audio book which I copied into the folder. Unfortunately without success. If I create the folder in a different location in the system, I get the following error message: ERROR [Watcher] Error: EACCES: permission denied, watch '/Hoerbuch_Test/A. Lee Martinez/Diner of horrors' Does this mean that the watcher generally does not work with mounted folders from a synology nas?
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

Yes the watcher is unreliable with network file systems. It is mentioned here https://nodejs.org/docs/latest/api/fs.html#caveats

This feature depends on the underlying operating system providing a way to be notified of file system changes.

If the underlying functionality is not available for some reason, then fs.watch() will not be able to function and may throw an exception. For example, watching files or directories can be unreliable, and in some cases impossible, on network file systems (NFS, SMB, etc) or host file systems when using virtualization software such as Vagrant or Docker.

@advplyr commented on GitHub (Nov 25, 2023): Yes the watcher is unreliable with network file systems. It is mentioned here https://nodejs.org/docs/latest/api/fs.html#caveats > This feature depends on the underlying operating system providing a way to be notified of file system changes. > > If the underlying functionality is not available for some reason, then fs.watch() will not be able to function and may throw an exception. For example, watching files or directories can be unreliable, and in some cases impossible, on network file systems (NFS, SMB, etc) or host file systems when using virtualization software such as Vagrant or Docker.
Author
Owner

@Hallo951 commented on GitHub (Nov 25, 2023):

And why did it work in version 2.3.3?

How do I find out if my Linux system can do this?

@Hallo951 commented on GitHub (Nov 25, 2023): And why did it work in version 2.3.3? How do I find out if my Linux system can do this?
Author
Owner

@advplyr commented on GitHub (Nov 25, 2023):

I'm not sure but I don't think it has to do with the server upgrade. You will have to do more testing. It should work fine if you are not using an NFS drive

@advplyr commented on GitHub (Nov 25, 2023): I'm not sure but I don't think it has to do with the server upgrade. You will have to do more testing. It should work fine if you are not using an NFS drive
Author
Owner

@Hallo951 commented on GitHub (Nov 25, 2023):

Ok, so the watcher cannot monitor NFS, smb or what ever mounted folders. I found the following on the page you linked to the currently implemented watcher:

https://github.com/fabiospampinato/watcher/issues/24

If I understand it correctly, the watcher "pollex" linked in this post can monitor mounted and local folders and even faster than the current watcher. Would that perhaps be a way to solve the problem?

Here the link to pollex:
https://github.com/fabiospampinato/pollex

@Hallo951 commented on GitHub (Nov 25, 2023): Ok, so the watcher cannot monitor NFS, smb or what ever mounted folders. I found the following on the page you linked to the currently implemented watcher: [https://github.com/fabiospampinato/watcher/issues/24](https://github.com/fabiospampinato/watcher/issues/24) If I understand it correctly, the watcher "pollex" linked in this post can monitor mounted and local folders and even faster than the current watcher. Would that perhaps be a way to solve the problem? Here the link to pollex: [https://github.com/fabiospampinato/pollex](https://github.com/fabiospampinato/pollex)
Author
Owner

@mikiher commented on GitHub (Nov 26, 2023):

Though I have not tried pollex, I would generally not recommend using a polling watcher.

I tried at some point to replace watcher with chokidar in polling mode (usePolling=true) as the watcher in Audiobookshelf.
The polling interval has to be be tied closely with the size of the watched library, otherwise the watcher will bring your system down hogging your cpu capacity, and even if you set the interval to be large enough, you will still see cpu spikes when most of the polling happens. I therefore ditched the idea for integrating a polling watcher, and wrote abswatcher, an external solution for a Windows Docker watcher (where the internal ABS watcher doesn't work for the same reasons your doesn't work).

Perhaps pollex is mitigating those issues somehow, but I'd bet it still suffers from the same fundamental issues. The author himself says:

Unless you really want this I'd recommend using a normal filesystem watcher instead, since polling can get expensive.

@mikiher commented on GitHub (Nov 26, 2023): Though I have not tried pollex, I would generally not recommend using a polling watcher. I tried at some point to replace [watcher](https://github.com/fabiospampinato/watcher) with [chokidar](https://github.com/paulmillr/chokidar) in polling mode (usePolling=true) as the watcher in Audiobookshelf. The polling interval has to be be tied closely with the size of the watched library, otherwise the watcher will bring your system down hogging your cpu capacity, and even if you set the interval to be large enough, you will still see cpu spikes when most of the polling happens. I therefore ditched the idea for integrating a polling watcher, and wrote [abswatcher](https://github.com/mikiher/abswatcher), an external solution for a Windows Docker watcher (where the internal ABS watcher doesn't work for the same reasons your doesn't work). Perhaps pollex is mitigating those issues somehow, but I'd bet it still suffers from the same fundamental issues. The author himself says: > Unless you really want this I'd recommend using a [normal](https://github.com/fabiospampinato/watcher) filesystem watcher instead, since polling can get expensive.
Author
Owner

@Hallo951 commented on GitHub (Nov 26, 2023):

Thanks for your experience. Do you have any idea how to get the watcher to monitor mounted folders via NFS?

@Hallo951 commented on GitHub (Nov 26, 2023): Thanks for your experience. Do you have any idea how to get the watcher to monitor mounted folders via NFS?
Author
Owner

@mikiher commented on GitHub (Nov 26, 2023):

There's unfortunately no magic here. For NFS mounted directories (and similar network based folders), there are only two possible solutions, as far as I know.

  1. A polling watcher (and I described above why it's probably a bad idea to use one).
  2. If you have access to the machine that hosts the NFS share you mounted, you can run on it something similar to abswatcher, which watches the folder locally, not using polling, and uses Audiobookshelf API to notify it of changes. abswatcher is not immediately suitable for this, because it was written to deal specifically with Docker mounts on Windows, but I guess it can be tweaked to deal with NFS mount if you somehow supply it with the relevant mount definitions (i.e. the mapping between the local directory that serves as an NFS share and the remote mount point)
@mikiher commented on GitHub (Nov 26, 2023): There's unfortunately no magic here. For NFS mounted directories (and similar network based folders), there are only two possible solutions, as far as I know. 1. A polling watcher (and I described above why it's probably a bad idea to use one). 2. If you have access to the machine that hosts the NFS share you mounted, you can run on it something similar to [abswatcher](https://github.com/mikiher/abswatcher), which watches the folder **_locally_**, not using polling, and uses Audiobookshelf API to notify it of changes. abswatcher is not immediately suitable for this, because it was written to deal specifically with Docker mounts on Windows, but I guess it can be tweaked to deal with NFS mount if you somehow supply it with the relevant mount definitions (i.e. the mapping between the local directory that serves as an NFS share and the remote mount point)
Author
Owner

@advplyr commented on GitHub (Nov 27, 2023):

@mikiher Thanks for the explanation. I agree on the poll watcher, some users have very large libraries and this would be difficult to maintain. We could potentially set an opt-in for this but it's still not a great solution I don't think.

This is the only reason that scheduling library scans was added. A library scan shouldn't be expensive since we are comparing the last modified times of files and if they haven't changed we aren't scanning further.

I'll close this for now since it isn't a bug but we should document this somewhere

@advplyr commented on GitHub (Nov 27, 2023): @mikiher Thanks for the explanation. I agree on the poll watcher, some users have very large libraries and this would be difficult to maintain. We could potentially set an opt-in for this but it's still not a great solution I don't think. This is the only reason that scheduling library scans was added. A library scan shouldn't be expensive since we are comparing the last modified times of files and if they haven't changed we aren't scanning further. I'll close this for now since it isn't a bug but we should document this somewhere
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#1546