starred/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2026-05-30 23:40:40 +02:00
Code Issues 1.0k Packages Projects Releases 10 Wiki Activity

[Bug]: Downloading audiobooks - poor performance on local LAN #2059

New Issue
Closed
opened 2026-04-25 00:03:03 +02:00 by adam · 10 comments
adam commented 2026-04-25 00:03:03 +02:00
Owner
Copy Link
Copy Source

Originally created by @codhopper on GitHub (Jun 16, 2024).

What happened?

I've been fighting with downloading audiobooks to my android phone for a while. Initially I thought it might have been my wifi, router, virtualisation layers, bug with pfsense, haproxy slowness etc.

Attempted to download them from a directly connected machine with 10gbit connection. iperf gets around 9.37gbit. Audiobook gets around 100mbit.

The CPU usage is also very high. Out of 2 or 4 cores allocated on the i5-10400 it seemed to use around 100% of a single core any way.

What did you expect to happen?

Audiobooks download in less than about 10 minutes. CPU load is lower.

Steps to reproduce the issue

  1. Download large audiobook
  2. Grab some coffee
  3. Finish download

Audiobookshelf version

v2.10.1

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Linux

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

zlib set to 9
user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1265M    0 1265M    0     0  14.3M      0 --:--:--  0:01:28 --:--:-- 15.4M

zlib set to 1
user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1279M    0 1279M    0     0  30.5M      0 --:--:--  0:00:41 --:--:-- 30.5M

zlib set to 0
user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1368M    0 1368M    0     0   168M      0 --:--:--  0:00:08 --:--:--  173M

Additional Notes

audiobookshelf hosted on a machine with the CPU "AMD GX-415GA SOC with Radeon(tm) HD Graphics" performed quite a lot worse. Around 5-6MB/s (around 60mbit).
Current machine is running a "Intel(R) Core(TM) i5-10400"

Adjusting the zlib compression level in /server/utils/zipHelpers.js speeds up the connection significantly (at the price of file size). It might even be decompressed on the app side? Most of my testing has been on the desktop.

Originally created by @codhopper on GitHub (Jun 16, 2024). ### What happened? I've been fighting with downloading audiobooks to my android phone for a while. Initially I thought it might have been my wifi, router, virtualisation layers, bug with pfsense, haproxy slowness etc. Attempted to download them from a directly connected machine with 10gbit connection. iperf gets around 9.37gbit. Audiobook gets around 100mbit. The CPU usage is also very high. Out of 2 or 4 cores allocated on the i5-10400 it seemed to use around 100% of a single core any way. ### What did you expect to happen? Audiobooks download in less than about 10 minutes. CPU load is lower. ### Steps to reproduce the issue 1. Download large audiobook 2. Grab some coffee 3. Finish download ### Audiobookshelf version v2.10.1 ### How are you running audiobookshelf? Docker ### What OS is your Audiobookshelf server hosted from? Linux ### If the issue is being seen in the UI, what browsers are you seeing the problem on? None ### Logs ```shell zlib set to 9 user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1265M 0 1265M 0 0 14.3M 0 --:--:-- 0:01:28 --:--:-- 15.4M zlib set to 1 user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1279M 0 1279M 0 0 30.5M 0 --:--:-- 0:00:41 --:--:-- 30.5M zlib set to 0 user@server-on-lan:~$ curl -o /dev/null http://192.168.70.2:10228/api/items/4f7088d8-3a90-4b95-b9df-8e2c13c7ba0a/download?token=eyJiwiatruncatedforj85I % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1368M 0 1368M 0 0 168M 0 --:--:-- 0:00:08 --:--:-- 173M ``` ### Additional Notes audiobookshelf hosted on a machine with the CPU "AMD GX-415GA SOC with Radeon(tm) HD Graphics" performed quite a lot worse. Around 5-6MB/s (around 60mbit). Current machine is running a "Intel(R) Core(TM) i5-10400" Adjusting the zlib compression level in /server/utils/zipHelpers.js speeds up the connection significantly (at the price of file size). It might even be decompressed on the app side? Most of my testing has been on the desktop.
adam added the bug label 2026-04-25 00:03:03 +02:00
adam closed this issue 2026-04-25 00:03:03 +02:00
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@codhopper commented on GitHub (Jun 16, 2024):

After a bit more investigation it looks like the app (v0.9.74-beta on android) does store the files unzipped. So the compression is used to concatenate files for transfer (and reduce bandwidth a bit).

The "bad performance" also seems to come a bit from the way nodejs is handling multi-threading. There might be a solution with both compression and reasonable performance, if the compression could fill a buffer or better share resources. From what I saw there were 4 worker threads (with 2 allocated virtual CPUs) causing quite a bit of context switching during the download.

@codhopper commented on GitHub (Jun 16, 2024): After a bit more investigation it looks like the app (v0.9.74-beta on android) does store the files unzipped. So the compression is used to concatenate files for transfer (and reduce bandwidth a bit). The "bad performance" also seems to come a bit from the way nodejs is handling multi-threading. There might be a solution with both compression and reasonable performance, if the compression could fill a buffer or better share resources. From what I saw there were 4 worker threads (with 2 allocated virtual CPUs) causing quite a bit of context switching during the download.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Jun 16, 2024):

Are you using the Abs android mobile app to download or are you using the web client to download?

The web client download button zips all the files and downloads. The android/ios mobile app downloads the audio files and cover image individually and doesn't do any file compression.

@advplyr commented on GitHub (Jun 16, 2024): Are you using the Abs android mobile app to download or are you using the web client to download? The web client download button zips all the files and downloads. The android/ios mobile app downloads the audio files and cover image individually and doesn't do any file compression.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@codhopper commented on GitHub (Jun 17, 2024):

Are you using the Abs android mobile app to download or are you using the web client to download?

The web client download button zips all the files and downloads. The android/ios mobile app downloads the audio files and cover image individually and doesn't do any file compression.

Using a linux server with curl for testing (as it bypassed the wifi, haproxy and the router). I haven't tried downloading a larger audiobook via the app lately. I did download a 300Mb m4b one recently and it did take a while compared to what I would expect (hope for). Possibly 2 minutes or so.

@codhopper commented on GitHub (Jun 17, 2024): > Are you using the Abs android mobile app to download or are you using the web client to download? > > The web client download button zips all the files and downloads. The android/ios mobile app downloads the audio files and cover image individually and doesn't do any file compression. Using a linux server with curl for testing (as it bypassed the wifi, haproxy and the router). I haven't tried downloading a larger audiobook via the app lately. I did download a 300Mb m4b one recently and it did take a while compared to what I would expect (hope for). Possibly 2 minutes or so.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Jun 17, 2024):

Is the app accessing the server directly over LAN (either using direct IP or NAT Loopback)? If the request is leaving your network before getting to the server you'll be limited by your Internet upload speed.

@nichwall commented on GitHub (Jun 17, 2024): Is the app accessing the server directly over LAN (either using direct IP or NAT Loopback)? If the request is leaving your network before getting to the server you'll be limited by your Internet upload speed.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@codhopper commented on GitHub (Jun 17, 2024):

I've done a few tests with the app on my phone:

  • WIFI speed around my house is much lower than I thought. Can get a solid connection at 140mbit, but it went as low as 32mbit in a bad position when connected to the distant access point. And in an average position it was getting 60-80mbit.
  • 1.37GB audiobook downloaded with a stopwatch in 2 separate tests. 1min 48seconds and 2min 2seconds. (around 90-100mbit)
  • CPU utilisation when downloading from the app was certainly much lower. Around 6-9% of the i5-10400 core.
  • Downloading the zip file from the browser seems like a very uncommon use case, but can be significantly limited by CPU power (8s with zlib level 0 vs 1m28s with zlib level 9)

Summary: My wifi is slow.

Just to help clarify:

  • The curl output was from a machine on the local wired network, direct connection, downloading with the link from the web interface (compressed)
  • 10gbit connection to the docker host, tested with iperf3 at around 9.37gbit
  • original phone tests were not very scientific
  • Migrated from the AMD GX-415GA to the i5-10400 and performance did improve a bit, but not very much. Almost all of my testing was done from a laptop with a gigabit NIC over the web interface (zip file download)
@codhopper commented on GitHub (Jun 17, 2024): I've done a few tests with the app on my phone: - WIFI speed around my house is much lower than I thought. Can get a solid connection at 140mbit, but it went as low as 32mbit in a bad position when connected to the distant access point. And in an average position it was getting 60-80mbit. - 1.37GB audiobook downloaded with a **stopwatch** in 2 separate tests. 1min 48seconds and 2min 2seconds. (around 90-100mbit) - CPU utilisation when downloading from the app was certainly much lower. Around 6-9% of the i5-10400 core. - Downloading the zip file from the browser seems like a very uncommon use case, but can be significantly limited by CPU power (8s with zlib level 0 vs 1m28s with zlib level 9) **Summary:** My wifi is slow. Just to help clarify: - The curl output was from a machine on the local wired network, direct connection, downloading with the link from the web interface (compressed) - 10gbit connection to the docker host, tested with iperf3 at around 9.37gbit - original phone tests were not very scientific - Migrated from the AMD GX-415GA to the i5-10400 and performance did improve a bit, but not very much. Almost all of my testing was done from a laptop with a gigabit NIC over the web interface (zip file download)
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Jun 28, 2024):

From this discussion and additional research, I wonder if just removing the compression from the download zip is worth it, because compressing compressed files (such as mp3 or m4b, and then jpeg covers) on average doesn't save a lot of bandwidth and increases CPU load (as originally mentioned at the beginning of this discussion).

Could also be a Boolean server setting of "compress zip downloads" enabled by default with a note that the setting reduces file size but makes the download take longer?

@nichwall commented on GitHub (Jun 28, 2024): From this discussion and additional research, I wonder if just removing the compression from the download zip is worth it, because compressing compressed files (such as mp3 or m4b, and then jpeg covers) on average doesn't save a lot of bandwidth and increases CPU load (as originally mentioned at the beginning of this discussion). Could also be a Boolean server setting of "compress zip downloads" enabled by default with a note that the setting reduces file size but makes the download take longer?
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Jun 28, 2024):

I used the compression level that was in the example from node-archiver, so no thought was put into that on my part. We can reduce it but it would be nice to get some general benchmarks

@advplyr commented on GitHub (Jun 28, 2024): I used the compression level that was in the example from node-archiver, so no thought was put into that on my part. We can reduce it but it would be nice to get some general benchmarks
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@santschi commented on GitHub (Jul 6, 2024):

I can confirm that I have the same issue. Oftentimes I have to restart downloads 10-20 times before they even start. They keep hanging at 0%.
If they start they are extremely slow (android 15-20 min vs 40s in the browser) both on the same device.
This problem started a few months ago. Before that starting a download reliably worked although it always was slow.
No errors are showing up in the server logs.
The application has no resource constraints on the server.

@santschi commented on GitHub (Jul 6, 2024): I can confirm that I have the same issue. Oftentimes I have to restart downloads 10-20 times before they even start. They keep hanging at 0%. If they start they are extremely slow (android 15-20 min vs 40s in the browser) both on the same device. This problem started a few months ago. Before that starting a download reliably worked although it always was slow. No errors are showing up in the server logs. The application has no resource constraints on the server.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Dec 8, 2024):

I sat down and did some more investigation (got similar results as @codhopper's initial results) using my desktop and Android phone. This is specifically about the zip file download in a browser, not downloading in the mobile apps.

I downloaded a variety of library items (single file, multi-track, mp3, m4b, opus) to get an idea of the differences in file size when using compression level 0 (no compression) and compression level 9 (what we currently use). The largest difference I saw was 3%, but it was usually closer to 2%.

Using my personal network speeds, I put the following table together, assuming a download of a 1 GB library item, before compression. The WiFi testing was done using both my mobile web browser and a stopwatch, so numbers were rounded to the nearest 10 Mbps.

Compression Level Size (MB) Speed (Mbps) Duration (s)
Ethernet (1 Gbps) 0 1024 900 9
Ethernet (1 Gbps) 9 993.28 60 132
WiFi (867 Mbps) 0 1024 70 117
WiFi (867 Mbps) 9 993.28 60 132
WAN (10 Mbps) 0 1024 10 819
WAN (10 Mbps) 9 993.28 10 795

I did notice the download was slightly faster over LAN WiFi when not using compression, but that may be due to other network factors and because my WiFi is just not fast in general.

Removing the compression for the zip file can increase the time for downloading when away from home because the bottleneck is the upload speed instead of the compression speed and the file is a little bigger without compression. For my specific hardware (Synology NAS), I would need an upload speed of ~60 Mbps to for the compression speed to become the bottleneck.

The download speed of around 60-70 Mbps over WiFi matches the download speed I am seeing in the Android app.

From this, I think we should get rid of compression for downloading library item zip files due to a minimal impact on total bandwidth and download time, and moves the bottleneck for LAN to the speed of WiFi instead of the speed of compression, which does not really save any space locally.

Edit to add: Removing the compression level would also let us set the content-length header because we know exactly how big the download should be so users have an actual "expected time remaining" instead of just being notified when the download is completed and having to guess.

@nichwall commented on GitHub (Dec 8, 2024): I sat down and did some more investigation (got similar results as @codhopper's initial results) using my desktop and Android phone. This is specifically about the `zip` file download in a browser, not downloading in the mobile apps. I downloaded a variety of library items (single file, multi-track, mp3, m4b, opus) to get an idea of the differences in file size when using compression level 0 (no compression) and compression level 9 (what we currently use). The largest difference I saw was 3%, but it was usually closer to 2%. Using my personal network speeds, I put the following table together, assuming a download of a 1 GB library item, before compression. The WiFi testing was done using both my mobile web browser and a stopwatch, so numbers were rounded to the nearest 10 Mbps. | | Compression Level | Size (MB) | Speed (Mbps) | Duration (s) | | -- | -- | -- | -- | -- | Ethernet (1 Gbps) | 0 | 1024 | 900 | 9 Ethernet (1 Gbps) | 9 | 993.28 | 60 | 132 WiFi (867 Mbps) | 0 | 1024 | 70 | 117 WiFi (867 Mbps) | 9 | 993.28 | 60 | 132 WAN (10 Mbps) | 0 | 1024 | 10 | 819 WAN (10 Mbps) | 9 | 993.28 | 10 | 795 I did notice the download was slightly faster over LAN WiFi when not using compression, but that may be due to other network factors and because my WiFi is just not fast in general. Removing the compression for the zip file can increase the time for downloading when away from home because the bottleneck is the upload speed instead of the compression speed and the file is a little bigger without compression. For my specific hardware (Synology NAS), I would need an upload speed of ~60 Mbps to for the compression speed to become the bottleneck. The download speed of around 60-70 Mbps over WiFi matches the download speed I am seeing in the Android app. From this, I think we should get rid of compression for downloading library item zip files due to a minimal impact on total bandwidth and download time, and moves the bottleneck for LAN to the speed of WiFi instead of the speed of compression, which does not really save any space locally. Edit to add: Removing the compression level would also let us set the `content-length` header because we know exactly how big the download should be so users have an actual "expected time remaining" instead of just being notified when the download is completed and having to guess.
adam commented 2026-04-25 00:03:04 +02:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Dec 30, 2024):

Fixed in v2.17.6.

@github-actions[bot] commented on GitHub (Dec 30, 2024): Fixed in [v2.17.6](https://github.com/advplyr/audiobookshelf/releases/tag/v2.17.6).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
Mirrored from GitHub Pull Request
 sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam (Adam Melkus)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#2059
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?