mirror of
https://github.com/advplyr/audiobookshelf.git
synced 2026-05-30 23:40:40 +02:00
[Bug]: OOM Crash when loading podcast library with lots of tracks #1363
Assignees
adam (Adam Melkus)
Clear assignees
adam (Adam Melkus)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: starred/audiobookshelf#1363
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @paxchristos on GitHub (Sep 10, 2023).
Describe the issue
Upgraded to 2.4.1 from 2.3.3, was able to scan podcast library (haven't been able to do so since 2.3.x, thank you for the fix!)
Selected Podcast library, then went to library and the server starts crashing. If you leave open the webpage for the library, it'll enter a loop of starting the same process and crashing the server until the page is closed.
Podcast Library stats:
21 library items
10799 tracks
Machine Stats:
ubuntu 22.04.03
i3 10100
16gb ram
256gb m2 nvme
Here's the exact OOM crash:
Here's a larger section of the log:
Steps to reproduce the issue
Audiobookshelf version
2.4.1
How are you running audiobookshelf?
Debian/PPA
@advplyr commented on GitHub (Sep 12, 2023):
Roughly how many episodes does your largest podcast have?
@paxchristos commented on GitHub (Sep 13, 2023):
~3800 for the largest
there are about 4 more in the ~750 -1k episode range
@jrhedman commented on GitHub (Sep 14, 2023):
It looks like it's the same error as I see on a few issues so they're probably related:
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory(I'm having the same issue and after updating to 2.4.1, I'm unable to open my podcast library without it crashing, unfortunately I didn't make note of version I was on previously).
I've been following this bug since this issue was created.
EDIT: My largest podcast has ~2175 episodes but it's not the culprit. I started getting this crash after adding a new podcast with ~670 episodes and it crashed after downloading half of them.
@paxchristos commented on GitHub (Sep 14, 2023):
Famous last words: I use podgrab to download podcasts, so notionally audiobookshelf shouldn't be downloading podcasts automatically, only serving them.
@jrhedman commented on GitHub (Sep 14, 2023):
Right, like I mentioned above, previously, my failure occurred when audiobookshelf was scheduled to download podcasts, but that doesn't mean it was failing at the downloading stage.
Since I'm failing when I open my podcast library as of now, I think it's the same error as before occurring when querying currently downloaded episodes (which it'd have to do to know what to download). That might explain why I'm seeing the error now after a fresh install as well.
@advplyr has noted in a few other places that maybe the sql query being done may need some optimization work for large folder.
@Dragonatorul commented on GitHub (Oct 27, 2023):
I'm seeing the same issue. I'm on v2.3.3 because of the performance issues with 3.4
In my case it's happening when downloading new episodes for a very large podcast.
@paxchristos commented on GitHub (Oct 29, 2023):
Just an update, crashing still happening with the same errors on 2.5.0 (docker)
@jrhedman commented on GitHub (Oct 30, 2023):
@paxchristos
Try adding
NODE_OPTIONS=--max_old_space_size=8192back in your Environment Variables in your Docker Compose as mentioned in #1926.This was removed a few releases ago since the OOM Crash was mostly resolved outside of edge cases like this.
@paxchristos commented on GitHub (Oct 30, 2023):
I forgot that I left
NODE_OPTIONS=--max_old_space_size=1024in .env from testing before. Changing it from 1gb to 8gb allows library to load w/o crashing, but once I click into a ~3800 track podcast, it crashes w/FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory@jrhedman commented on GitHub (Oct 31, 2023):
Is this when trying to download a 3800 track podcast list or opening an existing library?
I'm thinking it has something to do with loading all the track details into memory.
Maybe we need to look into figuring out how to split this into batches since increasing space size for larger and larger libraries is a losing game.
@aunefyren commented on GitHub (Oct 31, 2023):
I managed to be able to load the library of my podcast (1500 episodes) by increasing
--max_old_space_size,but starting playback causes the socket to disconnect. Server is in debug mode, but basically nothing appears in the log. Network log on front end saysNS_BINDING_ABORTEDand finally502error.Is this related at all? I am resuming playback from the homepage. Any other podcasts start just fine.
@advplyr commented on GitHub (Oct 31, 2023):
@aunefyren That's unrelated since it's coming from cloudflare.
@advplyr commented on GitHub (Oct 31, 2023):
Actually if the server is crashing then it is Abs, but not sure why cloudflare would return a 502. Note that the logs from Abs won't show errors that aren't caught. You would have to see those by looking at the container logs
@aunefyren commented on GitHub (Oct 31, 2023):
I think when the Cloudflare proxy fails to connect to ABS (because it crashes) it gives a 502 error. If I disable Cloudflare proxy I just get
NS_BINDING_ABORTEDin the browser.Sorry if I was unclear about logs. The server is in debug mode and the container logs is very empty for errors:
In this timeframe I loaded the home page, resumed playback on a podcast with 1500 tracks, and had it stop syncing progress. Same behavior on Android app, except playback never starts.
I can open another issue or redirect my comment if this is unrelated.
@advplyr commented on GitHub (Jan 1, 2024):
Updates were made in v2.7.1 for large podcast libraries. I'm not sure if the OOM issues will be resolved since I didn't change the data model but let me know if you get a chance to test v2.7.1
@aunefyren commented on GitHub (Jan 1, 2024):
My playback issues are resolved.
@jrhedman commented on GitHub (Jan 3, 2024):
Just updated and tested it out on both Web and iOS app and have no OOM issues now. Podcasts load much faster now as well.
Tested this on 2 sets of 1k+ episode podcasts.
Thanks @advplyr
@jrhedman commented on GitHub (Jan 5, 2024):
@advplyr
A few day's have gone by and I've noticed I've gotten the OOM crash again, it looks like it's failing to clear the cache during parsing for new episodes to download.
@jucor commented on GitHub (Jan 18, 2024):
I'm on 2.7.2 and getting the same
JavaScript heap out of memoryerror. I have been loading a lot of things: audiobooks and a lot of files from my podcast collection. Log of the docker file attached. Any idea how to prevent it, please? I have lots more RAM to spare on that machine, so would be happy with a setting to bump up the allocated RAM._arr-audiobookshelf-1_logs.txt
I will try the environment variables from https://github.com/advplyr/audiobookshelf/issues/1926#issuecomment-1639020501 to tell node.js to augment its memory allocation, see if it helps.