starred/audiobookshelf
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2026-05-30 23:40:40 +02:00
Code Issues 1.0k Packages Projects Releases 10 Wiki Activity

[Bug]: OOM When trying to download all episodes from a large podcast #2350

New Issue
Closed
opened 2026-04-25 00:06:24 +02:00 by adam · 5 comments
adam commented 2026-04-25 00:06:24 +02:00
Owner
Copy Link
Copy Source

Originally created by @mikiher on GitHub (Nov 10, 2024).

What happened?

I'm opening this to capture this bug opened on audiobookshelf-windows, as I don't believe this is a Windows-specific bug.

I was able to reproduce it both on the official audiobookshelf-windows 2.16.2 as well on dev server built from latest source, on Windows 11.

What did you expect to happen?

Expecting server not to crash

Steps to reproduce the issue

  1. Create a Podcasts library in an empty folder
  2. Go the Library's Add page
  3. Search for "stuff you should know"
  4. Select the first result ("Stuff You Should Know" by "iHeartPodcasts")
  5. Click Submit
  6. Click the Find Episodes button (magnifying glass)
  7. Check Select all episodes
  8. Click Download 2437 episodes

Server crash on OOM

Audiobookshelf version

v2.16.2

How are you running audiobookshelf?

Built from source

What OS is your Audiobookshelf server hosted from?

Windows

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

[2024-11-10 17:52:41.422] INFO: [Watcher] Initializing watcher for "Podcasts".
[2024-11-10 17:52:41.422] DEBUG: [Watcher] Init watcher for library folder path "F:/Podcasts3"
[2024-11-10 17:52:41.424] INFO: [Watcher] "Podcasts" Ready
[2024-11-10 17:53:05.786] DEBUG: [ApiCacheManager] count: 0 size: 0
[2024-11-10 17:53:05.789] DEBUG: Loaded filterdata in 0.00s
[2024-11-10 17:53:05.790] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"}
[2024-11-10 17:53:05.810] DEBUG: [ApiCacheManager] count: 1 size: 1059
[2024-11-10 17:53:05.811] DEBUG: [ApiCacheManager] Cache hit: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"}
[2024-11-10 17:53:05.868] DEBUG: [ApiCacheManager] count: 1 size: 1059
[2024-11-10 17:53:05.873] DEBUG: Loaded 0 of 0 items for "Continue Listening/Reading" in 0.00s
[2024-11-10 17:53:05.877] DEBUG: Loaded 0 of 0 episodes for "Newest Episodes" in 0.00s
[2024-11-10 17:53:05.881] DEBUG: Loaded 0 of 0 items for "Recently Added" in 0.00s
[2024-11-10 17:53:05.885] DEBUG: Loaded 0 of 0 items for "Listen/Read Again" in 0.00s
[2024-11-10 17:53:05.885] DEBUG: Loaded 0 personalized shelves in 0.01s
[2024-11-10 17:53:05.885] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612/personalized?include=rssfeed,numEpisodesIncomplete,share"}
[2024-11-10 17:53:05.885] DEBUG: [ApiCacheManager] Caching with 1800000 ms TTL
[2024-11-10 17:53:07.697] DEBUG: [ApiCacheManager] count: 2 size: 1302
[2024-11-10 17:53:07.697] DEBUG: [ApiCacheManager] Cache hit: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"}
[2024-11-10 17:53:07.750] DEBUG: [ApiCacheManager] count: 2 size: 1302
[2024-11-10 17:53:07.752] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612/podcast-titles"}
[2024-11-10 17:54:04.427] DEBUG: [iTunes] Searching for podcast with term "stuff you should know"
[2024-11-10 17:54:04.514] DEBUG: [iTunes] Podcast search for "stuff you should know" returned 31 results
[2024-11-10 17:54:11.896] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss"
[2024-11-10 17:54:12.510] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" success - parsing xml
[2024-11-10 17:55:49.654] DEBUG: [fileUtils] Downloading file to F:/Podcasts3/Stuff You Should Know/cover
[2024-11-10 17:55:49.714] INFO: [CoverManager] Downloaded libraryItem cover "F:/Podcasts3/Stuff You Should Know/cover.jpg" from url "https://is1-ssl.mzstatic.com/image/thumb/Podcasts126/v4/17/a1/ce/17a1ce39-f248-8750-351a-0201206df213/mza_6600839420822557618.jpg/600x600bb.jpg" for "Stuff You Should Know"
[2024-11-10 17:55:49.715] DEBUG: [LibraryItem] Success saving abmetadata to "F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\metadata\items\77f96e6e-ca32-44e7-956c-08cf18f3a298\metadata.json"
[2024-11-10 17:55:49.735] DEBUG: [ApiCacheManager] libraryItem.afterCreate: Clearing cache
[2024-11-10 17:55:49.740] DEBUG: [ApiCacheManager] podcast.afterCreate: Clearing cache
[2024-11-10 17:55:49.867] DEBUG: [FfmpegHelpers] Resize Image Cmd: ffmpeg -i F:/Podcasts3/Stuff You Should Know/cover.jpg -y -vf scale=400:-1 F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\metadata\cache\covers\77f96e6e-ca32-44e7-956c-08cf18f3a298_400.webp
[2024-11-10 17:55:49.926] DEBUG: [FfmpegHelpers] Image resized Successfully
[2024-11-10 17:59:10.690] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss"
[2024-11-10 17:59:11.286] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" success - parsing xml

<--- Last few GCs --->

[59600:00000201AFD58790]   634233 ms: Mark-Compact 3672.1 (4128.4) -> 3667.1 (4142.7) MB, 44.05 / 0.00 ms  (average mu = 0.970, current mu = 0.868) allocation failure; scavenge might not succeed
[59600:00000201AFD58790]   634346 ms: Mark-Compact 3682.8 (4142.7) -> 3679.1 (4156.2) MB, 102.03 / 0.00 ms  (average mu = 0.884, current mu = 0.098) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF7EDEC6E7B node::SetCppgcReference+16075
 2: 00007FF7EDE3D996 v8::base::CPU::num_virtual_address_bits+79190
 3: 00007FF7EDE3FBA5 v8::base::CPU::num_virtual_address_bits+87909
 4: 00007FF7EE8AD9E1 v8::Isolate::ReportExternalAllocationLimitReached+65
 5: 00007FF7EE897178 v8::Function::Experimental_IsNopFunction+1336
 6: 00007FF7EE6F8AA0 v8::Platform::SystemClockTimeMillis+659328
 7: 00007FF7EE6F5B28 v8::Platform::SystemClockTimeMillis+647176
 8: 00007FF7EE70AE3A v8::Platform::SystemClockTimeMillis+733978
 9: 00007FF7EE70B6B7 v8::Platform::SystemClockTimeMillis+736151
10: 00007FF7EE7142EE v8::Platform::SystemClockTimeMillis+772046
11: 00007FF7EE724E3A v8::Platform::SystemClockTimeMillis+840474
12: 00007FF7EE729908 v8::Platform::SystemClockTimeMillis+859624
13: 00007FF7EE38FB0B v8::internal::Version::GetString+89803
14: 00007FF7EE610B7C v8::ObjectTemplate::IsImmutableProto+192316
15: 00007FF7EE61214C v8::ObjectTemplate::IsImmutableProto+197900
16: 00007FF7EE6143F7 v8::ObjectTemplate::IsImmutableProto+206775
17: 00007FF7EE61764E v8::base::time_internal::TimeBase<v8::base::TimeTicks>::operator!=+9694
18: 00007FF7EE6135E7 v8::ObjectTemplate::IsImmutableProto+203175
19: 00007FF7EE6143F7 v8::ObjectTemplate::IsImmutableProto+206775
20: 00007FF7EE614C42 v8::ObjectTemplate::IsImmutableProto+208898
21: 00007FF7EE617252 v8::base::time_internal::TimeBase<v8::base::TimeTicks>::operator!=+8674
22: 00007FF7EE83D83A v8::SharedValueConveyor::SharedValueConveyor+258874
23: 00007FF7EE95EEAE v8::PropertyDescriptor::writable+676878
24: 00007FF76ED73C66

Additional Notes

Following the server process memory usage on Task Manager, I can confirm it quickly goes up to ~4Gb before it crashes.

Originally created by @mikiher on GitHub (Nov 10, 2024). ### What happened? I'm opening this to capture [this bug](https://github.com/mikiher/audiobookshelf-windows/issues/27) opened on audiobookshelf-windows, as I don't believe this is a Windows-specific bug. I was able to reproduce it both on the official audiobookshelf-windows 2.16.2 as well on dev server built from latest source, on Windows 11. ### What did you expect to happen? Expecting server not to crash ### Steps to reproduce the issue 1. Create a Podcasts library in an empty folder 2. Go the Library's `Add` page 3. Search for "stuff you should know" 4. Select the first result ("Stuff You Should Know" by "iHeartPodcasts") 5. Click `Submit` 6. Click the `Find Episodes` button (magnifying glass) 7. Check `Select all episodes` 8. Click `Download 2437 episodes` Server crash on OOM ### Audiobookshelf version v2.16.2 ### How are you running audiobookshelf? Built from source ### What OS is your Audiobookshelf server hosted from? Windows ### If the issue is being seen in the UI, what browsers are you seeing the problem on? None ### Logs ```shell [2024-11-10 17:52:41.422] INFO: [Watcher] Initializing watcher for "Podcasts". [2024-11-10 17:52:41.422] DEBUG: [Watcher] Init watcher for library folder path "F:/Podcasts3" [2024-11-10 17:52:41.424] INFO: [Watcher] "Podcasts" Ready [2024-11-10 17:53:05.786] DEBUG: [ApiCacheManager] count: 0 size: 0 [2024-11-10 17:53:05.789] DEBUG: Loaded filterdata in 0.00s [2024-11-10 17:53:05.790] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"} [2024-11-10 17:53:05.810] DEBUG: [ApiCacheManager] count: 1 size: 1059 [2024-11-10 17:53:05.811] DEBUG: [ApiCacheManager] Cache hit: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"} [2024-11-10 17:53:05.868] DEBUG: [ApiCacheManager] count: 1 size: 1059 [2024-11-10 17:53:05.873] DEBUG: Loaded 0 of 0 items for "Continue Listening/Reading" in 0.00s [2024-11-10 17:53:05.877] DEBUG: Loaded 0 of 0 episodes for "Newest Episodes" in 0.00s [2024-11-10 17:53:05.881] DEBUG: Loaded 0 of 0 items for "Recently Added" in 0.00s [2024-11-10 17:53:05.885] DEBUG: Loaded 0 of 0 items for "Listen/Read Again" in 0.00s [2024-11-10 17:53:05.885] DEBUG: Loaded 0 personalized shelves in 0.01s [2024-11-10 17:53:05.885] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612/personalized?include=rssfeed,numEpisodesIncomplete,share"} [2024-11-10 17:53:05.885] DEBUG: [ApiCacheManager] Caching with 1800000 ms TTL [2024-11-10 17:53:07.697] DEBUG: [ApiCacheManager] count: 2 size: 1302 [2024-11-10 17:53:07.697] DEBUG: [ApiCacheManager] Cache hit: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612?include=filterdata"} [2024-11-10 17:53:07.750] DEBUG: [ApiCacheManager] count: 2 size: 1302 [2024-11-10 17:53:07.752] DEBUG: [ApiCacheManager] Cache miss: {"user":"root","url":"/libraries/2f9b22d3-3d6e-4134-9f95-42d975198612/podcast-titles"} [2024-11-10 17:54:04.427] DEBUG: [iTunes] Searching for podcast with term "stuff you should know" [2024-11-10 17:54:04.514] DEBUG: [iTunes] Podcast search for "stuff you should know" returned 31 results [2024-11-10 17:54:11.896] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" [2024-11-10 17:54:12.510] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" success - parsing xml [2024-11-10 17:55:49.654] DEBUG: [fileUtils] Downloading file to F:/Podcasts3/Stuff You Should Know/cover [2024-11-10 17:55:49.714] INFO: [CoverManager] Downloaded libraryItem cover "F:/Podcasts3/Stuff You Should Know/cover.jpg" from url "https://is1-ssl.mzstatic.com/image/thumb/Podcasts126/v4/17/a1/ce/17a1ce39-f248-8750-351a-0201206df213/mza_6600839420822557618.jpg/600x600bb.jpg" for "Stuff You Should Know" [2024-11-10 17:55:49.715] DEBUG: [LibraryItem] Success saving abmetadata to "F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\metadata\items\77f96e6e-ca32-44e7-956c-08cf18f3a298\metadata.json" [2024-11-10 17:55:49.735] DEBUG: [ApiCacheManager] libraryItem.afterCreate: Clearing cache [2024-11-10 17:55:49.740] DEBUG: [ApiCacheManager] podcast.afterCreate: Clearing cache [2024-11-10 17:55:49.867] DEBUG: [FfmpegHelpers] Resize Image Cmd: ffmpeg -i F:/Podcasts3/Stuff You Should Know/cover.jpg -y -vf scale=400:-1 F:\VS Code Worksapces\audiobookshelf-local\audiobookshelf\metadata\cache\covers\77f96e6e-ca32-44e7-956c-08cf18f3a298_400.webp [2024-11-10 17:55:49.926] DEBUG: [FfmpegHelpers] Image resized Successfully [2024-11-10 17:59:10.690] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" [2024-11-10 17:59:11.286] DEBUG: [podcastUtils] getPodcastFeed for "https://www.omnycontent.com/d/playlist/e73c998e-6e60-432f-8610-ae210140c5b1/a91018a4-ea4f-4130-bf55-ae270180c327/44710ecc-10bb-48d1-93c7-ae270180c33e/podcast.rss" success - parsing xml <--- Last few GCs ---> [59600:00000201AFD58790] 634233 ms: Mark-Compact 3672.1 (4128.4) -> 3667.1 (4142.7) MB, 44.05 / 0.00 ms (average mu = 0.970, current mu = 0.868) allocation failure; scavenge might not succeed [59600:00000201AFD58790] 634346 ms: Mark-Compact 3682.8 (4142.7) -> 3679.1 (4156.2) MB, 102.03 / 0.00 ms (average mu = 0.884, current mu = 0.098) allocation failure; scavenge might not succeed <--- JS stacktrace ---> FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory ----- Native stack trace ----- 1: 00007FF7EDEC6E7B node::SetCppgcReference+16075 2: 00007FF7EDE3D996 v8::base::CPU::num_virtual_address_bits+79190 3: 00007FF7EDE3FBA5 v8::base::CPU::num_virtual_address_bits+87909 4: 00007FF7EE8AD9E1 v8::Isolate::ReportExternalAllocationLimitReached+65 5: 00007FF7EE897178 v8::Function::Experimental_IsNopFunction+1336 6: 00007FF7EE6F8AA0 v8::Platform::SystemClockTimeMillis+659328 7: 00007FF7EE6F5B28 v8::Platform::SystemClockTimeMillis+647176 8: 00007FF7EE70AE3A v8::Platform::SystemClockTimeMillis+733978 9: 00007FF7EE70B6B7 v8::Platform::SystemClockTimeMillis+736151 10: 00007FF7EE7142EE v8::Platform::SystemClockTimeMillis+772046 11: 00007FF7EE724E3A v8::Platform::SystemClockTimeMillis+840474 12: 00007FF7EE729908 v8::Platform::SystemClockTimeMillis+859624 13: 00007FF7EE38FB0B v8::internal::Version::GetString+89803 14: 00007FF7EE610B7C v8::ObjectTemplate::IsImmutableProto+192316 15: 00007FF7EE61214C v8::ObjectTemplate::IsImmutableProto+197900 16: 00007FF7EE6143F7 v8::ObjectTemplate::IsImmutableProto+206775 17: 00007FF7EE61764E v8::base::time_internal::TimeBase<v8::base::TimeTicks>::operator!=+9694 18: 00007FF7EE6135E7 v8::ObjectTemplate::IsImmutableProto+203175 19: 00007FF7EE6143F7 v8::ObjectTemplate::IsImmutableProto+206775 20: 00007FF7EE614C42 v8::ObjectTemplate::IsImmutableProto+208898 21: 00007FF7EE617252 v8::base::time_internal::TimeBase<v8::base::TimeTicks>::operator!=+8674 22: 00007FF7EE83D83A v8::SharedValueConveyor::SharedValueConveyor+258874 23: 00007FF7EE95EEAE v8::PropertyDescriptor::writable+676878 24: 00007FF76ED73C66 ``` ### Additional Notes Following the server process memory usage on Task Manager, I can confirm it quickly goes up to ~4Gb before it crashes.
adam added the bug label 2026-04-25 00:06:24 +02:00
adam closed this issue 2026-04-25 00:06:24 +02:00
adam commented 2026-04-25 00:06:25 +02:00
Author
Owner
Copy Link
Copy Source

@nichwall commented on GitHub (Nov 10, 2024):

Related to https://github.com/advplyr/audiobookshelf/issues/3338

@nichwall commented on GitHub (Nov 10, 2024): Related to https://github.com/advplyr/audiobookshelf/issues/3338
adam commented 2026-04-25 00:06:25 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 11, 2024):

Doing a search in issues that specific podcast has caused a few OOM crashes. Here is one https://github.com/advplyr/audiobookshelf/issues/2075#issuecomment-1879289014

I did a quick test and the issue is that the download queue in PodcastManager is storing large objects (PodcastEpisodeDownload which contains the PodcastEpisode).
OOM when adding episode 2260 to the queue.

image

We can fix this by refactoring the queue so that it only stores the fields from the RSS feed that are sent in by the request, and construct the PodcastEpisode when the download starts for that episode. This might make the PodcastEpisodeDownload class unnecessary.

@advplyr commented on GitHub (Nov 11, 2024): Doing a search in issues that specific podcast has caused a few OOM crashes. Here is one https://github.com/advplyr/audiobookshelf/issues/2075#issuecomment-1879289014 I did a quick test and the issue is that the download queue in `PodcastManager` is storing large objects (`PodcastEpisodeDownload` which contains the `PodcastEpisode`). OOM when adding episode 2260 to the queue. ![image](https://github.com/user-attachments/assets/4d162ef1-1f37-4830-bacd-a0d4fb0517de) We can fix this by refactoring the queue so that it only stores the fields from the RSS feed that are sent in by the request, and construct the `PodcastEpisode` when the download starts for that episode. This might make the `PodcastEpisodeDownload` class unnecessary.
adam commented 2026-04-25 00:06:25 +02:00
Author
Owner
Copy Link
Copy Source

@mikiher commented on GitHub (Nov 11, 2024):

I also checked, and I think that is not the actual issue.

I think the actual issue is emitting episode_download_queue_updated with the value of this.getDownloadQueueDetails() every time we queue a download.

I believe this causes creation of O(n^2) large objects in quick succession, before they're even sent over the socket, which causes the crash.

If you remove this emit call, or put it after the loop that queues all the downloads, the OOM issue is gone.

@mikiher commented on GitHub (Nov 11, 2024): I also checked, and I think that is not the actual issue. I think the actual issue is emitting `episode_download_queue_updated` with the value of `this.getDownloadQueueDetails()` every time we queue a download. I believe this causes creation of O(n^2) large objects in quick succession, before they're even sent over the socket, which causes the crash. If you remove this emit call, or put it after the loop that queues all the downloads, the OOM issue is gone.
adam commented 2026-04-25 00:06:25 +02:00
Author
Owner
Copy Link
Copy Source

@advplyr commented on GitHub (Nov 11, 2024):

Yep removing that it only takes up ~250MB of heap. The socket event should definitely not be sending the entire queue.

@advplyr commented on GitHub (Nov 11, 2024): Yep removing that it only takes up ~250MB of heap. The socket event should definitely not be sending the entire queue.
adam commented 2026-04-25 00:06:26 +02:00
Author
Owner
Copy Link
Copy Source

@github-actions[bot] commented on GitHub (Nov 17, 2024):

Fixed in v2.17.0.

@github-actions[bot] commented on GitHub (Nov 17, 2024): Fixed in [v2.17.0](https://github.com/advplyr/audiobookshelf/releases/tag/v2.17.0).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
authentication
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
pull-request
Mirrored from GitHub Pull Request
 sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam (Adam Melkus)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#2350
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?