[Enhancement]: Smart speed / trim silence #2330

Open
opened 2026-04-25 00:06:10 +02:00 by adam · 12 comments
Owner

Originally created by @DankMemeGuy on GitHub (Oct 28, 2024).

Type of Enhancement

None

Describe the Feature/Enhancement

Podcasting software "Overcast" includes a SmartSpeed feature that intelligently trims pauses in audio playback. This feature can result in a time savings of approximately 10-20% during playback. For podcasts or audiobooks, enabling this could significantly reduce listening time without altering the natural flow of the audio.

You can read more here https://medium.com/@eped/overcasts-smart-speed-vs-real-time-a759549ab48b

Why would this be helpful?

This feature would enhance the user experience by allowing listeners to consume content more efficiently, saving time while maintaining audio quality. It’s particularly useful for lengthy content like audiobooks and long-form podcasts, where small pauses add up, making the listening process quicker and more seamless.

Future Implementation (Screenshot)

Not needed

Audiobookshelf Server Version

2.16

Current Implementation (Screenshot)

No response

Originally created by @DankMemeGuy on GitHub (Oct 28, 2024). ### Type of Enhancement None ### Describe the Feature/Enhancement Podcasting software "Overcast" includes a SmartSpeed feature that intelligently trims pauses in audio playback. This feature can result in a time savings of approximately 10-20% during playback. For podcasts or audiobooks, enabling this could significantly reduce listening time without altering the natural flow of the audio. You can read more here https://medium.com/@eped/overcasts-smart-speed-vs-real-time-a759549ab48b ### Why would this be helpful? This feature would enhance the user experience by allowing listeners to consume content more efficiently, saving time while maintaining audio quality. It’s particularly useful for lengthy content like audiobooks and long-form podcasts, where small pauses add up, making the listening process quicker and more seamless. ### Future Implementation (Screenshot) Not needed ### Audiobookshelf Server Version 2.16 ### Current Implementation (Screenshot) _No response_
adam added the enhancement label 2026-04-25 00:06:10 +02:00
Author
Owner

@pwinnski commented on GitHub (Oct 30, 2024):

That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only.

All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast

@pwinnski commented on GitHub (Oct 30, 2024): That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only. All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast
Author
Owner

@DankMemeGuy commented on GitHub (Nov 1, 2024):

That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only.

All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast

I think it was more challenging in 2015 when he was building out his software, but times have changed. When using ChatGPT, this is what it suggested:

To implement an on-the-fly "Smart Speed" feature within AudioBookShelf, we’d ideally modify its playback engine to handle dynamic silence reduction during streaming rather than preprocessing the audio file. Since AudioBookShelf is a Node.js application, we'll focus on integrating this functionality within a Node.js audio pipeline.

Here's how you might go about adding this feature using ffmpeg to process audio in real-time. The following example script uses ffmpeg to dynamically remove silence and stream audio back to the player.

Prerequisites

Ensure ffmpeg is installed on your server, as it will perform real-time audio processing. You can install it using:

sudo apt-get install ffmpeg

Real-Time Silence Reduction in Node.js

Here’s an example of how you could add real-time silence reduction to AudioBookShelf's playback by creating a middleware function to process the audio stream. This approach hooks into the stream, detects silence, and trims it in real time before streaming the output to the player.

const { spawn } = require('child_process');
const express = require('express');
const fs = require('fs');
const app = express();

app.get('/stream/:audioFile', (req, res) => {
    const audioFilePath = `path/to/audiobooks/${req.params.audioFile}`;

    // Spawn an FFmpeg process to reduce silence dynamically
    const ffmpeg = spawn('ffmpeg', [
        '-i', audioFilePath,             // Input file
        '-af', 'silenceremove=start_periods=1:start_silence=0.5:start_threshold=-40dB:detection=peak', // Silence removal filter
        '-f', 'mp3',                     // Output format
        'pipe:1'                         // Output to stdout for streaming
    ]);

    // Set response headers
    res.setHeader('Content-Type', 'audio/mpeg');
    
    // Pipe FFmpeg output to the response
    ffmpeg.stdout.pipe(res);

    // Handle errors
    ffmpeg.stderr.on('data', (data) => {
        console.error(`FFmpeg error: ${data}`);
    });

    // Close response when FFmpeg finishes
    ffmpeg.on('close', () => {
        res.end();
    });

    // Handle client disconnects
    req.on('close', () => {
        ffmpeg.kill('SIGINT');
    });
});

// Start server
app.listen(3000, () => {
    console.log('AudioBookShelf streaming server is running on port 3000');
});

Explanation of the Code

  1. FFmpeg Audio Filter: We use the silenceremove filter from FFmpeg, which detects and removes silence based on customizable thresholds:

    • start_periods=1: Starts trimming after detecting one period of silence.
    • start_silence=0.5: Defines silence as any pause lasting 0.5 seconds or longer.
    • start_threshold=-40dB: Treats segments quieter than -40 dB as silence.
    • detection=peak: Uses peak values for detecting silence.
  2. Streaming with Express: This code sets up an Express route (/stream/:audioFile) to serve processed audio on-the-fly. FFmpeg streams directly to the client by piping its stdout to the HTTP response.

  3. Handling Connections: The server closes the FFmpeg process if the client disconnects (req.on('close')), saving resources.

Integration into AudioBookShelf

To integrate this directly into AudioBookShelf, you would:

  • Modify its existing audio streaming routes to pass audio files through this ffmpeg processing stream.
  • Ensure that your server can handle the CPU load, as real-time audio processing can be resource-intensive.

This approach avoids the need to pre-process entire files while providing an on-the-fly "Smart Speed" feature similar to Overcast's Smart Speed. This setup is suitable for streaming-based applications and works well within Node.js-based frameworks like AudioBookShelf.

Relevant:

  1. https://ffmpeg.org/ffmpeg-filters.html#silencedetect
  2. https://ffmpeg.org/ffmpeg-filters.html#silenceremove

That would be a streaming option, but a pre-processed option would be less intensive, but more storage (assuming the original file is kept intact). Unless I guess having an option at the time of upload on whether to trim silence, and whether to preserve the original or not. That would save the complexity of re-writing the audio engine since it would just need that FFMPEG pipeline at upload stage

@DankMemeGuy commented on GitHub (Nov 1, 2024): > That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only. > > All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast I think it was more challenging in 2015 when he was building out his software, but times have changed. When using ChatGPT, this is what it suggested: To implement an on-the-fly "Smart Speed" feature within [AudioBookShelf](https://github.com/advplyr/audiobookshelf), we’d ideally modify its playback engine to handle dynamic silence reduction during streaming rather than preprocessing the audio file. Since AudioBookShelf is a Node.js application, we'll focus on integrating this functionality within a Node.js audio pipeline. Here's how you might go about adding this feature using `ffmpeg` to process audio in real-time. The following example script uses `ffmpeg` to dynamically remove silence and stream audio back to the player. ### Prerequisites Ensure `ffmpeg` is installed on your server, as it will perform real-time audio processing. You can install it using: ```bash sudo apt-get install ffmpeg ``` ### Real-Time Silence Reduction in Node.js Here’s an example of how you could add real-time silence reduction to AudioBookShelf's playback by creating a middleware function to process the audio stream. This approach hooks into the stream, detects silence, and trims it in real time before streaming the output to the player. ```javascript const { spawn } = require('child_process'); const express = require('express'); const fs = require('fs'); const app = express(); app.get('/stream/:audioFile', (req, res) => { const audioFilePath = `path/to/audiobooks/${req.params.audioFile}`; // Spawn an FFmpeg process to reduce silence dynamically const ffmpeg = spawn('ffmpeg', [ '-i', audioFilePath, // Input file '-af', 'silenceremove=start_periods=1:start_silence=0.5:start_threshold=-40dB:detection=peak', // Silence removal filter '-f', 'mp3', // Output format 'pipe:1' // Output to stdout for streaming ]); // Set response headers res.setHeader('Content-Type', 'audio/mpeg'); // Pipe FFmpeg output to the response ffmpeg.stdout.pipe(res); // Handle errors ffmpeg.stderr.on('data', (data) => { console.error(`FFmpeg error: ${data}`); }); // Close response when FFmpeg finishes ffmpeg.on('close', () => { res.end(); }); // Handle client disconnects req.on('close', () => { ffmpeg.kill('SIGINT'); }); }); // Start server app.listen(3000, () => { console.log('AudioBookShelf streaming server is running on port 3000'); }); ``` ### Explanation of the Code 1. **FFmpeg Audio Filter**: We use the `silenceremove` filter from FFmpeg, which detects and removes silence based on customizable thresholds: - `start_periods=1`: Starts trimming after detecting one period of silence. - `start_silence=0.5`: Defines silence as any pause lasting 0.5 seconds or longer. - `start_threshold=-40dB`: Treats segments quieter than -40 dB as silence. - `detection=peak`: Uses peak values for detecting silence. 2. **Streaming with Express**: This code sets up an Express route (`/stream/:audioFile`) to serve processed audio on-the-fly. FFmpeg streams directly to the client by piping its `stdout` to the HTTP response. 3. **Handling Connections**: The server closes the FFmpeg process if the client disconnects (`req.on('close')`), saving resources. ### Integration into AudioBookShelf To integrate this directly into AudioBookShelf, you would: - Modify its existing audio streaming routes to pass audio files through this `ffmpeg` processing stream. - Ensure that your server can handle the CPU load, as real-time audio processing can be resource-intensive. This approach avoids the need to pre-process entire files while providing an on-the-fly "Smart Speed" feature similar to Overcast's Smart Speed. This setup is suitable for streaming-based applications and works well within Node.js-based frameworks like AudioBookShelf. Relevant: 1. https://ffmpeg.org/ffmpeg-filters.html#silencedetect 2. https://ffmpeg.org/ffmpeg-filters.html#silenceremove That would be a streaming option, but a pre-processed option would be less intensive, but more storage (assuming the original file is kept intact). Unless I guess having an option at the time of upload on whether to trim silence, and whether to preserve the original or not. That would save the complexity of re-writing the audio engine since it would just need that FFMPEG pipeline at upload stage
Author
Owner

@TaylorMichaelHall commented on GitHub (Nov 5, 2024):

That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only.

All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast

Pocketcasts has been doing this for about as long - https://support.pocketcasts.com/knowledge-base/playback-effects/#htoc-trim-silence

@TaylorMichaelHall commented on GitHub (Nov 5, 2024): > That feature is the result of Marco Arment having developed a custom audio engine for iOS. It is 100% a mobile client feature, and the result of a unique and huge effort by a very talented developer. It is also very much iOS/iPadOS/WatchOS only. > > All that to say, there's a reason that Overcast is the only app that does that, even ten years later! https://marco.org/2014/07/16/overcast Pocketcasts has been doing this for about as long - https://support.pocketcasts.com/knowledge-base/playback-effects/#htoc-trim-silence
Author
Owner

@Terminal-Grasshopper commented on GitHub (Jan 10, 2025):

this issue should be renamed to "trim silence" rather than "smart speed"

and yes it is an amazing feature in pocketcasts, UI/UX can be found in the link @TaylorMichaelHall inserted

@Terminal-Grasshopper commented on GitHub (Jan 10, 2025): this issue should be renamed to "trim silence" rather than "smart speed" and yes it is an amazing feature in pocketcasts, UI/UX can be found in the link @TaylorMichaelHall inserted
Author
Owner

@smeinecke commented on GitHub (Apr 19, 2025):

NewPipe (alternative android youtube client) has the same feature (named SkipSilence).

@smeinecke commented on GitHub (Apr 19, 2025): NewPipe (alternative android youtube client) has the same feature (named `SkipSilence`).
Author
Owner

@jonbaldie commented on GitHub (May 15, 2025):

Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests:

Image

I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something?

cc. @DankMemeGuy @TaylorMichaelHall et. al

@jonbaldie commented on GitHub (May 15, 2025): Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests: <img width="512" alt="Image" src="https://github.com/user-attachments/assets/1ac4846b-d28b-4f79-9c2c-76974a2164ab" /> I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something? cc. @DankMemeGuy @TaylorMichaelHall et. al
Author
Owner

@nichwall commented on GitHub (May 15, 2025):

I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something?

You need to do your work in a fork and open a PR from your fork.

@nichwall commented on GitHub (May 15, 2025): > I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something? > You need to do your work in a fork and open a PR from your fork.
Author
Owner

@jonbaldie commented on GitHub (May 16, 2025):

@nichwall good to know, thanks. I'm a little busy today so I'll do this next week.

@jonbaldie commented on GitHub (May 16, 2025): @nichwall good to know, thanks. I'm a little busy today so I'll do this next week.
Author
Owner

@DankMemeGuy commented on GitHub (Jul 31, 2025):

Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests:

Image I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something?

cc. @DankMemeGuy @TaylorMichaelHall et. al

This is wonderful! I'm glad I checked back in on this. I was listening to the books over the summer and thought of the feature again. Looking forward to it

@DankMemeGuy commented on GitHub (Jul 31, 2025): > Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests: > > <img alt="Image" width="512" src="https://private-user-images.githubusercontent.com/8376953/444160340-1ac4846b-d28b-4f79-9c2c-76974a2164ab.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM5OTY4MjksIm5iZiI6MTc1Mzk5NjUyOSwicGF0aCI6Ii84Mzc2OTUzLzQ0NDE2MDM0MC0xYWM0ODQ2Yi1kMjhiLTRmNzktOWMyYy03Njk3NGEyMTY0YWIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDczMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA3MzFUMjExNTI5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Yjc2MzE2YmVjNDM5NTMwOWRlNzIyYzQzMDhlYjNkMmM5YjRhNzY0MzI1N2YwN2E3NjRlMDllZTdjYmU1Mjc4MSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.OSNvQkN0SMjmMuAUZWlESUkQkn7N2MQ59sWTDj6P5cQ"> > I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something? > > cc. [@DankMemeGuy](https://github.com/DankMemeGuy) [@TaylorMichaelHall](https://github.com/TaylorMichaelHall) et. al This is wonderful! I'm glad I checked back in on this. I was listening to the books over the summer and thought of the feature again. Looking forward to it
Author
Owner

@timespacedecay commented on GitHub (Aug 6, 2025):

@nichwall good to know, thanks. I'm a little busy today so I'll do this next week.

🙏 Is this still something that can be implemented? This is such a great feature, I hope so! Thanks for your time and effort on it!

@timespacedecay commented on GitHub (Aug 6, 2025): > [@nichwall](https://github.com/nichwall) good to know, thanks. I'm a little busy today so I'll do this next week. 🙏 Is this still something that can be implemented? This is such a great feature, I hope so! Thanks for your time and effort on it!
Author
Owner

@scrollop commented on GitHub (Feb 26, 2026):

Is it possible

Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests:
Image

I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something?

cc. @DankMemeGuy @TaylorMichaelHall et. al

Could you share the fork?

@scrollop commented on GitHub (Feb 26, 2026): Is it possible > Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests: > <img alt="Image" width="512" src="https://private-user-images.githubusercontent.com/8376953/444160340-1ac4846b-d28b-4f79-9c2c-76974a2164ab.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzIxMDAyNzMsIm5iZiI6MTc3MjA5OTk3MywicGF0aCI6Ii84Mzc2OTUzLzQ0NDE2MDM0MC0xYWM0ODQ2Yi1kMjhiLTRmNzktOWMyYy03Njk3NGEyMTY0YWIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMjZUMDk1OTMzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjJhMDI1ZTU2MTMxMTcyOTk0NjAzNjYwZTViNjU5NjAzZmUxN2FmNjRjYmViNDA4MzBjNTFlNDcxZDYyMThmZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.CdCNJzsz39uWcRQ79UCAQK-8JLLOSFoboBq9wKGU7yQ"> > > I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something? > > cc. [@DankMemeGuy](https://github.com/DankMemeGuy) [@TaylorMichaelHall](https://github.com/TaylorMichaelHall) et. al Could you share the fork?
Author
Owner

@corbin-zip commented on GitHub (Mar 14, 2026):

@scrollop

Is it possible

Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests:
Image
I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something?
cc. @DankMemeGuy @TaylorMichaelHall et. al

Could you share the fork?

fork is on the user's profile here: https://github.com/jonbaldie/audiobookshelf/tree/feature/smart-speed-toggle

commits are available here: https://github.com/advplyr/audiobookshelf/compare/master...jonbaldie:audiobookshelf:feature/smart-speed-toggle

I would really love to see this feature! But I don't personally have the time to repackage @jonbaldie 's work and submit a PR (and I have essentially no experience with JavaScript). Perhaps the author or someone else on this thread has the time.

This is the only feature keeping me on AntennaPod for my podcast listening so I'd love to see this implemented.

Corbin

@corbin-zip commented on GitHub (Mar 14, 2026): @scrollop > Is it possible > > > Good news all, I've actually developed this feature in a local copy of the branch and it seems to work well in my tests: > > <img alt="Image" width="512" src="https://private-user-images.githubusercontent.com/8376953/444160340-1ac4846b-d28b-4f79-9c2c-76974a2164ab.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NzIxMDAyNzMsIm5iZiI6MTc3MjA5OTk3MywicGF0aCI6Ii84Mzc2OTUzLzQ0NDE2MDM0MC0xYWM0ODQ2Yi1kMjhiLTRmNzktOWMyYy03Njk3NGEyMTY0YWIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDIyNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjAyMjZUMDk1OTMzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjJhMDI1ZTU2MTMxMTcyOTk0NjAzNjYwZTViNjU5NjAzZmUxN2FmNjRjYmViNDA4MzBjNTFlNDcxZDYyMThmZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.CdCNJzsz39uWcRQ79UCAQK-8JLLOSFoboBq9wKGU7yQ"> > > I actually tried to push up to a feature branch and open a PR but I got a 403, does the maintainer need to activate write permissions per GitHub user or something? > > cc. [@DankMemeGuy](https://github.com/DankMemeGuy) [@TaylorMichaelHall](https://github.com/TaylorMichaelHall) et. al > > Could you share the fork? fork is on the user's profile here: https://github.com/jonbaldie/audiobookshelf/tree/feature/smart-speed-toggle commits are available here: https://github.com/advplyr/audiobookshelf/compare/master...jonbaldie:audiobookshelf:feature/smart-speed-toggle I would really love to see this feature! But I don't personally have the time to repackage @jonbaldie 's work and submit a PR (and I have essentially no experience with JavaScript). Perhaps the author or someone else on this thread has the time. This is the only feature keeping me on AntennaPod for my podcast listening so I'd love to see this implemented. Corbin
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/audiobookshelf#2330