Speech Noise Reduction: Technical Challenges and Browser-Based Methods

The part that surprised us first

When we started working on Remove Audio Noise, the tempting mental model was simple: find the noise, turn it down, keep the voice. Real recordings are less polite. A voice note is usually a pile-up of speech, room tone, microphone color, compression artifacts, and whatever was happening near the laptop fan that day.

The hardest decision is not whether something sounds bad. The hard decision is whether reducing it will also shave off consonants, breath, or the natural edge of the speaker's voice. A fan hum changes slowly, so it is forgiving. Keyboard clicks, music, crowd noise, and another person speaking are much less forgiving because they live close to the same frequencies as speech.

Why we process speech in frames

We do not treat an audio file as one giant object and ask a single question about it. The cleaner works on short runs of PCM samples. For each run, the engine has to estimate what looks like voice, what looks like background, and how much suppression is safe.

That frame-by-frame approach is practical in the browser, but it creates its own personality. React too quickly and the audio can sound nervous. Smooth too much and the processor feels late when the recording changes. The job is to keep the voice stable while still responding to the room around it.

The pipeline we ended up with

The local pipeline is intentionally boring in the best way. Every step happens on the user's device:

1. Read the file the user picked.
2. Decode MP3, M4A, AAC, OGG, or FLAC when the browser can decode that format.
3. Keep the original sample rate, or convert to the output rate the user chooses.
4. Feed PCM frames into the speech noise reduction engine.
5. Optionally apply adaptive gain when the voice level jumps around.
6. Write the processed samples into a 16-bit WAV file.

The important bit is what is not in that list: uploading the recording. The denoising engine runs locally with WebAssembly, and the output is built locally as a WAV file. That choice makes some engineering tasks harder, but it matches the promise we care about most: private audio cleanup without an account or a server-side copy.

Why there are only three strength levels

We could have exposed a panel full of knobs. We chose not to. Most people cleaning a meeting recording or podcast clip do not want to tune a speech enhancement system; they want to know whether low, medium, or high sounds better.

Low keeps more of the original texture. Medium is the setting we expect most voice recordings to start with. High is useful when steady background noise is obvious, but it can also make artifacts more noticeable. Those artifacts usually sound like watery motion, robotic edges, missing consonants, or volume pumping. Preview matters because the right answer depends on the recording, not on a universal rule.

Formats are where browsers keep us honest

WAV is predictable. MP3 is usually fine. The rest depends more on the browser and operating system than many users expect. M4A, AAC, FLAC, and OGG support is not identical everywhere, so the app has to check what the browser can actually decode instead of pretending every format works the same way.

Sample rate is another place where small decisions matter. Keeping the original rate avoids extra conversion. Converting to 8, 16, 32, or 48 kHz is useful for compatibility or smaller output, but resampling must stay continuous across chunks. If it does not, the result can drift or click at boundaries, which is exactly the kind of tiny defect that makes an otherwise clean file feel wrong.

Where this works best

The tool is built for practical speech cleanup: podcasts, interviews, lectures, meetings, voice notes, narration, and remote calls. If the background sound is steady and the voice is still clearly present, local denoising can make the recording much easier to listen to.

We do not think of it as forensic restoration or music mastering. If the voice is buried under another voice, distorted beyond recognition, or fighting a loud music bed, there is only so much a browser tool should promise. For everyday spoken audio, though, a private first pass in the browser is often enough to turn a rough recording into something usable.

Try it locally

You can use Remove Audio Noise to clean speech recordings directly in your browser. We built it so files are processed locally with WebAssembly and are not uploaded to a server.