Skip to content

Feature · WebRTC voice activity detection

WebRTC voice activity detection for voice agents.

WebRTC VAD is a fast, open-source tool that separates speech from silence. PyAI endpoints the conversation on its own servers.

Get a sandbox key$50 credit. No credit card required.
10, 20 or 30 ms audio frames8 to 48 kHz 16-bit mono PCMAggressiveness modes 0 to 3PyAI endpoints server side at $0.001/min

What PyAI gives you for WebRTC voice activity detection

  • WebRTC voice activity detection runs a Gaussian mixture model over the 80 Hz to 4 kHz energy bands and labels 10, 20 or 30 millisecond frames of 16-bit mono PCM, sampled at 8, 16, 32 or 48 kHz, as speech or not speech.
  • WebRTC VAD aggressiveness runs from mode 0 to mode 3. Mode 3 drops the most background noise but can clip soft consonants, while mode 0 keeps quiet speech and lets more noise through.
  • Developers reach WebRTC VAD through community bindings such as py-webrtcvad in Python and Swift wrappers on iOS. WebRTC VAD is fast and light, but some neural detectors can be more accurate on noisy audio in Picovoice's published noise benchmark.
  • PyAI does not use WebRTC VAD. PyAI endpoints the audio on its own servers and sends the end of turn about 196 ms after a user stops, with about 390 ms voice-to-voice, measured in region.

How does WebRTC voice activity detection work, step by step?

WebRTC VAD runs on the raw audio before anything else in a voice pipeline. On PyAI the same job runs server side, so you do not host it.

Audio is split into frames

The incoming PCM audio is cut into fixed 10, 20 or 30 millisecond frames, each one 16-bit mono at 8, 16, 32 or 48 kHz.

Each frame is scored

The Gaussian mixture model weighs energy across the 80 Hz to 4 kHz bands and returns a speech or non-speech label for that frame.

The mode sets the threshold

An aggressiveness mode from 0 to 3 decides how much noise is filtered. A higher mode rejects more noise and risks clipping soft speech.

Speech is handed on

Frames marked as speech pass to the next step, usually speech-to-text. On PyAI, PyAI endpoints the stream itself and returns speech events, so you skip this setup.

Where is WebRTC voice activity detection used?

WebRTC VAD shows up wherever audio has to be filtered before it is sent or processed.

Bandwidth

Dropping silent packets

VoIP and WebRTC apps use it to avoid encoding and sending silent audio, sometimes with discontinuous transmission, which saves bandwidth on a voice or video session.

Voice agents

Before speech-to-text

In a build-it-yourself stack it runs before speech-to-text, so the pipeline can tell when a user starts and stops speaking for turn-taking and barge-in.

Devices

Auto-mute and push-to-talk

Apps use it to drive a mute indicator or open the microphone only while someone is speaking, so background noise is not sent.

Before PyAI

Client-side gating

You can still run WebRTC VAD in a browser to hold back silent audio before you stream to PyAI, while PyAI endpoints the speech it receives.

WebRTC voice activity detection questions people ask

How does WebRTC VAD work?

WebRTC VAD cuts audio into 10, 20 or 30 millisecond frames and scores each one with a Gaussian mixture model across the 80 Hz to 4 kHz bands. An aggressiveness mode from 0 to 3 then sets how much background noise it rejects before it marks a frame as speech or silence.

What are WebRTC VAD aggressiveness modes?

Aggressiveness is a setting from 0 to 3. Mode 0 is the most permissive and keeps quiet speech, but lets more noise through. Mode 3 filters the most noise and suits clean audio, though it can clip soft consonants or trailing words. Most apps test a few modes to find the right fit.

Which fits better, WebRTC VAD or a neural VAD?

WebRTC VAD is lightweight, so it can suit low-resource or real-time use. Some neural detectors can be more accurate on noisy audio; Picovoice's benchmark illustrates why the result depends on the model and detection threshold. The right pick depends on how clean your audio is and how much compute you can spend on it.

Does PyAI use WebRTC VAD?

No. PyAI endpoints audio on its own servers with its own model, not the WebRTC Gaussian mixture model. In a live session PyAI sends the end of turn about 196 ms after the user stops and checks the words as well as the silence, so a pause to think does not cut the user off mid-sentence.

PyAI is voice infrastructure. It hands you the audio and transcript, and does not replace your own models or review tools.

Skip the VAD setup. Endpoint a live conversation on PyAI, free.

Start building with $50 creditSandbox keys never bill, and you can start without a credit card.

One stack: Omni, Hear, Speak, Recap and Trace.