Feature · WebRTC voice activity detection
WebRTC voice activity detection for voice agents.
WebRTC VAD is a fast, open-source tool that separates speech from silence. PyAI endpoints the conversation on its own servers.
What PyAI gives you for WebRTC voice activity detection
- WebRTC voice activity detection runs a Gaussian mixture model over the 80 Hz to 4 kHz energy bands and labels 10, 20 or 30 millisecond frames of 16-bit mono PCM, sampled at 8, 16, 32 or 48 kHz, as speech or not speech.
- WebRTC VAD aggressiveness runs from mode 0 to mode 3. Mode 3 drops the most background noise but can clip soft consonants, while mode 0 keeps quiet speech and lets more noise through.
- Developers reach WebRTC VAD through community bindings such as py-webrtcvad in Python and Swift wrappers on iOS. WebRTC VAD is fast and light, but some neural detectors can be more accurate on noisy audio in Picovoice's published noise benchmark.
- PyAI does not use WebRTC VAD. PyAI endpoints the audio on its own servers and sends the end of turn about 196 ms after a user stops, with about 390 ms voice-to-voice, measured in region.
How does WebRTC voice activity detection work, step by step?
WebRTC VAD runs on the raw audio before anything else in a voice pipeline. On PyAI the same job runs server side, so you do not host it.
Audio is split into frames
The incoming PCM audio is cut into fixed 10, 20 or 30 millisecond frames, each one 16-bit mono at 8, 16, 32 or 48 kHz.
Each frame is scored
The Gaussian mixture model weighs energy across the 80 Hz to 4 kHz bands and returns a speech or non-speech label for that frame.
The mode sets the threshold
An aggressiveness mode from 0 to 3 decides how much noise is filtered. A higher mode rejects more noise and risks clipping soft speech.
Speech is handed on
Frames marked as speech pass to the next step, usually speech-to-text. On PyAI, PyAI endpoints the stream itself and returns speech events, so you skip this setup.
Where is WebRTC voice activity detection used?
WebRTC VAD shows up wherever audio has to be filtered before it is sent or processed.
Dropping silent packets
VoIP and WebRTC apps use it to avoid encoding and sending silent audio, sometimes with discontinuous transmission, which saves bandwidth on a voice or video session.
Before speech-to-text
In a build-it-yourself stack it runs before speech-to-text, so the pipeline can tell when a user starts and stops speaking for turn-taking and barge-in.
Auto-mute and push-to-talk
Apps use it to drive a mute indicator or open the microphone only while someone is speaking, so background noise is not sent.
Client-side gating
You can still run WebRTC VAD in a browser to hold back silent audio before you stream to PyAI, while PyAI endpoints the speech it receives.
WebRTC voice activity detection questions people ask
How does WebRTC VAD work?
WebRTC VAD cuts audio into 10, 20 or 30 millisecond frames and scores each one with a Gaussian mixture model across the 80 Hz to 4 kHz bands. An aggressiveness mode from 0 to 3 then sets how much background noise it rejects before it marks a frame as speech or silence.
What are WebRTC VAD aggressiveness modes?
Aggressiveness is a setting from 0 to 3. Mode 0 is the most permissive and keeps quiet speech, but lets more noise through. Mode 3 filters the most noise and suits clean audio, though it can clip soft consonants or trailing words. Most apps test a few modes to find the right fit.
Which fits better, WebRTC VAD or a neural VAD?
WebRTC VAD is lightweight, so it can suit low-resource or real-time use. Some neural detectors can be more accurate on noisy audio; Picovoice's benchmark illustrates why the result depends on the model and detection threshold. The right pick depends on how clean your audio is and how much compute you can spend on it.
Does PyAI use WebRTC VAD?
No. PyAI endpoints audio on its own servers with its own model, not the WebRTC Gaussian mixture model. In a live session PyAI sends the end of turn about 196 ms after the user stops and checks the words as well as the silence, so a pause to think does not cut the user off mid-sentence.
PyAI is voice infrastructure. It hands you the audio and transcript, and does not replace your own models or review tools.