Performance you can verify.
Every PyAI model, scored on the metrics that matter for phone calls, accuracy, latency, turn-taking, grounding, with the exact method and the thresholds. We measure our own engine the way we'd want a vendor to measure theirs.
How each model scores
Each metric carries its value, the band it's graded against (good / warn / critical), a plain-English explainer, and where the number came from.
Hear
Speech-to-text
Word Error Rate
Fixture scorecard (telephony-8k + accented-noisy corpus)
4.76% across the telephony/accented corpus; 0% on clean audio. Live round-trip WER on clean audio measured 1.59%.
In plain terms: Word Error Rate is the share of words the transcript gets wrong (substituted, dropped, or inserted). Under 5% is roughly human-parity for clear speech; we hold that bar on real 8 kHz phone audio, which is the hard case. The top STT models all cluster at 2-5% WER on clean English, accuracy is table stakes, so we lead on latency and billing, not an accuracy crown.
Streaming first-partial latency
Live in-region, streaming partials (Parakeet-TDT-0.6b v4.2)
~185 ms (185-205 ms band) to the first partial transcript on real speech, in-region. The first partial is revisable, it's stable_text that firms up as more audio arrives, not a final. Sits inside Deepgram's published 150-300 ms interim band, the same fast class. Kept separate from batch throughput below, different metric, different thing.
In plain terms: On a live call this is how quickly the agent starts 'hearing' words, which is what lets it endpoint and barge-in on time. Early in-region measurement, not an SLA, a laptop adds ~250-300 ms of round-trip on top.
Batch transcription real-time factor
Live in-region, async batch tier
92-247× real-time on the async batch tier, in-region, an hour of audio transcribes in well under a minute. This is a throughput number and is deliberately NOT blended with the streaming first-partial latency above; they measure different things.
In plain terms: Real-time factor is how many seconds of audio you transcribe per second of wall-clock. 100× means a 100-minute recording is done in about a minute. It's about clearing archives fast, not about how snappy a live call feels.
Speak
Text-to-speech
Time to first audio byte (in-region)
Tested, warm-path streaming synthesis (n=5)
~53 ms to first audio byte on the warm path, ~98 ms cold, in-region (n=5). This is the first audio byte, not the first spoken word. A cloned voice streams its first byte in ~32 ms warm via the per-voice prefix cache. Under load on a shared/sandbox key the path can queue, a capacity fix tracked separately; it does not change the warm-path figure.
In plain terms: Time-to-first-byte is how long until the caller hears the first sound. Streaming it from the first byte (instead of waiting to render the whole clip) is what makes a voice agent feel responsive rather than frozen. Sub-100 ms is imperceptible, this puts Speak in the fastest class alongside Cartesia and ElevenLabs Flash, measured in-region.
Format & sample-rate correctness
Live run (wav/pcm/g711_ulaw)
Every requested container and sample rate is honored exactly, a hard contract check across wav @24kHz, pcm @8kHz, and g711_ulaw @8kHz.
In plain terms: When you ask for 8 kHz μ-law for Twilio, you get exactly that, not a resampled surprise. 100% means the audio drops into your telephony pipeline without a conversion step.
Omni
Realtime speech-to-speech agent
Voice-to-voice p50 (utterance-end → first audio)
Live in-region, self-tracking (Omni 402 / Agents 386 p50, pyai-latency-30m)
Median ~390 ms voice-to-voice, from the moment the caller stops speaking to the first byte of the agent's reply, measured in-region. This is human conversational pace, not a latency record; raw S2S models can be quicker. Early in-region measurement, not a guaranteed SLA. Continuously tracked by pyai-latency-30m.
In plain terms: This is the number that defines how human the agent feels. ~390 ms sits inside the natural pause between people talking; above ~800 ms it starts to feel like a walkie-talkie. We measure utterance-end to first audio in-region; a laptop adds ~250-300 ms of round-trip on top, and anchoring on session-open would inflate it by the length of what the caller said.
VAQI (voice-agent quality index)
Fixture scorecard (composite)
Composite 0-100: interruptions 40% + missed-response 40% + latency 20%. 94.3 reflects low false-barge and missed-response rates alongside the turn latency.
In plain terms: A single number for 'is this a good voice agent.' It penalizes talking over people (interruptions), failing to reply (missed responses), and lagging, the three things that make a call feel broken. Above 70 is good; we're at 94.
KB-grounded answer rate
Fixture scorecard
9 of 10 answers were both grounded in the bound knowledge base and keyword-correct.
In plain terms: When the agent answers, does it answer from your content (not made up), and is it right? 90% means it pulls the answer from your knowledge base and gets it correct nearly every time.
Where these numbers land against the field
Our figures are in-region measurements; competitor figures are public/advertised pricing and vendor latency self-claims, dated and flagged to verify. We claim the fastest class and an all-in lower than any packaged platform, never 'fastest' or 'cheapest.'
All-in voice-agent price per minute (USD)
Lower is better. PyAI in teal. The bar spans each platform’s typical all-in range; packaged platforms advertise a lower platform fee, then bill the model, voice, and telephony on top.
All-in, billed per second, speech + brain + telephony in one rate. $0.05/min Omni API, $0.08/min for the no-code Agents feature.
Advertised from ~$0.09/min self-serve → all-in $0.11-$0.14/min once LLM, voice, and telephony are added.
Advertised agent minutes + LLM → all-in $0.10-$0.15/min once LLM, voice, and telephony are added.
Advertised no-code plans → all-in $0.11-$0.16/min once LLM, voice, and telephony are added.
Advertised ~$0.05/min platform fee → all-in $0.10-$0.31/min once LLM, voice, and telephony are added.
Advertised ~$0.07/min platform → all-in $0.13-$0.31/min once LLM, voice, and telephony are added.
PyAI is one flat all-in rate, billed per second. Packaged figures are advertised pricing plus STT/LLM/TTS/telephony passthrough estimates as of June 2026, composite and config-dependent.
Speak time-to-first-byte, sub-100ms class
Lower is better. Not a ranking, figures use different conditions.
Speak is in-region, warm-path, first-audio-byte; competitor figures are vendor self-claims. No neutral benchmark exists, same fast class, verify in-region. On naturalness, ElevenLabs and Cartesia lead.
Hear first partial, same fast class as Deepgram
Lower is better. Bands show each provider’s published range.
Hear’s ~185 ms in-region first partial (revisable) sits inside Deepgram’s published 150-300 ms band. We don’t claim to beat Deepgram on latency; accuracy is independent (Artificial Analysis).
Published pricing, the price is the price
Per minute, billed per second. No platform fee, no seats, no minimums.These are published list rates, not a quote. Omni is all-in (speech + brain + telephony), lower than any packaged platform above. Trace (+$0.03/min) and Recap (+$0.02/min) are per-call add-ons.
What these metrics mean (and why they matter on a call)
WER
Word Error Rate = (substitutions + deletions + insertions) / reference words, over normalized tokens. The top STT models cluster at 2-5% on clean English, so accuracy is table stakes, not a differentiator.
TTFB
Time-to-first-audio-byte: wall time from request send to the first audio byte received (not the first spoken word). Reported on the warm, uncontended streaming path, in-region.
First partial
Streaming STT latency to the first partial transcript. The partial is revisable (stable_text that firms up as more audio arrives), and is kept separate from batch RTF, which is a throughput metric.
Voice-to-voice
Utterance-end to the first agent audio frame, the felt conversational latency. NOT session-anchored, which would inflate it by the caller's speech length. Human conversational pace, not a record.
Batch RTF
Real-time factor for async batch transcription: seconds of audio processed per second of wall-clock. A throughput number, distinct from streaming latency.
VAQI
Voice Agent Quality Index (0-100): interruptions 40% + missed-response 40% + latency 20%.
How we measure, and the honest caveats
An in-repo, CI-gated harness (evals/) plus a self-tracking in-region latency profile (pyai-latency-30m). The same scorers grade recorded fixtures and live API calls, so a live number is directly comparable to a fixture number.
- Every latency number here is measured in-region (us-central1), caller and engine co-located. These are early measurements, not guaranteed SLA percentiles, a laptop in another region adds ~250-300 ms of round-trip and is not a fair comparison.
- Fixture metrics (WER corpus, VAQI, KB-grounded) are recorded golden cases scored offline; live numbers (voice-to-voice, first partial, Speak TTFB, format correctness) are measured against api.pyai.com in-region.
- Live Hear WER uses a Speak→Hear round-trip (synthesize, then transcribe), so it reflects cleaner-than-telephony audio; the fixture WER on real 8 kHz call audio is the more demanding number.
- Streaming first-partial latency and batch real-time factor are different products of different things and are never blended into one 'speed' number.
- Voice-to-voice latency is measured on the live hybrid path; the p99 tail on a shared key can be inflated by per-key rate limits, a dedicated monitoring key is the fix.
One voice stack. Numbers with receipts.
Start with $50 in free credit. No card.
Agents is live in beta. Sandbox keys have daily limits and never touch billing.