Live speech to text for any voice agent
English speech to text for voice agents has one job that file transcription never faces: a person is waiting for an answer. As a real time speech to text API, Hear listens to a live stream from wherever the agent runs, whether a mobile app, a website widget, a kiosk or a support line, and returns partial text in about 200 ms. Hear also powers PyAI Omni, the speech to speech agent, so you can keep your own reasoning and voice or let Omni listen, think and reply in one session. The same stream gives you live English transcription for captions and notes. The same engine also works as a English speech to text generator for the transcripts your team keeps after each session.
Key takeaways
- First partial: about 200 ms after the user starts speaking.
- Accuracy: 1.59% WER on clean audio and 4.76% on 8 kHz phone, accented or noisy audio, PyAI's own benchmarks at industry parity, measured on English.
- Any channel: app, browser mic, car, smart device or phone line all feed the same WebSocket stream.
- Price: $0.001/min, billed by the whole minute, and an empty transcript costs nothing.
- Inside Omni: end of turn lands about 196 ms after the user stops, and voice to voice runs near 390 ms median.
- Drop in: whisper-1 compatible, so an existing client switches by changing the base URL.
How a live voice session becomes text
Hear works the same way whatever the speaker talks into, so one integration covers your app, your site and your devices.
Someone speaks into a mobile app, a browser mic, a dashboard unit or a line, and your code captures the audio.
Send PCM16 frames of about 20 ms each over a WebSocket as the words are spoken.
The first partial transcript arrives in about 200 ms and firms up while the user keeps going.
Your agent, or Omni, takes the text and answers, books, searches or controls a device.
What Hear handles in a live session
Spoken input is messy in every channel. Here is what Hear deals with between the microphone and your agent.
Partials mid sentence
Text starts arriving while the user is still talking, so a search or lookup can start on the first few words instead of the full request.
Cafés, cars and open offices
Background chatter and road noise are normal for an assistant. Hear measures 4.76% WER on noisy and accented audio, including 8 kHz phone audio, measured on English.
Many Englishes
Users speak US, UK, Indian, Nigerian or Australian English. Test samples from your own users, because accent mix changes results from one product to the next.
Product names and jargon
Load up to 5 custom vocabulary terms, such as an app name or a menu item, so they come back spelled the way your system stores them.
Numbers spoken aloud
With numerals handling, spoken times, prices and order codes return as digits your agent can parse without extra cleanup.
Turns and timestamps
Verbose JSON marks speaker turns with timestamps, which helps for session logs, captions and reviewing what an assistant heard.
Accuracy and speed
Hear figures are PyAI's own benchmarks, measured on English and reported at industry parity. Omni turn and voice to voice figures come from full Omni sessions.
Where a listening agent runs
An English voice assistant speech to text layer fits wherever people would rather talk than type. Six common places, none more important than the others:
In-app assistant
Let users ask a banking, travel or shopping app for something by voice and act on the transcript inside the app.
Website support bot
Add a mic button to a help widget so visitors describe a problem out loud instead of typing a long ticket.
In-car and hands-free
Take navigation, messaging and media requests from a driver whose hands stay on the wheel.
Smart home devices
Turn a spoken request to a speaker, thermostat or hub into text your device logic can route.
Dictation and accessibility
Give people who cannot type easily a way to write notes, forms and messages by speaking.
Customer support phone line
Answer an inbound support number, understand the caller's request and route or resolve it during the call.
Formats, limits and output
| Input | A live WebSocket stream of PCM16 audio from any app, browser or device, or a file up to 25 MB. OpenAI compatible, so a whisper-1 client works unchanged. |
|---|---|
| Output | Partial text while the user speaks, then a final transcript. Text, JSON or verbose JSON with timestamps and speaker turns. |
| Modes | Sync file call for short clips, live streaming for apps, devices and voice sessions, and background jobs for long recordings that run 92 to 247 times real time. |
| Tuning | Set English as the language, turn on numerals handling for times and prices, and load up to 5 custom vocabulary terms for names your users say often. |
| Billing | $0.001/min of audio, rounded to the whole minute. An empty transcript costs nothing, and a busy engine returns a retry, not a charge. |
Add live speech to text to your own assistant
Hear uses the OpenAI transcription format, so a voice assistant stack moves over by changing the base URL. Full docs: PyAI SDKs.
SDK
From your backend, the SDK opens one live stream per user session and hands each partial and final transcript to your logic the moment it lands.
MCP
Through the MCP server, an agent framework can call Hear as a tool, for example to transcribe a voice note a user recorded before choosing a reply.
CLI
The CLI streams a recording or your laptop mic and prints partials live, a quick way to check vocabulary terms before you ship.
Where Hear fits
PyAI is voice infrastructure. Hear, Speak and Omni run as one stack in one region, so listening, turn detection, reasoning and speech share a session instead of hopping between vendors. Hear is not a full assistant builder and does not decide what your agent says. Run Hear inside your own assistant, or use Omni when you want everything handled together. Figures are published on PyAI benchmarks.
Speech to text for voice agents FAQ
How quickly does Hear return text?
Hear sends the first partial transcript about 200 ms after the user starts speaking, then updates it as more words arrive. Inside an Omni session, the end of a turn is detected roughly 196 ms after the speaker stops, and a spoken reply comes back near 390 ms median, which keeps a conversation feeling natural.
How accurate is Hear for English?
Hear scores 1.59% word error rate on clean audio and 4.76% on 8 kHz phone, accented or noisy audio. Both figures are PyAI's own benchmarks, measured on English and sitting at industry parity. Run a sample of real sessions from your own app or device before launch to see how your users sound.
Can Hear run inside my own voice assistant?
Yes. Open a WebSocket for each session, stream PCM16 audio from the app, browser or device, and pass partial and final text to your own reasoning and voice steps. Because Hear is whisper-1 compatible, many existing assistants switch over by changing only the base URL in their client.
What does Hear cost?
Hear costs $0.001 per minute of audio, billed by the whole minute, with no platform fee. An empty transcript is free. New accounts start with $50 in credit and no credit card, and sandbox keys never bill, so you can test a full assistant flow before paying anything.
When should I pick Hear over Omni?
Pick Hear when you already have agent logic and a voice and only need fast transcripts. Pick Omni when you want PyAI to listen, detect turns, reason and speak in one session. Omni's turn detector holds on unfinished thoughts and digit strings, which is how its 390 ms voice to voice median is measured.
Prices checked November 2026. Hear figures from pyai.com/benchmarks.