Omni
The definitive all-in-one AI voice agent model. Hybrid speech-to-speech, fused LLM brain, emotion-aware voices, and tool calling.
Omni is a hybrid speech-to-speech engine with a fused LLM brain that hears, thinks, and speaks natively, so there is no STT-LLM-TTS pipeline to stitch. It runs at human conversational pace (~390 ms median voice-to-voice, in-region) with barge-in and native 8 kHz telephony tuning, over a single WebSocket.
Answered by a PyAI Omni agent · may be recorded.
- Price
- $0.05/min
- Endpoint
wss://api.pyai.com/v1/omni- Scope
omni:session- Model id
pyai-omni-realtime
Watch Omni run a call, end to end
Acme Dental’s front-desk agent - every step powered by PyAI.
Hear <-> Speak repeats every turn, with barge-in - then the call becomes data.
Each step is a PyAI model. Press play to watch them hand off - or tap any stage to jump.
- Hear
pyai-hear - Omni brain
pyai-omni-realtime - Speak
pyai-voice
Transcript, summary, intent, and disposition - emitted as JSON when the call ends.
What you get
The whole agent in one model
One hybrid speech-to-speech model with a fused LLM brain replaces the STT, VAD, turn-detection, LLM, RAG, and TTS chain other platforms stitch together.
Tool calling that does real work
Enable prebuilt hosted tools by name or declare your own functions in the configure frame; the brain calls them mid-call and speaks the result.
Grounded in your knowledge
Point Omni at your own knowledge endpoint and it pulls the right facts on every turn: fast, fail-open, and customer-hosted.
Voices with emotion, even yours
Speak back in expressive stock voices, or clone a voice for free and drop the voice_id into the configure frame.
Built for the phone, a billion conversations deep
Purpose-built for 8 kHz telephony, backed by over a billion conversations across the SaaS Labs portfolio (JustCall, Helpwise, ServiceAgent), at ~390 ms median voice-to-voice, in-region.
Capabilities
- Hybrid speech-to-speech
- Fused LLM brain
- Tool / function calling
- Knowledge-base grounding
- Emotion-aware voices
- Built-ins: transfer, DTMF, hold, end-call
- Voice-to-voice ~390 ms p50 (in-region) + barge-in
- Telephony-native (8 kHz)
- OpenAI-compatible
~390 ms median voice-to-voice (utterance-end to first audio, in-region) with barge-in; an early measurement, not an SLA. Per-second billing includes hearing, reasoning, retrieval, tool calls, and speech.
Everything Omni comes loaded with
One JSON configure frame after connect sets the whole agent; PyAI stores nothing per agent.
voice_idAny stock voice, your free clone, or a prompt-designed voice.
personaThe system prompt: role, tone, and rules for the call.
kb_endpointYour knowledge endpoint; Omni grounds every turn against it.
tools[]Custom functions the brain can call mid-call, declared inline.
greetingThe first line spoken the instant the call connects.
consent_lineA spoken consent and disclosure line for compliance flows.
Already on Vapi, Retell, or OpenAI Realtime?
You don't rewrite your agent, you delete the plumbing. Omni is OpenAI-realtime-compatible, so most code stays the same.
import Vapi from "@vapi-ai/web";
const vapi = new Vapi(process.env.VAPI_PUBLIC_KEY);
vapi.start({
transcriber: { provider: "deepgram", model: "nova-3" },
model: { provider: "openai", model: "gpt-4o" },
voice: { provider: "11labs", voiceId: "burt" },
});// One socket: STT + reasoning + retrieval + TTS.
const ws = new WebSocket(
"wss://api.pyai.com/v1/omni?format=pcm16&rate=24000",
["pyai-key." + apiKey],
);
ws.onmessage = (e) => playAudioFrame(e.data); // audio outPoint at PyAI, use the Omni model, send your key. Most code stays put. Full Vapi migration guide
Start in minutes
# From OpenAI Realtime, Vapi, or Retell: point at PyAI, use the Omni model.
url = "wss://api.pyai.com/v1/realtime?model=pyai-omni-realtime"
# native: wss://api.pyai.com/v1/omni?format=pcm16&rate=24000
# audio up, agent audio down - one socket, whole agent.FAQ
What can Omni actually do out of the box?
Everything a phone agent needs in one model: hear the caller, reason with a fused LLM brain, call your custom tools/functions, ground answers in your knowledge base, and speak back in emotion-aware voices with natural turn-taking and barge-in. Built-in call control (transfer_to_human, send_dtmf, play_hold, collect, end_call) ships in the box, and you tune voice, persona, greeting, language, model tier, and a consent line per call via one configure frame.
Does Omni support tool / function calling?
Yes, three ways. Enable prebuilt hosted tools by name with zero setup (knowledge search, web search, weather, currency, unit conversion, math, date/time, geocoding, news). Register your own as a server tool and Omni calls your signed webhook from a network-isolated executor, which works on phone calls, since your code never has to be on the socket. Or run tools client-side in a browser app. Omni's brain fills the arguments from the conversation, calls the tool mid-call, and speaks the result, with sensible timeouts and fail-open behavior so audio never stalls. Your endpoints and secrets never touch the model. Live call control (transfer to a human, send DTMF, play hold, hang up) runs in your telephony transport today, and is handled for you on managed Telephony (coming soon).
How does knowledge-base grounding work?
Point Omni at your own knowledge endpoint in the configure frame. On each turn the engine fetches the relevant facts (fast and fail-open) and grounds its answer in them. The knowledge stays customer-hosted (you don't ship your data to PyAI), and retrieval is folded into the call rather than added as a separate RAG hop.
Can Omni use my own voice, with emotion?
Yes. Use any expressive stock voice, or bring your own: voice cloning enrollment and prompt-to-voice design are free. Drop the resulting voice_id into the configure frame and Omni speaks with it: emotion-aware, in the voice that fits your brand and language.
How is this different from Vapi, Retell, ElevenLabs, or Bland?
Those platforms orchestrate a pipeline (STT, VAD, turn detection, an LLM, RAG, and TTS), and you pay the latency and fragility of every hop. Omni is a single hybrid speech-to-speech model with a fused LLM brain that does all of it in one call, purpose-built for telephony.
I'm already on another provider, how hard is the switch?
About two lines. Omni is OpenAI-realtime-compatible: point your client at wss://api.pyai.com/v1/realtime?model=pyai-omni-realtime and use your PyAI key. Most of your code stays the same.
Why does telephony / 8 kHz matter?
Phone lines carry 8 kHz narrowband audio, far less detail than studio sound. Models trained on clean wideband audio lose accuracy on real calls. Omni is built for narrowband, so it stays accurate on the lines you actually run on.
Is there an OpenAI-realtime-compatible URL?
Yes: wss://api.pyai.com/v1/realtime?model=pyai-omni-realtime. The native surface is wss://api.pyai.com/v1/omni?format=pcm16&rate=24000.
Build with Omni today.
Start free with $50.00 in credit - no card. Your test key works instantly.
No credit card - OpenAI-compatible - cancel anytime