Feature · Emotion detection
Emotion detection for voice agents: what PyAI covers.
PyAI hands you conversation words and recordings from any app, site or line, while reading the feeling stays with your own model or reviewers.
What PyAI gives you for emotion detection
- Emotion detection in a voice conversation has only words and voice to read, so voice-focused tools work from the PyAI transcript, the stereo recording, or both.
- An acoustic emotion model you host can study PyAI's saved stereo recording for pitch, pace and energy, since PyAI itself attaches no emotion label and never measures vocal tone.
- Speak and Cast emotion settings only shape how generated PyAI voices sound, and never read or score how a user feels.
How does an upset user show up in PyAI after the session?
Three short sessions, each with a strong feeling behind it. Watch what PyAI records and what it leaves for a person to judge. The script is invented for illustration.
Pick a moment, then press play
How does reading user emotion work with PyAI session data?
PyAI produces the conversation data in one record. Any emotion reading happens later, in your model or in a reviewer's head.
The session is saved
When an Omni session ends, the engine writes the transcript, duration, language and metadata, plus a stereo recording if the agent allows recording.
Words are summarized
Recap writes decisions, action items, moments and risk signals, and topic trackers match the example phrases your team supplied.
Rules are checked
Trace runs post-session rules for phrases present or missing and stores each finding at the exact second of the recording.
Someone judges the feeling
A reviewer listens in the session workspace, or your code fetches the recording through the PyAI API for an acoustic model.
Feeling shows in the words
Trackers, Trace rules and Recap risk signals pick up phrases like "I'm furious" or "cancel today".
Feeling shows only in the voice
PyAI flags nothing, because Hear and Recap read words. The recording is the input your own model needs.
Which voice agent teams ask for emotion detection, and why?
Requests to read user feelings usually hide a narrower job. Each job below names the PyAI output that covers it and the gap that remains.
Callers in hardship
Phrases such as "lost my job" or "can't pay this month" match a hardship tracker, so a trained human calls back instead of the next automated attempt.
Anxious patients
A tracker for words like "chest pain" sends the intake session to a review inbox the same day, while tone of voice is left to the clinician who listens.
Angry repeat users
Recap risk signals plus a Trace alert to Slack put the third complaint in a week in front of a team lead before the user churns.
Training an acoustic model
Stereo session recordings fetched through the Omni API become labeled training audio after human raters mark each clip, which keeps the emotion labels in-house.
Distressed claimants
Bereavement or accident wording triggers a moment with an owner and due date, and the follow-up lands on the Recap attention list.
Emotion detection questions about voice agents
Does PyAI detect emotions in voice sessions?
No. PyAI outputs no emotion label or score on any session. PyAI provides the transcript, an optional stereo recording, Recap risk signals and Trace findings. A reviewer listening in the session workspace, or an acoustic model your team runs on the recording, makes the emotional judgment instead of PyAI.
Is emotion detection the same as sentiment analysis?
No. Sentiment analysis sorts opinion into positive, negative or neutral, mostly from text. Emotion detection names a specific feeling such as anger, fear or joy and can also read faces or vocal tone. A user can be negative and calm, or positive and excited, so the two labels answer different questions.
Is emotion recognition legal at work in the EU?
The EU AI Act, Regulation 2024/1689, Article 5(1)(f), bans AI systems that infer emotions of people in workplaces and education institutions from 2 February 2025, except for medical or safety reasons. Scoring your own agents' emotions falls close to that line, so take legal advice before deploying. This is not legal advice.
What does the emotion setting in PyAI Speak do?
The emotion setting in PyAI Speak, and emotion with intensity per line in Cast, controls how a generated voice sounds, for example warm or urgent narration. Those settings shape PyAI's own speech output only. Neither Speak nor Cast listens to a user, so neither one detects or reports how a person feels.
PyAI is voice infrastructure. It hands you the audio and transcript, and does not replace your own models or review tools.