Skip to content

Twilio SDK · @pyai/twilio v0.4.2

Give your phone number a natural voice agent.

Bridge a Twilio Media Stream to Omni for the complete speech-to-speech loop.

Convert Twilio audio to the Omni stream through OmniAgent.bridge.
Let the bridge handle audio pacing and interruptions.
Validate Twilio signatures before accepting webhooks and sockets.

From download to working audio

Node.js 22+. A PyAI key, a Twilio phone number and auth token, and an HTTPS tunnel or deployed server. Set PUBLIC_ORIGIN to its origin.

Terminal · macOS / Linux
curl -fSL https://pyai.com/starters/pyai-twilio.tar.gz -o pyai-twilio.tar.gz
tar -xzf pyai-twilio.tar.gz
cd pyai-twilio
npm install
cp .env.example .env
# Fill in .env, then:
npm start

Edit the downloaded .env before running. Keep it out of Git. On Windows, use WSL for these shell commands.

What happens when you run it?

After starting the server, set your Twilio number’s incoming voice webhook to https://YOUR-HOST/voice (POST). Carrier charges apply when you make a call.

Take it into your application

Choose a voice from the API catalog and check its language coverage. Reuse clients, consume streaming audio immediately, and cancel interrupted responses. PCM, WAV and G.711 stream; MP3 and Opus are buffered.

The complete entry point

This is the same file included in the download. Copy it here, or get the full project with its dependency and environment files.

server.mjs
import http from "node:http";
import { WebSocketServer } from "ws";
import twilio from "twilio";
import { OmniAgent, connectStreamTwiML } from "@pyai/twilio";

const { PYAI_API_KEY, TWILIO_AUTH_TOKEN, PUBLIC_ORIGIN, PORT = "8080" } = process.env;
if (!PYAI_API_KEY || !TWILIO_AUTH_TOKEN || !PUBLIC_ORIGIN) {
  throw new Error("Set PYAI_API_KEY, TWILIO_AUTH_TOKEN and PUBLIC_ORIGIN in .env");
}
const origin = new URL(PUBLIC_ORIGIN);
if (origin.protocol !== "https:" || origin.pathname !== "/" || origin.search || origin.hash) {
  throw new Error("PUBLIC_ORIGIN must be an HTTPS origin, for example https://your-tunnel.example");
}
function authorized(req, params = {}) {
  // Use the configured public URL, never the untrusted Host header.
  return twilio.validateRequest(TWILIO_AUTH_TOKEN,
    req.headers["x-twilio-signature"] || "", `${origin.origin}${req.url}`, params);
}
const server = http.createServer(async (req, res) => {
  if (req.url === "/health") { res.writeHead(200).end("ok"); return; }
  if (req.method !== "POST" || req.url !== "/voice") { res.writeHead(404).end(); return; }
  try {
    const chunks = [];
    let size = 0;
    for await (const chunk of req) {
      size += chunk.length;
      if (size > 65536) { res.writeHead(413).end(); return; }
      chunks.push(chunk);
    }
    const fields = new URLSearchParams(Buffer.concat(chunks).toString("utf8"));
    const params = {};
    for (const name of new Set(fields.keys())) {
      const values = fields.getAll(name);
      params[name] = values.length === 1 ? values[0] : values;
    }
    if (!authorized(req, params)) { res.writeHead(403).end(); return; }
    res.writeHead(200, { "Content-Type": "text/xml" }).end(
      connectStreamTwiML(`wss://${origin.host}/media`),
    );
  } catch { res.writeHead(400).end(); }
});
const sockets = new WebSocketServer({ noServer: true, maxPayload: 65536 });
server.on("upgrade", (req, socket, head) => {
  if (req.url !== "/media" || !authorized(req)) {
    socket.end("HTTP/1.1 403 Forbidden\r\nConnection: close\r\n\r\n");
    return;
  }
  sockets.handleUpgrade(req, socket, head, (ws) => sockets.emit("connection", ws));
});
sockets.on("connection", (ws) => {
  OmniAgent.bridge(ws, {
    apiKey: PYAI_API_KEY,
    voice: process.env.PYAI_VOICE || "stock_emma_en_gb",
    persona: "You are a friendly support assistant. Keep spoken replies brief.",
    onError: () => console.error("Voice session failed; inspect your PyAI session in the console."),
  });
});
server.listen(Number(PORT), "0.0.0.0", () => {
  console.log(`Twilio incoming voice webhook: ${origin.origin}/voice (POST)`);
});

Build with Twilio in your coding agent

Install Node.js 22+ and your coding agent first. Connect PyAI’s MCP server, then ask it to call get_started. For a sandbox test, ask it to call create_sandbox_key once. Keep live credentials in your environment.

Codex · terminal
codex mcp add pyai -- npx -y @pyai/mcp@0.2.0
Claude Code · terminal
claude mcp add --transport stdio --scope project pyai -- npx -y @pyai/mcp@0.2.0
Cursor · .cursor/mcp.json
{
  "mcpServers": {
    "pyai": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@pyai/mcp@0.2.0"
      ]
    }
  }
}

Merge the pyai entry into your existing mcpServers configuration. Enable it in Cursor’s MCP settings.

Paste into your coding agent
Read https://pyai.com/skill.md and https://pyai.com/sdks/twilio.md.
Use the matching starter to add PyAI voice to this project.
Keep credentials in environment variables. Run a short synthetic
audio test and report the result and any missing credentials.
Do not place a phone call or send a message unless I ask.

Want a reusable local skill? Copy the Codex and Claude Code skill install commands. Any tool that reads Markdown can use skill.md.

Twilio SDK questions

What is included in the download?

The Twilio starter includes its complete entry point, pinned PyAI package version, dependency manifest, environment template, and run instructions.

What do I need before running it?

A PyAI key, a Twilio phone number and auth token, and an HTTPS tunnel or deployed server. Set PUBLIC_ORIGIN to its origin.

Can I use this with Cursor, Codex or Claude Code?

Yes. Connect the PyAI MCP server and give your agent the Twilio Markdown guide. It includes the complete starter source and setup commands.

How do I test streaming performance?

Measure first received audio and playback readiness separately in your application. Network location, load, text, and voice affect timing. Playing the saved file does not measure streaming latency.

← Explore all SDKs