OpenAI GPT-Live

GPT-Live (gpt-live-1) is OpenAI's full-duplex voice model, a different API from Realtime. Only your server can open a session, with your project key, so there is no client credential: the source sends its WebRTC offer to your endpoint, your endpoint opens the session and answers with OpenAI's reply.

import { OpenAILiveVoiceSource } from "@sinua/voice/openai";

// GPT-Live has no browser credential: your endpoint opens the session with your key
// (see the server tab) and answers the source's SDP offer. The source POSTs `{ sdp }`
// to it on every connect, and again when a session expires.
export const voice = new OpenAILiveVoiceSource({ sessionUrl: "/api/voice/openai-live" });

It sends transcripts of both speakers (transcriptTiming segments), synced to the played audio by default.

Your endpoint

• The source POSTs { "sdp": … } as JSON to sessionUrl, with Authorization: Bearer when you give it a credential of your own (credentialUrl or a provider, as on the Credentials page).
• createOpenAILiveSession in @sinua/voice/server opens the session (POST /v1/live/sessions); openAILiveResponse answers with OpenAI's 201 JSON. A bare SDP answer works too.
• Your server sets the model, voice, instructions, delegation and any prior conversation when it opens the session.
• sessionUrl can't be api.openai.com, and a credential for it is never a raw sk-… key.

How the states map

GPT-Live sends no turn events, so the source reads the call itself, the same way on every platform:

disconnect() goes idle at once, then closes the session cleanly so the final usage is recorded. A session that expires or drops is replaced by a new one with a fresh request to your endpoint.

Requests, errors and audio

• Web, your request: headers adds headers to the call to your endpoint, as an object or a function called per request (for a CSRF token, say), and fetch replaces the fetch it uses.
• Web, errors: a failed request rejects with a VoiceHttpError that carries the status and the response body, so you can tell a 401 from a 500.
• iOS, the audio session: Sinua sets the session up before the call. audioSession: .speaker (the default) plays through the speaker, .receiver through the earpiece, and .unmanaged leaves the session to your app.
• When speaking ends: after a short reply the agent counts as speaking for about 0.7 s more, after a long answer up to 1.7 s, so the state doesn't flicker between phrases. Talking over it (barge-in) is still instant.

Packages