Gemini Live

The Gemini source streams audio to Gemini Live and plays the model's reply, reporting state and audio to the visual. Long conversations cross Gemini's connection limit transparently through session resumption.

import { GeminiLiveVoiceSource } from "@sinua/voice/gemini";

// Your endpoint mints an ephemeral token (`auth_tokens/…`) with the Gemini API key, which
// never reaches the browser (see Credentials). A reconnect resumes the session with a new
// token. The model, voice and instructions are locked into the token on the server.
export const voice = new GeminiLiveVoiceSource({ credentialUrl: "/api/voice/gemini" });

It sends transcripts of both speakers (transcriptTiming none), synced to the played audio by default.

Credentials

Your server mints an ephemeral auth token (auth_tokens/…) with your API key and locks the model, voice and instructions into it, so the client can't change them: mintGeminiLiveCredential in @sinua/voice/server. Point the source at that endpoint with credentialUrl, the same on every platform. See Credentials.

Packages

Good to know

• Ephemeral tokens only. With an auth_tokens/… token, the source uses Gemini's constrained endpoint automatically. A raw API key is refused on every platform; for local development use npx @sinua/voice dev-proxy.
• Reconnects are handled. When Gemini announces the end of a connection (goAway) or the socket closes, the source resumes the session on a new connection, with a new token from your endpoint.
• Start from a user gesture on the web, since the browser only allows the microphone and audio playback after one. On iOS, add NSMicrophoneUsageDescription; on Android, declare RECORD_AUDIO. Say where the audio goes: the user's voice is sent to Google, so your iOS purpose string (and, on Android, the explanation you show before the system prompt, which carries no app text) should say so. For example: "Your voice is sent to Google so the assistant can hear you." App Store review expects the purpose string to explain the use.