Universal.Operative.Sdk.Realtime.GoogleGemini 1.0.0

About

Universal.Operative.Sdk.Realtime.GoogleGemini adapts the Gemini Live API to Universal.Operative.Sdk.Realtime's provider-neutral IModel/IRealtimeSession/SessionEvent model, via Google's own Google.GenAI package (which ships a typed BidiGenerateContent client directly — the existing turn-based Universal.Google.Gemini.Client has no Live API bindings).

This is the second realtime adapter, deliberately chosen next (before Nova Sonic) to stress-test two things the OpenAI adapter didn't exercise: native session resumption, and parallel/async tool-call correlation.

How to Use

Installation

dotnet add package Universal.Operative.Sdk
dotnet add package Universal.Operative.Sdk.Realtime
dotnet add package Universal.Operative.Sdk.Realtime.GoogleGemini

Quick start

using Universal.Operative.Sdk.Realtime;
using Universal.Operative.Sdk.Realtime.GoogleGemini;

var model = new Model(apiKey, modelId: "gemini-2.5-flash-native-audio-preview-09-2025");

IRealtimeSession session = await model.ConnectAsync(new RealtimeSessionOptions
{
    SystemPrompt = [new TextBlock("You are a helpful voice assistant.")],
    Tools = [myTool],
});

IOperative operative = OperativeBuilder.Create(session).AddTool(myTool).Build();

await foreach (SessionOutputEvent e in operative.RunAsync())
{
    switch (e)
    {
        case SessionReadyEvent: Console.WriteLine("connected"); break;
        case TranscriptDeltaEvent t: Console.WriteLine($"{t.Role}: {t.Text}"); break;
        case AudioOutputDeltaEvent a: /* play a.Audio (24kHz PCM16) */ break;
        case SessionExpiringEvent expiring: /* about to disconnect -- see resumption below */ break;
    }
}

Session resumption

Session implements IResumableSession (Capabilities.HasFlag(SessionCapabilities.NativeResumption) is true). Gemini pushes a fresh resumption handle over the life of the connection; suspend and resume with it directly:

if (session is IResumableSession resumable)
{
    SessionHandle handle = await resumable.SuspendAsync();
    // ... later, possibly after a restart ...
    IRealtimeSession resumed = await resumable.ResumeAsync(handle);
}

Fidelity notes

• Capabilities: AudioInProcess | ToolCalling | Transcripts | NativeResumption.
• Tool results: the full ToolResult content is JSON-serialized into Gemini's function-response object ({"result": ...} / {"error": ...}), not flattened to a bare string the way OpenAI's wire format requires.
• Async tool scheduling: SubmitToolResultEvent.SchedulingHint maps to Gemini's FunctionResponse.Scheduling (interrupt/whenIdle/silent) — a client-set hint on the response, not something the model attaches to the original call.
• Barge-in: automatic (server-side VAD) only; InterruptRequestEvent throws NotSupportedException since Gemini has no client-sent cancel event, and manual activityStart/activityEnd isn't wired up.
• Tool choice: RealtimeSessionOptions.ToolChoice throws NotSupportedException -- Google.GenAI.Types.LiveConnectConfig has no tool-choice-equivalent field.
• Raw passthrough: video frames, UsageMetadata, and voice-activity signals beyond interrupted aren't promoted to canonical events -- they arrive as RawPassthroughEvent.
• Auth: direct Gemini API key only; Vertex AI auth isn't wired up.
• Audio format: fixed at Model.InputSampleRateHertz (16kHz) in and Model.OutputSampleRateHertz (24kHz) out, mono PCM16 -- not configurable, unlike the other adapters.