Universal.Operative.Sdk.Realtime 1.0.0

About

Universal.Operative.Sdk.Realtime is the duplex-stream/voice half of the Universal.Operative.Sdk family — the counterpart to Universal.Operative.Sdk.Discrete's turn-based model, for building agents against live, bidirectional session APIs (OpenAI Realtime, Gemini Live, Amazon Nova Sonic, Grok Voice) instead of discrete request/response turns.

This package ships the provider-neutral core only: the canonical SessionEvent model, IRealtimeSession/IResumableSession, IModel, IOperative/OperativeBuilder/OperativeFactory, the ITransformer<TEvent> hook pipeline, and Conversation/ConversationWriter/ConversationReader for recording and replaying a session. It has no provider adapters on its own — pair it with a provider package (e.g. Universal.Operative.Sdk.Realtime.OpenAI) for a concrete IModel.

How to Use

Installation

dotnet add package Universal.Operative.Sdk
dotnet add package Universal.Operative.Sdk.Realtime

Core concepts

• SessionEvent — the canonical event model, split into SessionOutputEvent (what IRealtimeSession.Output emits) and SessionInputEvent (what IRealtimeSession.SendAsync accepts) so the two directions are a compile-time distinction, not just a naming convention. Covers what's shared across providers — transcripts, audio deltas, tool calls, interruption, turn/session lifecycle — plus a RawPassthroughEvent (output) / RawSendEvent (input) escape-hatch pair for whatever a given provider does that hasn't been promoted into the canonical shape. Every event carries Raw, a list of RawEventRefs tracing it back to the provider-native event(s) it came from, and the whole hierarchy is JSON-serializable (see Conversation below) so nothing is ever silently lost between the canonical, raw, and recorded views.
• IRealtimeSession — one live duplex connection to a provider. Capabilities (a SessionCapabilities flags enum) tells you what this particular session can actually do — e.g. whether audio flows through the canonical stream at all (AudioInProcess), which is false for topologies where the media plane bypasses your process entirely (WebRTC-direct, SIP passthrough) even though tool-calling and transcripts might still work.
• IModel — connects and returns an IRealtimeSession. Every adapter's primary (WebSocket) connection mode shares this same ConnectAsync shape; provider-specific connection modes that don't (SIP passthrough, browser-direct WebRTC) stay off this interface as extra methods on the concrete Model type instead.
• IResumableSession — for providers with session resumption, native (Gemini's handle) or client-assisted (an adapter that synthesizes the same contract by splicing history into a fresh session, e.g. for Nova Sonic). Check Capabilities for which kind you have.
• IOperative / OperativeBuilder / OperativeFactory — wraps an already-connected IRealtimeSession and is the intended entry point for both directions of traffic: RunAsync() relays the (transformer-applied) event stream and dispatches ToolCallRequestedEvents to your registered ITools automatically (reusing the exact same ITool/ToolResult contract as the turn-based side), and SendAsync is where anything you send — including the tool results RunAsync generates on your behalf — passes through before reaching the session. OperativeBuilder is the one-shot fluent path for a single session; OperativeFactory mints one IOperative per session from a reusable tool/transformer configuration, for a host spawning many concurrent sessions (e.g. one per incoming call).
• ITransformer<TEvent> — the hook seam for observing, rewriting, or suppressing one event at a time as it passes through an IOperative, registered separately per direction via OperativeBuilder.AddOutputTransformer/AddInputTransformer. An output transform only shapes what a consumer (and anything downstream, like a recorder) sees — it can't undo what the provider already did; an input transform has real teeth, since it runs before the event is ever sent.
• Conversation / ConversationWriter / ConversationReader — the durable, serializable counterpart to a live session (deliberately sharing Discrete.Conversation's name: same role, disambiguated by namespace). ConversationWriter appends one JSON line per event as a session runs; ConversationReader reads it back, either fully materialized or as a lazy stream. RecordingSession is a transparent IRealtimeSession decorator that wires a session straight into a writer; RecordedSession replays a Conversation back out as a fake IRealtimeSession with no provider connection, for debugging or as an adapter-agnostic test fixture.
• AudioAdapter — a duplex wrapper over an already-connected IOperative that resamples PCM16 audio in both directions, so your microphone/speaker hardware rate doesn't need to match a provider's wire rate. WriteMicrophoneAudioAsync/CommitMicrophoneAudioAsync accept audio at your chosen input rate and resample it before sending; PlaybackAudio yields provider output audio resampled to your chosen playback rate; Events yields every other SessionOutputEvent unchanged. Owns the wrapped IOperative's RunAsync() pump internally -- disposing the adapter disposes the operative.

Quick start

using Universal.Operative.Sdk.Realtime;

IRealtimeSession session = /* from a provider's IModel — e.g. someProvider.ConnectAsync(options) */;

IOperative operative = OperativeBuilder.Create(session)
    .AddTool(myTool)
    .Build();

await foreach (SessionOutputEvent e in operative.RunAsync())
{
    switch (e)
    {
        case TranscriptDeltaEvent t: Console.WriteLine($"{t.Role}: {t.Text}"); break;
        case AudioOutputDeltaEvent a: /* play a.Audio */ break;
        case InterruptedEvent: /* stop playback immediately */ break;
        case ToolCallRequestedEvent call: Console.WriteLine($"Calling {call.Name}..."); break;
    }
}

await operative.SendAsync(new AppendAudioInputEvent { Audio = micChunk });

License

Free for noncommercial use under the Universal.Operative Noncommercial License. Source is closed; commercial use requires a separate license from Andrew Ong.