ElBruno.Speech.Abstractions 1.6.1

ElBruno.Speech

Local-first speech runtime for .NET — VAD → STT → LLM → TTS 🎙️

A reusable, local-first speech runtime for .NET 8 and .NET 10. Built on Microsoft.Extensions.AI provider boundaries — works with local models (Whisper, VibeVoice, Qwen-TTS, Ollama) and cloud providers (Azure OpenAI, OpenAI) without changing orchestration code.

Audio input
    ↓
Audio normalization · resampling · framing · buffering
    ↓
Voice Activity Detection (Silero VAD)
    ↓
Turn detection · utterance assembly
    ↓
ISpeechToTextClient
    ↓
IChatClient
    ↓
Streaming text segmentation
    ↓
ITextToSpeechClient
    ↓
Audio output

What's New

Packages

Package	NuGet	Downloads	Description
ElBruno.Speech.Abstractions			Audio types, VAD contracts, session interfaces, error types
ElBruno.Speech.Audio			WAV I/O, PCM conversion, resampling, framing, ring buffers
ElBruno.Speech.Vad.Silero			Silero VAD via ONNX Runtime — streaming voice activity detection
ElBruno.Speech.Stt.Parakeet			Experimental opt-in multilingual Parakeet TDT v3 local STT, available since 1.6.0-preview.1
ElBruno.Speech.Stt.Whisper	NuGet	Downloads	Experimental opt-in multilingual Whisper large-v3 local CPU STT, available since 1.6.0-preview.2
ElBruno.Speech.Tts.Ollama	1.6.0	Downloads	Ollama Phonon TTS via ITextToSpeechClient, including incremental audio streaming
ElBruno.Speech.Pipeline			VAD → STT → LLM → TTS orchestration, bounded channels, barge-in
ElBruno.Speech.AspNetCore			WebSocket endpoint, session registry, health checks
ElBruno.Speech.NAudio			Windows microphone and speaker via NAudio
ElBruno.Speech.OpenTelemetry			Activities, metrics, Aspire-compatible instrumentation
ElBruno.Speech.Cli			elbrunospeech dotnet tool
ElBruno.Speech.BlazorComponents			Blazor Server RCL — 6 ready-to-use speech pipeline UI components

Installation

dotnet add package ElBruno.Speech.Pipeline
dotnet add package ElBruno.Speech.Vad.Silero

Ollama Phonon text-to-speech

dotnet add package ElBruno.Speech.Tts.Ollama --version 1.6.0

Register services.AddOllamaSpeech() from ElBruno.Speech.Tts.Ollama to resolve Microsoft.Extensions.AI.ITextToSpeechClient. The provider supports configurable endpoint, model, voice, WAV/PCM output, cancellation, and incremental HTTP audio streaming. See the provider guide and complete C# demo.

No options are required: services.AddOllamaSpeech() uses http://localhost:11434/, model phonon, the default voice, and WAV output. Supply OllamaSpeechOptions only to override settings, such as a remote server. From 1.6.0-preview.4, the library resolves the speech route automatically; /v1 bases and existing full speech endpoints also work. The older preview.3 requires the full route.

Current platform limit: Ollama speech runs on Apple Silicon Macs with MLX. Windows/Linux support is pending upstream, with no promised date. A Windows C# client can call a compatible remote Ollama server. On the local Windows host, Ollama 0.40.11 rejected ollama pull phonon because MLX was unavailable. Community reports on issue #106 confirm real WAV generation and playback on Apple M2 with .NET 8 and M2 Max with .NET 10 using preview.5. These are user-reported English-speech checks, not comprehensive voice, streaming, quality, or performance benchmarks. No standalone Phonon runtime or non-Ollama fallback is included.

Install the CLI tool globally:

dotnet tool install -g ElBruno.Speech.Cli
elbrunospeech --help

Quick Start

// Register providers
services.AddWhisper(o => o.Model = KnownWhisperModels.WhisperBaseEn);
services.AddLocalLLMs(o => o.Model = KnownModels.Phi35MiniInstruct);
services.AddVibeVoiceTTS(o => o.SampleRate = 24_000);
services.AddSileroVad(o => o.MinimumSilenceDuration = TimeSpan.FromMilliseconds(500));

// Build the pipeline
services.AddSpeechPipeline(builder =>
{
    builder
        .UseVoiceActivityDetector<SileroVoiceActivityDetector>()
        .UseSpeechToText(sp => sp.GetRequiredService<ISpeechToTextClient>())
        .UseChatClient(sp => sp.GetRequiredService<IChatClient>())
        .UseTextToSpeech(sp => sp.GetRequiredService<ITextToSpeechClient>())
        .UseSentenceChunking(o =>
        {
            o.MinimumCharacters = 24;
            o.MaximumCharacters = 220;
            o.FlushTimeout = TimeSpan.FromMilliseconds(350);
        })
        .UseBargeIn(o => o.CancelOnSpeechStart = true);
});

// Use it
var pipeline = sp.GetRequiredService<ISpeechPipeline>();
await using var session = await pipeline.CreateSessionAsync();

await session.WriteAudioAsync(frame);
await foreach (var update in session.GetUpdatesAsync())
{
    // SpeechStartedUpdate, FinalTranscriptUpdate, AssistantAudioChunkUpdate, ...
}

Experimental multilingual Parakeet STT

An opt-in local Parakeet TDT v3 provider is available in 1.6.0-preview.1 as ElBruno.Speech.Stt.Parakeet, using the verified elbruno ONNX model bundle. It supports Spanish and other model languages without changing Whisper defaults. Native token-start metadata is preserved; word/segment intervals and language ID are explicitly unavailable. See usage, requirements, and evaluation limits.

Opt-in Whisper large-v3

ElBruno.Speech.Stt.Whisper is included in 1.6.0. It adds a separate local whisper.cpp CPU provider without replacing the existing ElBruno.Whisper package or changing defaults. It uses a pinned, explicitly downloaded multilingual large-v3 model. See setup, native requirements, and limitations and the fixed-reference FLEURS comparison.

CLI Tool

elbrunospeech devices                        # list audio input/output devices
elbrunospeech transcribe recording.wav       # transcribe a WAV file
elbrunospeech vad recording.wav             # run voice activity detection
elbrunospeech talk "Hello world" out.wav    # synthesize text to WAV

Observability

builder.Services.AddOpenTelemetry()
    .AddSpeechPipelineTelemetry();           // meter: ElBruno.Speech, source: ElBruno.Speech
// Aspire automatically configures OTLP export via OTEL_EXPORTER_OTLP_ENDPOINT

Samples

Related Repositories

• ElBruno.Whisper — ISpeechToTextClient via ONNX Whisper
• ElBruno.LocalLLMs — IChatClient for local LLMs
• ElBruno.VibeVoiceTTS — ITextToSpeechClient via VibeVoice
• ElBruno.QwenTTS — ITextToSpeechClient via Qwen3-TTS
• ElBruno.HuggingFace.Downloader — model downloads

Requirements

• .NET 8.0 or .NET 10.0
• Windows, Linux, or macOS
• NAudio package requires Windows (microphone/speaker I/O)

Building from Source

git clone https://github.com/elbruno/ElBruno.Speech.git
cd ElBruno.Speech
dotnet restore
dotnet build
dotnet test --filter "Category!=Integration"

Speech test fixtures

Reusable Spanish/English speech fixtures, CC BY 4.0 licensing, offline validation, and opt-in model evaluation are documented in Speech test fixtures.

Working with Copilot and Squad

Normal Copilot sessions can work on this repository without loading Squad. Squad is optional: select the Squad custom agent in your client or run copilot --agent squad to use its team orchestration and safety rules. The coordinator integrity check runs only in that custom agent.

If a normal session reports that the Squad coordinator is missing or truncated, update your checkout to include the instruction-scope fix and start a new Copilot session so it reloads .github/copilot-instructions.md. Existing sessions can retain the old instructions. If you select Squad and its loaded instructions are incomplete, stop orchestration and restart with the complete custom agent; do not add a canary to bypass the check.

📄 License

MIT — see LICENSE.

The FLEURS audio and reference transcripts under tests/fixtures/speech are separately licensed CC BY 4.0. See their attribution and license notices.

👋 About the Author

Hi! I'm ElBruno 🧡, a passionate developer and content creator exploring AI, .NET, and modern development practices.

Made with ❤️ by ElBruno

If you like this project, consider following my work across platforms:

• 📻 Podcast: No Tienen Nombre — Spanish-language episodes on AI, development, and tech culture
• 💻 Blog: ElBruno.com — Deep dives on embeddings, RAG, .NET, and local AI
• 📺 YouTube: youtube.com/elbruno — Demos, tutorials, and live coding
• 🔗 LinkedIn: @elbruno — Professional updates and insights
• 𝕏 Twitter: @elbruno — Quick tips, releases, and tech news

🙏 Acknowledgments

• Microsoft.Extensions.AI — ISpeechToTextClient, IChatClient, ITextToSpeechClient interfaces
• Silero VAD — voice activity detection model
• ONNX Runtime — local model inference
• NAudio — Windows audio I/O