Starlings.Generation.OnnxGenAI 0.2.0

There is a newer version of this package available.
See the version list below for details.
dotnet add package Starlings.Generation.OnnxGenAI --version 0.2.0
                    
NuGet\Install-Package Starlings.Generation.OnnxGenAI -Version 0.2.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Starlings.Generation.OnnxGenAI" Version="0.2.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Starlings.Generation.OnnxGenAI" Version="0.2.0" />
                    
Directory.Packages.props
<PackageReference Include="Starlings.Generation.OnnxGenAI" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Starlings.Generation.OnnxGenAI --version 0.2.0
                    
#r "nuget: Starlings.Generation.OnnxGenAI, 0.2.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Starlings.Generation.OnnxGenAI@0.2.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Starlings.Generation.OnnxGenAI&version=0.2.0
                    
Install as a Cake Addin
#tool nuget:?package=Starlings.Generation.OnnxGenAI&version=0.2.0
                    
Install as a Cake Tool

Starlings.Generation.OnnxGenAI

Text generation on ONNX Runtime GenAI 0.17.1 behind a Starling adapter, with an owned decode loop. Prompt ids come from the owned tokenizer (Starlings.Tokenizers); GenAI's tokenizer and template engine are not used. The API may still change.

  • GenAIModel.Load(directory, options): explicit provider (CPU, or CUDA with TF32 off by default), no silent fallback, config overlays (threads, pad id) recorded in AppliedOverlays, reference-counted disposal, package version check. On CUDA the native calls of concurrent sequences are serialized by default (SerializeNativeCalls): concurrent GenAI 0.17.1 generators on the CUDA provider produced wrong tokens.
  • TextGenerator: greedy or seeded sampling, MaxNewTokens, stop token ids (never part of the completion) and stop strings (text cut before the match), streaming with UTF-8-safe incremental detokenization (IncrementalDetokenizer), optional chunked prefill, cancellation checked before the prefill, between prefill chunks and between decode steps (not inside a step), usage (PromptTokens, CompletionTokens excluding the stop token) and stage timings. Synchronous by design: each step is blocking native work.
  • Advanced: GenAISequence (append ids, step, copy the last logits).

Most applications use it through a model package such as Starlings.Models.Qwen35.

Maturity

Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.

Profile Maturity Platforms
qwen3.5-0.8b/ortgenai-fp16x-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
qwen3.5-0.8b/ortgenai-fp32x-cpu stable linux-x64 cpu: supported; windows-x64 cpu: untested
qwen3.5-0.8b/ortgenai-fp32x-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested

Native runtime

The package references only Microsoft.ML.OnnxRuntimeGenAI.Managed (exactly 0.17.1; it brings Microsoft.Extensions.AI.Abstractions). The application references exactly one native package of the same version:

Deployment Package Notes
CPU Microsoft.ML.OnnxRuntimeGenAI 0.17.1 Brings Microsoft.ML.OnnxRuntime 1.30.0. Natives for linux-x64, linux-arm64, win-x64, win-arm64, osx-arm64; exercised on linux-x64 only.
CUDA Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.17.1 Brings Microsoft.ML.OnnxRuntime.Gpu 1.30.0. linux-x64 and win-x64 natives (CUDA 13 build); the CUDA runtime and cuBLAS are not shipped. Exercised on linux-x64 only.

The model directory must come from the onnxruntime-genai builder of the same version. Documentation: ai-dotnet/README.md.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (2)

Showing the top 2 NuGet packages that depend on Starlings.Generation.OnnxGenAI:

Package Downloads
Starlings.Models.Qwen35.OnnxGenAI

Qwen/Qwen3.5-0.8B (revision 2fc06364) chat generation in-process on ONNX Runtime GenAI 0.17.1: Qwen35ChatModel over the model definition Starlings.Models.Qwen35 (the pinned chat template rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers profile qwen35-bpe-chat: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy or seeded sampling, stop tokens and strings, streaming, cancellation between steps, usage). Ships the asset lock of the ONNX Runtime GenAI bundle, not the weights. Profiles qwen3.5-0.8b/ortgenai-fp32x (CPU; CUDA with TF32 off) and ortgenai-fp16x (CUDA), Linux x64, Stable. The application adds Microsoft.ML.OnnxRuntimeGenAI or Microsoft.ML.OnnxRuntimeGenAI.Cuda. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.QwenChat.OnnxGenAI

Qwen/Qwen3-0.6B (revision c1899de2, thinking disabled) and Qwen/Qwen2.5-0.5B-Instruct (revision 7ae55760) text chat generation in-process on ONNX Runtime GenAI 0.17.1: QwenChatModel over the model definitions Starlings.Models.QwenChat (the pinned chat templates rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy, stop tokens and strings, streaming, cancellation between steps, usage). Ships no weights: QwenChatModelAssets downloads the profile's bundle explicitly from Starling's public Hugging Face hosting (or it is built locally from the pinned original), and Load verifies every file against the profile's digests. Profiles qwen3-0.6b/ortgenai-fp32-cpu and qwen2.5-0.5b-instruct/ortgenai-fp32-cpu, Linux x64. The application adds Microsoft.ML.OnnxRuntimeGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.5.0 165 11/3/2026
0.4.0 115 11/3/2026
0.3.0 258 10/19/2026
0.2.0 357 10/17/2026
0.1.0-preview.6 115 10/14/2026