Starlings.Generation.OnnxGenAI
0.2.0
See the version list below for details.
dotnet add package Starlings.Generation.OnnxGenAI --version 0.2.0
NuGet\Install-Package Starlings.Generation.OnnxGenAI -Version 0.2.0
<PackageReference Include="Starlings.Generation.OnnxGenAI" Version="0.2.0" />
<PackageVersion Include="Starlings.Generation.OnnxGenAI" Version="0.2.0" />
<PackageReference Include="Starlings.Generation.OnnxGenAI" />
paket add Starlings.Generation.OnnxGenAI --version 0.2.0
#r "nuget: Starlings.Generation.OnnxGenAI, 0.2.0"
#:package Starlings.Generation.OnnxGenAI@0.2.0
#addin nuget:?package=Starlings.Generation.OnnxGenAI&version=0.2.0
#tool nuget:?package=Starlings.Generation.OnnxGenAI&version=0.2.0
Starlings.Generation.OnnxGenAI
Text generation on ONNX Runtime GenAI 0.17.1 behind a Starling adapter, with an owned decode loop. Prompt ids come from the owned tokenizer (Starlings.Tokenizers); GenAI's tokenizer and template engine are not used. The API may still change.
GenAIModel.Load(directory, options): explicit provider (CPU, or CUDA with TF32 off by default), no silent fallback, config overlays (threads, pad id) recorded inAppliedOverlays, reference-counted disposal, package version check. On CUDA the native calls of concurrent sequences are serialized by default (SerializeNativeCalls): concurrent GenAI 0.17.1 generators on the CUDA provider produced wrong tokens.TextGenerator: greedy or seeded sampling,MaxNewTokens, stop token ids (never part of the completion) and stop strings (text cut before the match), streaming with UTF-8-safe incremental detokenization (IncrementalDetokenizer), optional chunked prefill, cancellation checked before the prefill, between prefill chunks and between decode steps (not inside a step), usage (PromptTokens,CompletionTokensexcluding the stop token) and stage timings. Synchronous by design: each step is blocking native work.- Advanced:
GenAISequence(append ids, step, copy the last logits).
Most applications use it through a model package such as Starlings.Models.Qwen35.
Maturity
Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.
| Profile | Maturity | Platforms |
|---|---|---|
qwen3.5-0.8b/ortgenai-fp16x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
qwen3.5-0.8b/ortgenai-fp32x-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
qwen3.5-0.8b/ortgenai-fp32x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
Native runtime
The package references only Microsoft.ML.OnnxRuntimeGenAI.Managed (exactly 0.17.1; it brings Microsoft.Extensions.AI.Abstractions). The application references exactly one native package of the same version:
| Deployment | Package | Notes |
|---|---|---|
| CPU | Microsoft.ML.OnnxRuntimeGenAI 0.17.1 |
Brings Microsoft.ML.OnnxRuntime 1.30.0. Natives for linux-x64, linux-arm64, win-x64, win-arm64, osx-arm64; exercised on linux-x64 only. |
| CUDA | Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.17.1 |
Brings Microsoft.ML.OnnxRuntime.Gpu 1.30.0. linux-x64 and win-x64 natives (CUDA 13 build); the CUDA runtime and cuBLAS are not shipped. Exercised on linux-x64 only. |
The model directory must come from the onnxruntime-genai builder of the same version. Documentation: ai-dotnet/README.md.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Microsoft.ML.OnnxRuntimeGenAI.Managed (= 0.17.1)
- Starlings.Tokenizers (= 0.3.0)
NuGet packages (2)
Showing the top 2 NuGet packages that depend on Starlings.Generation.OnnxGenAI:
| Package | Downloads |
|---|---|
|
Starlings.Models.Qwen35.OnnxGenAI
Qwen/Qwen3.5-0.8B (revision 2fc06364) chat generation in-process on ONNX Runtime GenAI 0.17.1: Qwen35ChatModel over the model definition Starlings.Models.Qwen35 (the pinned chat template rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers profile qwen35-bpe-chat: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy or seeded sampling, stop tokens and strings, streaming, cancellation between steps, usage). Ships the asset lock of the ONNX Runtime GenAI bundle, not the weights. Profiles qwen3.5-0.8b/ortgenai-fp32x (CPU; CUDA with TF32 off) and ortgenai-fp16x (CUDA), Linux x64, Stable. The application adds Microsoft.ML.OnnxRuntimeGenAI or Microsoft.ML.OnnxRuntimeGenAI.Cuda. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.QwenChat.OnnxGenAI
Qwen/Qwen3-0.6B (revision c1899de2, thinking disabled) and Qwen/Qwen2.5-0.5B-Instruct (revision 7ae55760) text chat generation in-process on ONNX Runtime GenAI 0.17.1: QwenChatModel over the model definitions Starlings.Models.QwenChat (the pinned chat templates rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy, stop tokens and strings, streaming, cancellation between steps, usage). Ships no weights: QwenChatModelAssets downloads the profile's bundle explicitly from Starling's public Hugging Face hosting (or it is built locally from the pinned original), and Load verifies every file against the profile's digests. Profiles qwen3-0.6b/ortgenai-fp32-cpu and qwen2.5-0.5b-instruct/ortgenai-fp32-cpu, Linux x64. The application adds Microsoft.ML.OnnxRuntimeGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 0.5.0 | 165 | 11/3/2026 |
| 0.4.0 | 115 | 11/3/2026 |
| 0.3.0 | 258 | 10/19/2026 |
| 0.2.0 | 357 | 10/17/2026 |
| 0.1.0-preview.6 | 115 | 10/14/2026 |