Starlings.Generation.Abstractions 0.4.0

There is a newer version of this package available.
See the version list below for details.
dotnet add package Starlings.Generation.Abstractions --version 0.4.0
                    
NuGet\Install-Package Starlings.Generation.Abstractions -Version 0.4.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Starlings.Generation.Abstractions" Version="0.4.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Starlings.Generation.Abstractions" Version="0.4.0" />
                    
Directory.Packages.props
<PackageReference Include="Starlings.Generation.Abstractions" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Starlings.Generation.Abstractions --version 0.4.0
                    
#r "nuget: Starlings.Generation.Abstractions, 0.4.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Starlings.Generation.Abstractions@0.4.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Starlings.Generation.Abstractions&version=0.4.0
                    
Install as a Cake Addin
#tool nuget:?package=Starlings.Generation.Abstractions&version=0.4.0
                    
Install as a Cake Tool

Starlings.Generation.Abstractions

Backend-neutral types for Starling's chat models: everything a generation runtime and a model definition share, without ONNX Runtime. Dependencies: the owned tokenizer Starlings.Tokenizers and Microsoft.Extensions.AI.Abstractions 10.10.1 only. The ONNX Runtime GenAI backend is Starlings.Generation.OnnxGenAI; the model definitions are Starlings.Models.Qwen35 and Starlings.Models.QwenChat. The API may still change.

Namespace Starlings.Generation:

  • Request and result: GenerationOptions (MaxNewTokens, StopSequences, AdditionalStopTokenIds, Sampling, PrefillChunkTokens, ReturnLogprobs, TopLogprobs, Guidance; Validate() applies the decode loop's checks), SamplingOptions, GenerationGuidance (experimental: JSON Schema, Lark grammar or regex, optional fast-forward), GenerationResult, GenerationUpdate, GenerationFinishReason, GenerationUsage, GenerationTimings, TokenLogprob, TopTokenLogprob and GenerationContextLengthException.
  • Text: StopSequenceMatcher (stop strings over streamed text, never emitting part of one) and IncrementalDetokenizer (UTF-8-safe streaming detokenization with the owned tokenizer).
  • Chat: ChatTurn (ChatTurnRole system, user, assistant or tool; content, reasoning, ToolCalls), ToolCall (name and JSON arguments), ToolDefinition (the JSON a template renders), IChatTemplate (render turns, tools and the thinking switch to prompt text), IToolCallParser (split a streamed completion into CompletionParts), ParsedCompletion.
  • ModelProfile: a pinned checkpoint's repository and revision, owned tokenizer profile and digest, template digest, end-of-sequence ids, vocabulary size, hidden size, layers, head size, context length and licence.
  • PythonJson: Python json.dumps(ensure_ascii=False), str() and repr(float) of JSON values, as chat templates write tool data; data Python could not hold the same way throws PythonJsonException.
  • IGenerationSequence: the asynchronous seam of one request's decoding state (AppendTokensAsync, StepAsync, CopyLastLogitsAsync, Dispose). GenAISequence in Starlings.Generation.OnnxGenAI implements it; other backends can.
  • GenerationChatMapping: ToChatFinishReason, ToUsageDetails and ToChatRole for Microsoft.Extensions.AI adapters.

Status: the moved request, result and text types keep the behaviour (and the profile maturity below) they had in Starlings.Generation.OnnxGenAI. The new chat seams (ChatTurn, ToolDefinition, ToolCall, IChatTemplate, IToolCallParser, ParsedCompletion), ModelProfile, PythonJson and GenerationChatMapping are experimental. IGenerationSequence is experimental for GenAISequence (each call wraps the synchronous call and completes synchronously) and untested for any other backend: no asynchronous implementation exists in this repository yet.

using Starlings.Generation;
using Starlings.Models.Qwen35;

IChatTemplate template = Qwen35ChatTemplate.Instance;       // the same text as Qwen35ChatTemplate.Render
string prompt = template.Render([ChatTurn.User("What is the capital of France?")]);
IToolCallParser parser = Qwen35ToolCallParser.Create(tools: [], enableThinking: false);
ParsedCompletion parsed = ParsedCompletion.Parse(parser, completionText);

Maturity

Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.

Profile Maturity Platforms
qwen2.5-0.5b-instruct/ortgenai-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
qwen3-0.6b/ortgenai-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
qwen3.5-0.8b/ortgenai-fp16x-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
qwen3.5-0.8b/ortgenai-fp32x-cpu stable linux-x64 cpu: supported; windows-x64 cpu: untested
qwen3.5-0.8b/ortgenai-fp32x-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested

The types moved here from Starlings.Generation.OnnxGenAI unchanged in behaviour; the migration from 0.3.0 is in the release notes. Documentation: ai-dotnet/README.md.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (4)

Showing the top 4 NuGet packages that depend on Starlings.Generation.Abstractions:

Package Downloads
Starlings.Models.Qwen35

Qwen/Qwen3.5-0.8B (revision 2fc06364) model definition without a runtime: the pinned chat template rendered in C# for text and tool-calling conversations (system, user, assistant and tool messages, tool definitions and calls, thinking switch; images rejected), a parser for the template's tool-call format (whole or streamed), the llguidance tool-call grammar builder, per-request chat options, the checkpoint's ModelProfile (repository, revision, owned tokenizer profile qwen35-bpe-chat, template digest, end-of-sequence ids, decoder shape) and the asset lock of the Hub checkpoint itself (safetensors, config, tokenizer, template). No ONNX Runtime dependency. To run the model in-process on ONNX Runtime GenAI, reference Starlings.Models.Qwen35.OnnxGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Generation.OnnxGenAI

Text generation on ONNX Runtime GenAI 0.17.1 behind a Starling adapter: model loading with explicit provider selection (no silent fallback, TF32 off by default), runtime config overlays, reference-counted disposal, and an owned decode loop over pre-tokenized ids (greedy or seeded sampling, max new tokens, stop token ids and stop strings, streaming with UTF-8-safe incremental detokenization through the owned tokenizer Starlings.Tokenizers, cancellation between decode steps, usage accounting). The backend-neutral options, results, stop matcher and detokenizer are in Starlings.Generation.Abstractions. References only Microsoft.ML.OnnxRuntimeGenAI.Managed: the application adds Microsoft.ML.OnnxRuntimeGenAI (CPU) or Microsoft.ML.OnnxRuntimeGenAI.Cuda. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.QwenChat

Qwen/Qwen3-0.6B (revision c1899de2, thinking disabled) and Qwen/Qwen2.5-0.5B-Instruct (revision 7ae55760) model definitions without a runtime: the pinned chat templates rendered in C# for system, user, assistant and tool messages with tool definitions and calls (tool calling experimental; multimodal input rejected), a parser for the templates' JSON tool-call format (whole or streamed), per-request chat options, and each checkpoint's identity and decoder shape (QwenChatCheckpoint, ModelProfile) with tokenizer verification through the owned tokenizer Starlings.Tokenizers, and the asset lock of each Hub checkpoint itself (safetensors, config, generation config, tokenizer files). No ONNX Runtime dependency. To run the models in-process on ONNX Runtime GenAI, reference Starlings.Models.QwenChat.OnnxGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.Lfm3

MODEL LICENSE: LFM Open License v1.0, not MIT or Apache-2.0. Commercial use of LFM3 is licensed only to entities below USD 10 million annual revenue (section 5); read LICENSE.txt in the package before use. LiquidAI/LFM3-1.7B text generation on ONNX Runtime (Liquid's ONNX export LiquidAI/LFM3-1.7B-ONNX, revision c680759a): the LFM3 chat template for text turns, a managed greedy decode loop over the export's step graph with its convolution and attention caches, and the owned tokenizer (Starlings.Tokenizers profile lfm3-bpe-chat). Ships the asset lock, not the weights; acquire them explicitly. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.5.0 191 11/3/2026
0.4.0 149 11/3/2026