Starlings.Generation.Abstractions
0.4.0
See the version list below for details.
dotnet add package Starlings.Generation.Abstractions --version 0.4.0
NuGet\Install-Package Starlings.Generation.Abstractions -Version 0.4.0
<PackageReference Include="Starlings.Generation.Abstractions" Version="0.4.0" />
<PackageVersion Include="Starlings.Generation.Abstractions" Version="0.4.0" />
<PackageReference Include="Starlings.Generation.Abstractions" />
paket add Starlings.Generation.Abstractions --version 0.4.0
#r "nuget: Starlings.Generation.Abstractions, 0.4.0"
#:package Starlings.Generation.Abstractions@0.4.0
#addin nuget:?package=Starlings.Generation.Abstractions&version=0.4.0
#tool nuget:?package=Starlings.Generation.Abstractions&version=0.4.0
Starlings.Generation.Abstractions
Backend-neutral types for Starling's chat models: everything a generation runtime and a model definition share, without ONNX Runtime. Dependencies: the owned tokenizer Starlings.Tokenizers and Microsoft.Extensions.AI.Abstractions 10.10.1 only. The ONNX Runtime GenAI backend is Starlings.Generation.OnnxGenAI; the model definitions are Starlings.Models.Qwen35 and Starlings.Models.QwenChat. The API may still change.
Namespace Starlings.Generation:
- Request and result:
GenerationOptions(MaxNewTokens,StopSequences,AdditionalStopTokenIds,Sampling,PrefillChunkTokens,ReturnLogprobs,TopLogprobs,Guidance;Validate()applies the decode loop's checks),SamplingOptions,GenerationGuidance(experimental: JSON Schema, Lark grammar or regex, optional fast-forward),GenerationResult,GenerationUpdate,GenerationFinishReason,GenerationUsage,GenerationTimings,TokenLogprob,TopTokenLogprobandGenerationContextLengthException. - Text:
StopSequenceMatcher(stop strings over streamed text, never emitting part of one) andIncrementalDetokenizer(UTF-8-safe streaming detokenization with the owned tokenizer). - Chat:
ChatTurn(ChatTurnRolesystem, user, assistant or tool; content, reasoning,ToolCalls),ToolCall(name and JSON arguments),ToolDefinition(the JSON a template renders),IChatTemplate(render turns, tools and the thinking switch to prompt text),IToolCallParser(split a streamed completion intoCompletionParts),ParsedCompletion. ModelProfile: a pinned checkpoint's repository and revision, owned tokenizer profile and digest, template digest, end-of-sequence ids, vocabulary size, hidden size, layers, head size, context length and licence.PythonJson: Pythonjson.dumps(ensure_ascii=False),str()andrepr(float)of JSON values, as chat templates write tool data; data Python could not hold the same way throwsPythonJsonException.IGenerationSequence: the asynchronous seam of one request's decoding state (AppendTokensAsync,StepAsync,CopyLastLogitsAsync,Dispose).GenAISequenceinStarlings.Generation.OnnxGenAIimplements it; other backends can.GenerationChatMapping:ToChatFinishReason,ToUsageDetailsandToChatRolefor Microsoft.Extensions.AI adapters.
Status: the moved request, result and text types keep the behaviour (and the profile maturity below) they had in Starlings.Generation.OnnxGenAI. The new chat seams (ChatTurn, ToolDefinition, ToolCall, IChatTemplate, IToolCallParser, ParsedCompletion), ModelProfile, PythonJson and GenerationChatMapping are experimental. IGenerationSequence is experimental for GenAISequence (each call wraps the synchronous call and completes synchronously) and untested for any other backend: no asynchronous implementation exists in this repository yet.
using Starlings.Generation;
using Starlings.Models.Qwen35;
IChatTemplate template = Qwen35ChatTemplate.Instance; // the same text as Qwen35ChatTemplate.Render
string prompt = template.Render([ChatTurn.User("What is the capital of France?")]);
IToolCallParser parser = Qwen35ToolCallParser.Create(tools: [], enableThinking: false);
ParsedCompletion parsed = ParsedCompletion.Parse(parser, completionText);
Maturity
Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.
| Profile | Maturity | Platforms |
|---|---|---|
qwen2.5-0.5b-instruct/ortgenai-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
qwen3-0.6b/ortgenai-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
qwen3.5-0.8b/ortgenai-fp16x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
qwen3.5-0.8b/ortgenai-fp32x-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
qwen3.5-0.8b/ortgenai-fp32x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
The types moved here from Starlings.Generation.OnnxGenAI unchanged in behaviour; the migration from 0.3.0 is in the release notes. Documentation: ai-dotnet/README.md.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Microsoft.Extensions.AI.Abstractions (>= 10.10.1)
- Starlings.Tokenizers (= 0.3.4)
NuGet packages (4)
Showing the top 4 NuGet packages that depend on Starlings.Generation.Abstractions:
| Package | Downloads |
|---|---|
|
Starlings.Models.Qwen35
Qwen/Qwen3.5-0.8B (revision 2fc06364) model definition without a runtime: the pinned chat template rendered in C# for text and tool-calling conversations (system, user, assistant and tool messages, tool definitions and calls, thinking switch; images rejected), a parser for the template's tool-call format (whole or streamed), the llguidance tool-call grammar builder, per-request chat options, the checkpoint's ModelProfile (repository, revision, owned tokenizer profile qwen35-bpe-chat, template digest, end-of-sequence ids, decoder shape) and the asset lock of the Hub checkpoint itself (safetensors, config, tokenizer, template). No ONNX Runtime dependency. To run the model in-process on ONNX Runtime GenAI, reference Starlings.Models.Qwen35.OnnxGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Generation.OnnxGenAI
Text generation on ONNX Runtime GenAI 0.17.1 behind a Starling adapter: model loading with explicit provider selection (no silent fallback, TF32 off by default), runtime config overlays, reference-counted disposal, and an owned decode loop over pre-tokenized ids (greedy or seeded sampling, max new tokens, stop token ids and stop strings, streaming with UTF-8-safe incremental detokenization through the owned tokenizer Starlings.Tokenizers, cancellation between decode steps, usage accounting). The backend-neutral options, results, stop matcher and detokenizer are in Starlings.Generation.Abstractions. References only Microsoft.ML.OnnxRuntimeGenAI.Managed: the application adds Microsoft.ML.OnnxRuntimeGenAI (CPU) or Microsoft.ML.OnnxRuntimeGenAI.Cuda. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.QwenChat
Qwen/Qwen3-0.6B (revision c1899de2, thinking disabled) and Qwen/Qwen2.5-0.5B-Instruct (revision 7ae55760) model definitions without a runtime: the pinned chat templates rendered in C# for system, user, assistant and tool messages with tool definitions and calls (tool calling experimental; multimodal input rejected), a parser for the templates' JSON tool-call format (whole or streamed), per-request chat options, and each checkpoint's identity and decoder shape (QwenChatCheckpoint, ModelProfile) with tokenizer verification through the owned tokenizer Starlings.Tokenizers, and the asset lock of each Hub checkpoint itself (safetensors, config, generation config, tokenizer files). No ONNX Runtime dependency. To run the models in-process on ONNX Runtime GenAI, reference Starlings.Models.QwenChat.OnnxGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Lfm3
MODEL LICENSE: LFM Open License v1.0, not MIT or Apache-2.0. Commercial use of LFM3 is licensed only to entities below USD 10 million annual revenue (section 5); read LICENSE.txt in the package before use. LiquidAI/LFM3-1.7B text generation on ONNX Runtime (Liquid's ONNX export LiquidAI/LFM3-1.7B-ONNX, revision c680759a): the LFM3 chat template for text turns, a managed greedy decode loop over the export's step graph with its convolution and attention caches, and the owned tokenizer (Starlings.Tokenizers profile lfm3-bpe-chat). Ships the asset lock, not the weights; acquire them explicitly. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
GitHub repositories
This package is not used by any popular GitHub repositories.