Starlings.Models.Abstractions
0.5.0
dotnet add package Starlings.Models.Abstractions --version 0.5.0
NuGet\Install-Package Starlings.Models.Abstractions -Version 0.5.0
<PackageReference Include="Starlings.Models.Abstractions" Version="0.5.0" />
<PackageVersion Include="Starlings.Models.Abstractions" Version="0.5.0" />
<PackageReference Include="Starlings.Models.Abstractions" />
paket add Starlings.Models.Abstractions --version 0.5.0
#r "nuget: Starlings.Models.Abstractions, 0.5.0"
#:package Starlings.Models.Abstractions@0.5.0
#addin nuget:?package=Starlings.Models.Abstractions&version=0.5.0
#tool nuget:?package=Starlings.Models.Abstractions&version=0.5.0
Starlings.Models.Abstractions
Types the Starlings model packages share, with no ONNX Runtime dependency (it references only Starlings.Acquisition, which needs nothing beyond the BCL). Code that works with several models can depend on these instead of each package's own types. The API may still change.
| Type | What it is | Implemented by |
|---|---|---|
CapabilityStatus |
Supported, Unsupported, Experimental, Untested. A missing entry never means supported. |
the capability tables of Starlings.Models.E5, Starlings.Models.EmbeddingGemma2, Starlings.Models.Gliner2 (XCapabilities.Statuses) |
ModelProfileCapabilities |
The capability record of one registry profile, generated from its manifest (see below) | XCapabilities.ProfileCapabilities of Starlings.Models.E5, Starlings.Models.EmbeddingGemma2, Starlings.Models.Gliner2, Starlings.Models.Bert, Starlings.Models.Kev.Onnx (KevCapabilities), Starlings.Models.Qwen35.OnnxGenAI (Qwen35Capabilities), Starlings.Models.QwenChat.OnnxGenAI (QwenChatCapabilities), Starlings.Models.Lfm3 and Starlings.Models.Whisper |
ITextEmbedder |
Dimension, MaxSequenceLength, EmbedBatch(texts, tokenCounts, cancellationToken), CountTokens(text), CountTokens(texts), Validate(text) (a TokenLimitCheck) |
E5TextEmbedder, EmbeddingGemma2TextEmbedder (no prompt), BertTextEmbedder (prompt none) |
IReranker |
Score(query, documents, cancellationToken): one raw score per document |
BertReranker |
Reranking, RerankedItem<T> |
Rerank a first-stage retriever's top-k (for example a Microsoft.Extensions.VectorData SearchAsync) with an IReranker or any Func<string, IReadOnlyList<string>, Task<float[]>> (Experimental, see below) |
|
IModelAssets<TFiles> |
A package's asset lock and explicit acquisition (DownloadAsync, OpenLocal, OpenDirectory) as one object |
E5ModelAssets.Assets, EmbeddingGemma2ModelAssets.Assets, Gliner2ModelAssets.Assets |
TokenLimitCheck |
TokenCount, MaxTokens, ExceedsLimit, TokenCountIsLowerBound: an input checked against the limit without embedding it |
returned by ITextEmbedder.Validate |
IChunkingTokenizer, TokenSpan |
Untruncated CountTokens, SpecialTokenCount and GetTokenSpans (UTF-16 offsets of each token): what TextChunker needs |
E5Tokenizer, EmbeddingGemma2Tokenizer, BertTextEmbedder.ChunkingTokenizer |
TextChunker, TextChunkerOptions, TextChunk |
Sliding windows of tokens with overlap over a model's tokenizer, returning chunk offsets; optional pooled embedding (Experimental, see below) | |
InputTooLongException |
An ArgumentException for input above the limit, with InputIndex, TokenCount, MaxTokens and TokenCountIsLowerBound |
thrown by E5Encoder for id sequences above MaxSequenceLength; base of EmbeddingGemma2InputTooLongException and Gliner2InputTooLongException |
Maturity
Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.
| Profile | Maturity | Platforms |
|---|---|---|
all-minilm-l6-v2/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-base-en-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-m3/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-reranker-v2-m3/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-small-en-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
embeddinggemma-2/image-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
embeddinggemma-2/image-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
embeddinggemma-2/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: experimental |
embeddinggemma-2/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
embeddinggemma-2/video-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
embeddinggemma-2/video-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
gliner2-base-v1/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; linux-x64 cuda: supported; windows-x64 cpu: experimental |
gliner2-base-v1/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
kev-0.8b/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested |
kev-0.8b/onnx-fp32-cuda |
experimental | linux-x64 cuda: experimental; windows-x64 cuda: untested |
kev-4b/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested |
kev-4b/onnx-fp32-cuda |
experimental | linux-x64 cuda: experimental; windows-x64 cuda: untested |
lfm3-1.7b/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: untested; windows-x64 cpu: untested |
ms-marco-minilm-l6-v2/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
multilingual-e5-small/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: experimental |
multilingual-e5-small/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
nomic-embed-text-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
qwen2.5-0.5b-instruct/ortgenai-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
qwen3-0.6b/ortgenai-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
qwen3.5-0.8b/ortgenai-fp16x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
qwen3.5-0.8b/ortgenai-fp32x-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
qwen3.5-0.8b/ortgenai-fp32x-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
qwen3.5-0.8b/ortgenai-int4-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: unsupported; windows-x64 cpu: untested |
whisper-large-v3-turbo/onnx-fp16-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: untested; windows-x64 cpu: untested |
Capability records
ModelProfileCapabilities is generated, not hand-written: model-pipeline catalog writes each package's ProfileCapabilities.g.cs (in the namespace of the project's RootNamespace) from the registry manifests (model-pipeline/registry/<profile>/manifest.yaml), the same input as the published catalog (docs/catalog/catalog.json), and CI fails when a generated file is stale. Each package's contract tests check every record against its manifest with an independent reader.
| Field | Manifest source | Status |
|---|---|---|
ProfileId |
id |
contract |
Maturity |
maturity |
contract |
MaxInputTokens |
behavior.max_input_tokens |
contract |
Truncation (Rejects, TruncatesRight, TruncatesLeft, Chunks) |
behavior.truncation |
contract |
Dimension (null for a profile without embeddings) |
behavior.defaults.dimension |
contract |
QualifiedProviders |
providers of the compatibility rows with status supported |
contract |
Tasks |
behavior.tasks |
informational |
Platforms |
every compatibility row |
informational |
StopTokenIds |
behavior.defaults.stop_token_ids |
informational |
Capabilities |
the capabilities map, key for key |
informational |
Contract fields of a Stable profile change only with a new profile or a release note under the support policy. The fine-grained, package-wide keys (for example batch.token_budget) stay in each package's XCapabilities.Statuses.
using Starlings.Models;
using Starlings.Models.E5;
ModelProfileCapabilities e5 = E5Capabilities.ProfileCapabilities[E5Capabilities.StableProfile];
// e5.MaxInputTokens == 512, e5.Truncation == TruncationPolicy.TruncatesRight, e5.Dimension == 384
static float[][] EmbedAll(ITextEmbedder embedder, IReadOnlyList<string> texts)
{
var counts = new int[texts.Count];
return embedder.EmbedBatch(texts, counts);
}
Token counting and chunking (experimental)
Every text embedder counts tokens without embedding: CountTokens(text) and CountTokens(texts) give the tokens a text needs, special tokens included, before any truncation (so a truncating profile can report more than MaxSequenceLength). Validate(text) returns a TokenLimitCheck and never throws; ExceedsLimit means the profile would truncate the text (e5, BERT) or reject it (EmbeddingGemma 2). An implementation may skip tokenization when the length alone proves a text too long; TokenCountIsLowerBound is then true. Capability token_counting: experimental.
TextChunker splits a long text into windows of at most MaxTokens tokens (special tokens and Prefix included), each repeating OverlapTokens tokens of the previous one, using the model's own tokenizer through IChunkingTokenizer. Each TextChunk has Start, Length, Text and TokenCount; every chunk is tokenized again with the prefix and shrunk until it fits, because a substring can need more tokens alone than inside the whole text. Text between two tokens at a chunk boundary (usually a space) belongs to neither chunk.
var chunker = new TextChunker(e5.ChunkingTokenizer!, new TextChunkerOptions
{
MaxTokens = e5.MaxSequenceLength,
OverlapTokens = 32,
Prefix = E5TextEmbedder.PassagePrefix,
});
IReadOnlyList<TextChunk> chunks = chunker.Chunk(document);
float[] pooled = chunker.EmbedPooled(e5, document);
EmbedPooled embeds every chunk (prefix prepended) in one EmbedBatch call and returns the mean of the L2-normalized chunk vectors, L2-normalized again (TextChunker.Pool). A pooled vector is not the reference model's output and carries no parity claim: the reference truncates or rejects such text, and no fidelity gate covers pooled vectors. Each chunk's own embedding is the model's output for that chunk text, with the profile's usual fidelity. Capability chunking: experimental.
Reranking retrieval results (experimental)
Microsoft.Extensions.AI has no reranker abstraction, so Reranking connects an IReranker (today BertReranker, for example bge-reranker-v2-m3) to a retrieval pipeline without referencing Microsoft.Extensions.VectorData or Microsoft.Extensions.AI: the candidates are any sequence, and a selector picks the text to score.
reranker.AsScoringFunction(cancellationToken)returns aFunc<string, IReadOnlyList<string>, Task<float[]>>:Score(query, documents)as a delegate, for pipelines that take a scorer rather than anIReranker.reranker.RerankAsync(query, candidates, text, top, cancellationToken)reads anIAsyncEnumerable<T>(such as the results ofSearchAsync) to the end, scores all candidates with oneScorecall, and returnsRerankedItem<T>(Item, Index, Score)by descending score, ties in input order (a NaN score last), keeping at mosttop.Reranking.RerankAsync(query, candidates, text, score, top, cancellationToken)does the same with any scorer delegate.reranker.Rerank(...)is the synchronous form over anIEnumerable<T>.
Rerank scores on the calling thread. The delegate from AsScoringFunction scores synchronously and returns a completed task (failures and cancellation are in the task, never thrown synchronously), as the Starlings IEmbeddingGenerator implementations do by default. RerankAsync validates its arguments synchronously, then scores synchronously after the last candidate arrives, on the thread that resumed it: with a vector store whose SearchAsync completes asynchronously, that is a thread-pool thread, and the returned task is not complete when the method returns. Wrap the call in Task.Run if scoring must not run on the thread that reads the results. No candidates means no reranker call and an empty result. A text selector that returns null throws ArgumentException; a scorer whose score count differs from the candidate count throws InvalidOperationException. Scores are the reranker's raw output (a logit for BertReranker), so RerankedItem<T>.Score is comparable only within one call; the first-stage score stays available on the item.
using Microsoft.Extensions.VectorData;
using Starlings.Models;
using Starlings.Models.Bert;
// collection: a VectorStoreCollection<string, Passage> whose embedding generator is, for example, e5's.
using var reranker = BertReranker.Create(BertModelProfile.BuiltIn("bge-reranker-v2-m3"), rerankerSnapshot);
string question = "Which river flows through Cologne?";
IReadOnlyList<RerankedItem<VectorSearchResult<Passage>>> best = await reranker.RerankAsync(
question,
collection.SearchAsync("query: " + question, top: 20), // the embedder's top-k, with e5's query prefix
hit => hit.Record.Text, // the reranker scores the plain query and passage text
top: 3);
foreach (var r in best)
Console.WriteLine($"{r.Score:F3} (was #{r.Index + 1}, {r.Item.Score:F3}) {r.Item.Record.Text}");
The helpers are API only: their maturity is the reranker profile's (every reranker profile is Experimental; see the Starlings.Models.Bert README).
Migrating from the per-package types
The per-package status enums (Starlings.Models.E5.CapabilityStatus, EmbeddingGemma2CapabilityStatus, Gliner2CapabilityStatus), the tables typed with them (XCapabilities.All, XCapabilities.Profiles) and EmbeddingGemma2InputTooLongException.MaximumTokens are marked [Obsolete] and are removed in the next minor version. Use Starlings.Models.CapabilityStatus, XCapabilities.Statuses, XCapabilities.ProfileCapabilities[profile].Capabilities and MaxTokens. The enum values have the same names and order, so a cast keeps the meaning. A file that imports both Starlings.Models and Starlings.Models.E5 sees two types named CapabilityStatus until the obsolete one is removed; qualify it or use an alias (using Status = Starlings.Models.CapabilityStatus;).
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Starlings.Acquisition (= 0.5.0)
NuGet packages (8)
Showing the top 5 NuGet packages that depend on Starlings.Models.Abstractions:
| Package | Downloads |
|---|---|
|
Starlings.Models.EmbeddingGemma2
google/embeddinggemma-2 (revision 914f7f89) text and image embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers, profile gemma-bpe-embeddinggemma2: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right-padded batching, mean pooling over every token including the prompt, L2 normalization, Matryoshka sizes 768/512/256/128, explicit named prompts (never added silently), inputs above 8,192 tokens rejected. Images: image-only and interleaved text-and-image embeddings from already-decoded RGB (no native dependency; the upstream processor restated bit for bit, including PyTorch's uint8 antialiased bicubic resize), default budget 280 soft tokens, vision graph plus multimodal text graph; text-only input keeps the released text graph. Videos: video-only and interleaved text/image/video embeddings from decoded RGB frames at the checkpoint settings (1 fps, at most 32 frames, 140 soft tokens per frame, no timestamps; upstream sampling restated exactly; frames without timing rejected unless opted in), same graphs; decoding is the optional FFmpeg adapter. Audio is not supported. Ships the asset locks (Starling's ONNX graphs and the pinned tokenizer.json), not the weights; acquire them explicitly. Profiles embeddinggemma-2/onnx-fp32-cpu, onnx-fp32-cuda, image-fp32-cpu, image-fp32-cuda, video-fp32-cpu and video-fp32-cuda (Linux x64). Apache-2.0 weights; deployments must adhere to the Gemma Prohibited Use Policy. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.E5
intfloat/multilingual-e5-small (revision 614241f6) sentence embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right padding, mean pooling, L2 normalization. Ships the model's asset lock, not the weights; acquire them explicitly. Profiles multilingual-e5-small/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Gliner2
fastino/gliner2-base-v1 (revision f9634218, GLiNER2 2.0.0) English entity extraction with labels and a threshold on ONNX Runtime: GLiNER2's Python text semantics (normalization, word splitter, lower-casing) and schema prefix through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer), one ONNX graph, GLiNER2's span decoding, offsets into the original text. Ships the asset lock, not the weights; acquire them explicitly. Profiles gliner2-base-v1/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Qwen35.OnnxGenAI
Qwen/Qwen3.5-0.8B (revision 2fc06364) chat generation in-process on ONNX Runtime GenAI 0.17.1: Qwen35ChatModel over the model definition Starlings.Models.Qwen35 (the pinned chat template rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers profile qwen35-bpe-chat: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy or seeded sampling, stop tokens and strings, streaming, cancellation between steps, usage). Ships the asset lock of the ONNX Runtime GenAI bundle, not the weights. Profiles qwen3.5-0.8b/ortgenai-fp32x (CPU; CUDA with TF32 off) and ortgenai-fp16x (CUDA), Linux x64, Stable. The application adds Microsoft.ML.OnnxRuntimeGenAI or Microsoft.ML.OnnxRuntimeGenAI.Cuda. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.QwenChat.OnnxGenAI
Qwen/Qwen3-0.6B (revision c1899de2, thinking disabled) and Qwen/Qwen2.5-0.5B-Instruct (revision 7ae55760) text chat generation in-process on ONNX Runtime GenAI 0.17.1: QwenChatModel over the model definitions Starlings.Models.QwenChat (the pinned chat templates rendered in C#, the tool-call parser, prompt ids from the owned tokenizer Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no GenAI tokenizer) and Starling's decode loop (greedy, stop tokens and strings, streaming, cancellation between steps, usage). Ships no weights: QwenChatModelAssets downloads the profile's bundle explicitly from Starling's public Hugging Face hosting (or it is built locally from the pinned original), and Load verifies every file against the profile's digests. Profiles qwen3-0.6b/ortgenai-fp32-cpu and qwen2.5-0.5b-instruct/ortgenai-fp32-cpu, Linux x64. The application adds Microsoft.ML.OnnxRuntimeGenAI. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
GitHub repositories
This package is not used by any popular GitHub repositories.