Starlings.Runtime.Onnx 0.5.0

dotnet add package Starlings.Runtime.Onnx --version 0.5.0
                    
NuGet\Install-Package Starlings.Runtime.Onnx -Version 0.5.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Starlings.Runtime.Onnx" Version="0.5.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="Starlings.Runtime.Onnx" Version="0.5.0" />
                    
Directory.Packages.props
<PackageReference Include="Starlings.Runtime.Onnx" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add Starlings.Runtime.Onnx --version 0.5.0
                    
#r "nuget: Starlings.Runtime.Onnx, 0.5.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package Starlings.Runtime.Onnx@0.5.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=Starlings.Runtime.Onnx&version=0.5.0
                    
Install as a Cake Addin
#tool nuget:?package=Starlings.Runtime.Onnx&version=0.5.0
                    
Install as a Cake Tool

Starlings.Runtime.Onnx

ONNX Runtime session ownership for Starlings models: provider selection with no silent fallback (CPU, or CUDA with TF32 always off), native/managed version check, diagnostics (versions, providers, options, model SHA-256, graph signature), cancellation (at node boundaries on CPU, after the current run on CUDA) and reference-counted disposal. The API may still change.

Maturity

Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.

Profile Maturity Platforms
all-minilm-l6-v2/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
bge-base-en-v1.5/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
bge-m3/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
bge-reranker-v2-m3/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
bge-small-en-v1.5/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
embeddinggemma-2/image-fp32-cpu stable linux-x64 cpu: supported; windows-x64 cpu: untested
embeddinggemma-2/image-fp32-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
embeddinggemma-2/onnx-fp32-cpu stable linux-x64 cpu: supported; windows-x64 cpu: experimental
embeddinggemma-2/onnx-fp32-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
embeddinggemma-2/video-fp32-cpu stable linux-x64 cpu: supported; windows-x64 cpu: untested
embeddinggemma-2/video-fp32-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
gliner2-base-v1/onnx-fp32-cpu stable linux-x64 cpu: supported; linux-x64 cuda: supported; windows-x64 cpu: experimental
gliner2-base-v1/onnx-fp32-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
kev-0.8b/onnx-fp32-cpu experimental linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested
kev-0.8b/onnx-fp32-cuda experimental linux-x64 cuda: experimental; windows-x64 cuda: untested
kev-4b/onnx-fp32-cpu experimental linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested
kev-4b/onnx-fp32-cuda experimental linux-x64 cuda: experimental; windows-x64 cuda: untested
ms-marco-minilm-l6-v2/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
multilingual-e5-small/onnx-fp32-cpu stable linux-x64 cpu: supported; windows-x64 cpu: experimental
multilingual-e5-small/onnx-fp32-cuda stable linux-x64 cuda: supported; windows-x64 cuda: untested
nomic-embed-text-v1.5/onnx-fp32-cpu experimental linux-x64 cpu: experimental; windows-x64 cpu: untested
whisper-large-v3-turbo/onnx-fp16-cpu experimental linux-x64 cpu: experimental; linux-x64 cuda: untested; windows-x64 cpu: untested

API

using Microsoft.ML.OnnxRuntime;
using Starlings.Runtime.Onnx;

using var session = OnnxModelSession.Create("model.onnx", new OnnxSessionOptions
{
    Provider = OnnxExecutionProvider.Cpu,          // or Cuda; no fallback either way
    ExpectedModelSha256 = "<lowercase hex sha256>", // optional; mismatch throws OnnxModelIntegrityException
});
Console.WriteLine(session.Diagnostics.OrtVersion);
using var outputs = session.Run(inputNames, inputValues, outputNames, cancellationToken);
  • OnnxModelSession.Create(modelPath | byte[] model, options) loads once; Run is safe to call concurrently. One Run overload returns outputs the caller owns and disposes (before the session); another writes into caller-allocated OrtValues; a third takes inputs as a dictionary keyed by input name (Run(inputs, outputNames = null), null meaning every graph output). Input values stay caller-owned. Inputs, Outputs, Diagnostics, Placement and IsDisposed describe the session. Dispose is idempotent; runs already executing complete first.
  • I/O binding: CreateIoBinding() returns an ONNX Runtime OrtIoBinding (caller-owned, one per concurrent caller, disposed before the session); RunWithBinding(binding, cancellationToken) runs on the bound values with the same cancellation, placement and lifetime rules as Run. CreateDeviceMemoryInfo() describes the session's device (the CUDA device, or CPU memory) for BindOutputToDevice. On CUDA this is how inputs already on the device avoid the per-run copy. Status: experimental on CPU, untested on CUDA.
  • Warmup: Warmup(inputs, runs = 1) runs on given inputs and discards the outputs; WarmupWithZeroInputs(symbolicDimensions = null, runs = 1) builds zero-filled tensors from the graph signature (each symbolic dimension from the dictionary, or 1). Both return each run's duration. With CapturePlacement the first warmup run is the profiled one. Status: experimental.
  • OnnxSessionOptions (immutable record): Provider (default Cpu), CudaDeviceId (default 0), Cuda (typed CUDA provider options, below), IntraOpNumThreads and InterOpNumThreads (null: defaults, below), ExecutionMode (default Sequential), GraphOptimizationLevel (default All), OptimizedModelFilePath (default null), LogId and LogSeverity (default Error), ExpectedModelSha256, ComputeModelSha256 (default true), KnownModelSha256 (a digest the caller already established, for example ModelSnapshot.GetVerifiedSha256: reported as Diagnostics.ModelSha256 with ModelSha256Source = Known and compared with ExpectedModelSha256 without reading the file again), CapturePlacement (default false; when true, profiles the first run and publishes Placement, an OnnxPlacementReport of nodes per provider), and the memory settings under "Memory on Linux".
  • OnnxSessionDiagnostics: native and managed versions, available and session providers, provider options, thread counts, execution mode, memory settings, graph optimization level, optimized model path, log id and severity, model source, size, SHA-256, metadata, and input and output signatures (OnnxTensorInfo).
  • OnnxInterventionDiagnostics (experimental): the intervention identity of a session, a companion to OnnxSessionDiagnostics built by the model package that loaded it. ForOriginalGraph(session) reports interventions off; ForDerivedGraph(session, originalModelSha256, derivedGraphSha256, profileIds, interventionInputNames) reports the original (base) graph identity, the loaded derived graph identity (compared with the session's digest when it has one, and a mismatch throws OnnxModelIntegrityException; LoadedModelSha256Source says whether the session hashed the file (Computed), took the caller's KnownModelSha256 without re-reading it (Known), or had no digest (null)), the profile ids, and the graph inputs split into InterventionInputs and ModelInputs; a declared input the graph lacks throws ArgumentException. A derived intervention graph loads through the ordinary Create(path, options), and OnnxSessionOptions has no intervention setting. Design: docs/interventions/architecture.md.
  • OnnxRuntimeInfo: NativeVersion, ManagedVersion, AvailableProviders, CpuProviderName, CudaProviderName, TryInitializeProvider and ConfigureLogging (below).
  • Exceptions: OnnxRuntimeUnavailableException (native library missing or of another version), OnnxProviderUnavailableException (the requested provider cannot be used; never followed by a fallback), OnnxModelIntegrityException (SHA-256 mismatch). All three derive from OnnxSessionException, so one catch (OnnxSessionException) handles them. Invalid options throw ArgumentException; a cancelled run throws OperationCanceledException and the session stays usable; a failing run throws ONNX Runtime's OnnxRuntimeException.

Provider probe

OnnxRuntimeInfo.TryInitializeProvider(provider, deviceId, out problem) (or TryInitializeProvider(sessionOptions, out problem), which also passes OnnxSessionOptions.Cuda) checks without loading a model that the native library loads and matches the managed binding and, for CUDA, that the native package has the CUDA provider and that libonnxruntime_providers_cuda with its CUDA and cuDNN dependencies loads and accepts the options. It returns false with the reason that session creation would throw. It allocates no device memory, so a missing, busy or full device can still fail at session creation. On the CPU native package it returns false for CUDA with the package to reference instead. Status: experimental on CPU; untested on CUDA (no GPU run).

CUDA provider options

OnnxSessionOptions.Cuda (OnnxCudaProviderOptions) passes typed options to the CUDA provider; each null member keeps ONNX Runtime's default:

Member ONNX Runtime option Values
GpuMemoryLimitBytes gpu_mem_limit positive bytes
ArenaExtendStrategy arena_extend_strategy NextPowerOfTwo (kNextPowerOfTwo), SameAsRequested (kSameAsRequested)
CudnnConvolutionAlgorithmSearch cudnn_conv_algo_search Exhaustive, Heuristic, Default
EnableCudaGraph enable_cuda_graph true or false; needs every node on CUDA, fixed shapes and I/O binding at fixed device addresses

The options are validated on every build, CPU-only included, and ignored for a CPU session. use_tf32=0 is always set and is not an option. A non-default cuDNN algorithm search can change convolution results in the last bits; the qualified CUDA profiles were measured with the defaults, and no model package's parity claim covers other values. Diagnostics.ProviderOptions reports the options as ONNX Runtime returns them. Status: untested on CUDA (no GPU run with these options).

AdditionalProviderOptions passes further CUDA provider options as raw key-value strings, after the typed ones, for options the typed members do not cover. Starlings does not validate their names or values: it only rejects a blank key, a null value, or a key that Starlings or a typed member sets (device_id, use_tf32, gpu_mem_limit, arena_extend_strategy, cudnn_conv_algo_search, enable_cuda_graph), even when the typed member is null. ONNX Runtime rejects an unknown key or a bad value when the CUDA provider is created (OnnxProviderUnavailableException). No model package's parity claim covers options passed this way. Status: untested on CUDA.

OnnxSessionOptions.Validate() (and OnnxCudaProviderOptions.Validate()) is public: it runs the same checks as session creation, without a model or GPU, so a configuration can be checked at startup.

Logging, optimization level and the optimized graph

  • LogSeverity (default Error, the level before this option existed) and LogId set the session's ONNX Runtime log filter and logger id.
  • OnnxRuntimeInfo.ConfigureLogging(sink, minimumSeverity = Warning, logId = null) sends ONNX Runtime's messages (OnnxRuntimeLogMessage: severity, category, log id, code location, message) to a delegate instead of standard error. ONNX Runtime takes a logging function only when it creates its process-wide environment, so call it once, before any other ONNX Runtime use in the process; afterwards it throws InvalidOperationException. The sink runs on ONNX Runtime's threads, possibly concurrently; an exception it throws is dropped. Status: experimental.
  • GraphOptimizationLevel (None, Basic, Extended, Layout, All; default All, the level every profile was qualified at). Another level can change results in the last bits. OptimizedModelFilePath writes the optimized graph to a file (never the model file itself); it can contain provider-specific nodes, so it loads with the provider it was written for. Status: experimental.
  • ExecutionMode = Parallel runs independent nodes concurrently on InterOpNumThreads threads; in the default sequential mode ONNX Runtime creates no inter-op pool and InterOpNumThreads is only recorded. Each node computes the same values in either mode. Status: experimental on CPU, untested on CUDA.

DirectML (Windows) and CoreML (macOS) are not offered: they need hardware this repository has no runner for.

The text encoders of Starlings.Models.E5, Starlings.Models.EmbeddingGemma2 and Starlings.Models.Bert share one internal implementation of right-padded packing, mean and CLS pooling with L2 normalization, and the chunk run (EncoderCore). It is not public API.

Native runtime

The package references only Microsoft.ML.OnnxRuntime.Managed (exactly 1.30.0). The application references exactly one native package of the same version:

Deployment Package
CPU Microsoft.ML.OnnxRuntime 1.30.0
CUDA, Linux x64 Microsoft.ML.OnnxRuntime.Gpu.Linux 1.30.0 (needs the CUDA 13 runtime, cuBLAS and cuDNN 9 on the loader path; not shipped)

Both ship libonnxruntime under the same file name. Without a native package, or with a different version, session creation fails with OnnxRuntimeUnavailableException naming the package to add.

Native package check

The package ships MSBuild targets in buildTransitive/, so every executable or test project that references it, directly or through a model package, checks its package graph after restore: warning STARLINGS001 when no ONNX Runtime native package is present, error STARLINGS002 when both Microsoft.ML.OnnxRuntime and a Microsoft.ML.OnnxRuntime.Gpu* package are (a GenAI native package counts through the ONNX Runtime package it brings). Class libraries and browser projects (browser-* runtime identifiers, or Starlings.Runtime.OnnxWeb in the graph) are not checked. Turn it off with <StarlingsOnnxRuntimeNativeCheck>false</StarlingsOnnxRuntimeNativeCheck>, for example when the application deploys its own ONNX Runtime library. CUDA build recipe, Linux GPU dependencies and Native AOT status: native runtimes guide.

Threads in containers

When IntraOpNumThreads is not set and a CPU quota or affinity mask limits the process (Environment.ProcessorCount below the host's online CPUs on Linux), the session uses Environment.ProcessorCount intra-op threads. ONNX Runtime's own default ignores a container's CPU quota and starts one thread per physical core of the host (147 threads, each with its own glibc malloc arena, for one CUDA e5 session in a 31-CPU container on a 256-CPU host). On an unconstrained host the ONNX Runtime default is kept. OnnxSessionDiagnostics.IntraOpNumThreads reports the count used. Evidence: CUDA soak growth.

Most applications use this package through a model package such as Starlings.Models.E5. Documentation: ai-dotnet/README.md.

Micro-batching (experimental)

Starlings.Runtime.Onnx.Batching.MicroBatcher<TItem, TResult> coalesces items queued by concurrent callers into batches for one batch function: at most MicroBatchingOptions.MaxRows items and MaxTokens padded token cells (rows x longest row, from a per-item token count), dispatched when full or when the oldest item has waited MaxWait; an optional group key keeps items that must not share a batch apart. One dispatcher thread runs one batch at a time with a token the batcher owns: a caller's cancellation removes its queued item, or cancels only its own task once dispatched (the batch token is cancelled only when every caller of the batch has cancelled). At most QueueCapacity items wait; further callers wait for a slot (backpressure). An exception from the batch function faults every caller of that batch, so callers validate items before queueing them. Disposal faults queued items with ObjectDisposedException, lets a running batch finish and waits for the dispatcher. The e5 and EmbeddingGemma 2 embedding generators and the GLiNER2 extractor expose it as an opt-in MicroBatching option.

Memory on Linux

When a session's native memory is released (after Dispose and the last in-flight run), the package calls glibc's malloc_trim(0) on Linux. glibc otherwise keeps the freed memory of the session (graph, initializers, ONNX Runtime's CPU arena, run buffers) resident in its heaps, so a process that loads and unloads models keeps up to about a model's worth of freed memory per malloc arena. On the hosted qualification runner the resident memory after unloading fell from about 1,480 to 400 to 500 MiB for EmbeddingGemma 2 text and from about 1,110 to 260 MiB for GLiNER2 without MALLOC_MMAP_THRESHOLD_ (evidence).

  • malloc_trim only returns memory that is already free. It changes no allocator setting (unlike mallopt or the MALLOC_* environment variables) and touches no live allocation. It runs once per released session, not on the request path; it costs a few milliseconds per release, and the next load faults the returned pages in again. The opt-in trims after load and after each run (below) call the same function.

  • It is a no-op on Windows and macOS, and on Linux without glibc (musl: the first release finds no libc.so.6 and turns it off; OnnxSessionDiagnostics.TrimNativeHeapOnRelease can report true for sessions created before that).

  • Opt out per session with OnnxSessionOptions.TrimNativeHeapOnRelease = false, or for the whole process (also for sessions a model package creates) with the AppContext switch Starlings.Runtime.Onnx.DisableNativeHeapTrim, for example in the application's project file:

    <ItemGroup>
      <RuntimeHostConfigurationOption Include="Starlings.Runtime.Onnx.DisableNativeHeapTrim" Value="true" />
    </ItemGroup>
    
  • CPU memory arena: OnnxSessionOptions.CpuMemoryArena turns ONNX Runtime's CPU arena (enable_cpu_mem_arena) on or off; null, the default, keeps ONNX Runtime's default (on) for sessions created directly and lets a model package choose for the sessions it creates (EmbeddingGemma 2 turns it off on CPU). With the arena, ONNX Runtime keeps the CPU buffers of every run in regions it does not return while the session lives; their resident size can climb toward the region size over many runs, and regions of up to 32 MiB freed at release stay resident in glibc's per-thread heaps, which malloc_trim cannot return. Outputs are the same either way. OnnxSessionDiagnostics.CpuMemoryArena reports the effective setting. Evidence: EmbeddingGemma 2 hosted soak.

  • Dedicated run threads: with OnnxSessionOptions.DedicatedRunThreads = true, runs execute on a small process-wide set of long-lived threads owned by this package, and the caller blocks until its run completes. A thread is started only when every existing one is busy, up to Environment.ProcessorCount; further concurrent runs wait for a free one. null, the default, runs on the calling thread for sessions created directly and lets a model package choose (EmbeddingGemma 2 and e5 use them on CPU). Why: without ONNX Runtime's CPU arena, a run allocates and frees its buffers with malloc on the thread that calls it, and glibc serves each thread from its own malloc arena. The free memory at the top of a non-main arena stays resident: malloc_trim does not return it, and free returns it only above the dynamic trim threshold (up to 64 MiB). When runs come from a rotating set of thread-pool threads, more and more arenas each keep 20 to 60 MiB, and resident memory steps up over an hour. On the dedicated threads the same few arenas serve every run, so that memory is reused. Results are the same either way. Cancellation works the same (the token sets ONNX Runtime's terminate flag); a run started from a dedicated thread executes inline. OnnxSessionDiagnostics.DedicatedRunThreads reports the setting. Evidence: EmbeddingGemma 2 soak reliability.

  • Trims after load and after each run: OnnxSessionOptions.TrimNativeHeapAfterLoad calls malloc_trim(0) once when the session has been created, and OnnxSessionOptions.TrimNativeHeapAfterRun calls it on the thread that ran, after each run. null, the default, means off for sessions created directly and lets a model package choose (the EmbeddingGemma 2 image and video path turns both on for its CPU sessions). Why: creating a large session allocates and frees copies of the graph and its initializers, and glibc keeps that memory resident while the session lives (about 400 MiB for the EmbeddingGemma 2 vision and multimodal sessions, varying from load to load). A run without ONNX Runtime's arena leaves its freed buffers as free chunks in the running thread's heap (60 to 190 MiB per thread for EmbeddingGemma 2 images, rising in steps). The trim returns both; it cannot return the top chunk of a non-main heap. The run trim is part of the run (milliseconds) and the next run faults the pages in again, so it suits runs that take seconds, not short ones. The AppContext switch above turns both off. OnnxSessionDiagnostics.TrimNativeHeapAfterLoad and TrimNativeHeapAfterRun report them. Evidence: EmbeddingGemma 2 image loaded soak.

  • Transparent huge pages: with /sys/kernel/mm/transparent_hugepage/enabled set to always, the kernel's khugepaged collapses partly used 2 MiB ranges of the heaps into huge pages and fills in their absent pages, so resident memory can rise over time with no allocation behind it. A library cannot change that policy for its host; madvise (the Ubuntu default) or never avoids it.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (6)

Showing the top 5 NuGet packages that depend on Starlings.Runtime.Onnx:

Package Downloads
Starlings.Models.EmbeddingGemma2

google/embeddinggemma-2 (revision 914f7f89) text and image embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers, profile gemma-bpe-embeddinggemma2: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right-padded batching, mean pooling over every token including the prompt, L2 normalization, Matryoshka sizes 768/512/256/128, explicit named prompts (never added silently), inputs above 8,192 tokens rejected. Images: image-only and interleaved text-and-image embeddings from already-decoded RGB (no native dependency; the upstream processor restated bit for bit, including PyTorch's uint8 antialiased bicubic resize), default budget 280 soft tokens, vision graph plus multimodal text graph; text-only input keeps the released text graph. Videos: video-only and interleaved text/image/video embeddings from decoded RGB frames at the checkpoint settings (1 fps, at most 32 frames, 140 soft tokens per frame, no timestamps; upstream sampling restated exactly; frames without timing rejected unless opted in), same graphs; decoding is the optional FFmpeg adapter. Audio is not supported. Ships the asset locks (Starling's ONNX graphs and the pinned tokenizer.json), not the weights; acquire them explicitly. Profiles embeddinggemma-2/onnx-fp32-cpu, onnx-fp32-cuda, image-fp32-cpu, image-fp32-cuda, video-fp32-cpu and video-fp32-cuda (Linux x64). Apache-2.0 weights; deployments must adhere to the Gemma Prohibited Use Policy. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.E5

intfloat/multilingual-e5-small (revision 614241f6) sentence embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right padding, mean pooling, L2 normalization. Ships the model's asset lock, not the weights; acquire them explicitly. Profiles multilingual-e5-small/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.Gliner2

fastino/gliner2-base-v1 (revision f9634218, GLiNER2 2.0.0) English entity extraction with labels and a threshold on ONNX Runtime: GLiNER2's Python text semantics (normalization, word splitter, lower-casing) and schema prefix through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer), one ONNX graph, GLiNER2's span decoding, offsets into the original text. Ships the asset lock, not the weights; acquire them explicitly. Profiles gliner2-base-v1/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.Kev.Onnx

Kev decision models (jaredpalmer/kev-0.8b and jaredpalmer/kev-4b, Kev 1.0) on ONNX Runtime: one state and typed questions (noul, choice, score) in, a calibrated probability per option out, in one prefill pass without generating text. KevDecisionModel over the model definitions Starlings.Models.Kev (request rendering, delimiter escaping and token ids from the owned tokenizer Starlings.Tokenizers profile qwen2-bpe-kev), the Qwen3.5 backbone as an fp32 ONNX graph (the LoRA merged; ONNX Runtime GenAI builder without the LM head), the state run once and every question as its own row on the cached state, and Kev's pointer head. Ships the asset locks, not the weights. Profiles kev-0.8b/onnx-fp32-cpu and kev-4b/onnx-fp32-cpu, and onnx-fp32-cuda on the same bundles (TF32 off). The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

Starlings.Models.Lfm3

MODEL LICENSE: LFM Open License v1.0, not MIT or Apache-2.0. Commercial use of LFM3 is licensed only to entities below USD 10 million annual revenue (section 5); read LICENSE.txt in the package before use. LiquidAI/LFM3-1.7B text generation on ONNX Runtime (Liquid's ONNX export LiquidAI/LFM3-1.7B-ONNX, revision c680759a): the LFM3 chat template for text turns, a managed greedy decode loop over the export's step graph with its convolution and attention caches, and the owned tokenizer (Starlings.Tokenizers profile lfm3-bpe-chat). Ships the asset lock, not the weights; acquire them explicitly. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.5.0 273 11/3/2026
0.4.0 220 11/3/2026
0.3.0 247 10/19/2026
0.2.0 258 10/17/2026
0.1.0-preview.6 150 10/14/2026