Starlings.Runtime.Onnx
0.5.0
dotnet add package Starlings.Runtime.Onnx --version 0.5.0
NuGet\Install-Package Starlings.Runtime.Onnx -Version 0.5.0
<PackageReference Include="Starlings.Runtime.Onnx" Version="0.5.0" />
<PackageVersion Include="Starlings.Runtime.Onnx" Version="0.5.0" />
<PackageReference Include="Starlings.Runtime.Onnx" />
paket add Starlings.Runtime.Onnx --version 0.5.0
#r "nuget: Starlings.Runtime.Onnx, 0.5.0"
#:package Starlings.Runtime.Onnx@0.5.0
#addin nuget:?package=Starlings.Runtime.Onnx&version=0.5.0
#tool nuget:?package=Starlings.Runtime.Onnx&version=0.5.0
Starlings.Runtime.Onnx
ONNX Runtime session ownership for Starlings models: provider selection with no silent fallback (CPU, or CUDA with TF32 always off), native/managed version check, diagnostics (versions, providers, options, model SHA-256, graph signature), cancellation (at node boundaries on CPU, after the current run on CUDA) and reference-counted disposal. The API may still change.
Maturity
Maturity is per profile, generated from the registry manifests by model-pipeline catalog (do not edit this table by hand). Gates, evidence and capabilities per profile: the model catalog.
| Profile | Maturity | Platforms |
|---|---|---|
all-minilm-l6-v2/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-base-en-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-m3/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-reranker-v2-m3/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
bge-small-en-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
embeddinggemma-2/image-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
embeddinggemma-2/image-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
embeddinggemma-2/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: experimental |
embeddinggemma-2/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
embeddinggemma-2/video-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: untested |
embeddinggemma-2/video-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
gliner2-base-v1/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; linux-x64 cuda: supported; windows-x64 cpu: experimental |
gliner2-base-v1/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
kev-0.8b/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested |
kev-0.8b/onnx-fp32-cuda |
experimental | linux-x64 cuda: experimental; windows-x64 cuda: untested |
kev-4b/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: experimental; windows-x64 cpu: untested |
kev-4b/onnx-fp32-cuda |
experimental | linux-x64 cuda: experimental; windows-x64 cuda: untested |
ms-marco-minilm-l6-v2/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
multilingual-e5-small/onnx-fp32-cpu |
stable | linux-x64 cpu: supported; windows-x64 cpu: experimental |
multilingual-e5-small/onnx-fp32-cuda |
stable | linux-x64 cuda: supported; windows-x64 cuda: untested |
nomic-embed-text-v1.5/onnx-fp32-cpu |
experimental | linux-x64 cpu: experimental; windows-x64 cpu: untested |
whisper-large-v3-turbo/onnx-fp16-cpu |
experimental | linux-x64 cpu: experimental; linux-x64 cuda: untested; windows-x64 cpu: untested |
API
using Microsoft.ML.OnnxRuntime;
using Starlings.Runtime.Onnx;
using var session = OnnxModelSession.Create("model.onnx", new OnnxSessionOptions
{
Provider = OnnxExecutionProvider.Cpu, // or Cuda; no fallback either way
ExpectedModelSha256 = "<lowercase hex sha256>", // optional; mismatch throws OnnxModelIntegrityException
});
Console.WriteLine(session.Diagnostics.OrtVersion);
using var outputs = session.Run(inputNames, inputValues, outputNames, cancellationToken);
OnnxModelSession.Create(modelPath | byte[] model, options)loads once;Runis safe to call concurrently. OneRunoverload returns outputs the caller owns and disposes (before the session); another writes into caller-allocatedOrtValues; a third takes inputs as a dictionary keyed by input name (Run(inputs, outputNames = null), null meaning every graph output). Input values stay caller-owned.Inputs,Outputs,Diagnostics,PlacementandIsDisposeddescribe the session.Disposeis idempotent; runs already executing complete first.- I/O binding:
CreateIoBinding()returns an ONNX RuntimeOrtIoBinding(caller-owned, one per concurrent caller, disposed before the session);RunWithBinding(binding, cancellationToken)runs on the bound values with the same cancellation, placement and lifetime rules asRun.CreateDeviceMemoryInfo()describes the session's device (the CUDA device, or CPU memory) forBindOutputToDevice. On CUDA this is how inputs already on the device avoid the per-run copy. Status: experimental on CPU, untested on CUDA. - Warmup:
Warmup(inputs, runs = 1)runs on given inputs and discards the outputs;WarmupWithZeroInputs(symbolicDimensions = null, runs = 1)builds zero-filled tensors from the graph signature (each symbolic dimension from the dictionary, or 1). Both return each run's duration. WithCapturePlacementthe first warmup run is the profiled one. Status: experimental. OnnxSessionOptions(immutable record):Provider(defaultCpu),CudaDeviceId(default 0),Cuda(typed CUDA provider options, below),IntraOpNumThreadsandInterOpNumThreads(null: defaults, below),ExecutionMode(defaultSequential),GraphOptimizationLevel(defaultAll),OptimizedModelFilePath(default null),LogIdandLogSeverity(defaultError),ExpectedModelSha256,ComputeModelSha256(default true),KnownModelSha256(a digest the caller already established, for exampleModelSnapshot.GetVerifiedSha256: reported asDiagnostics.ModelSha256withModelSha256Source = Knownand compared withExpectedModelSha256without reading the file again),CapturePlacement(default false; when true, profiles the first run and publishesPlacement, anOnnxPlacementReportof nodes per provider), and the memory settings under "Memory on Linux".OnnxSessionDiagnostics: native and managed versions, available and session providers, provider options, thread counts, execution mode, memory settings, graph optimization level, optimized model path, log id and severity, model source, size, SHA-256, metadata, and input and output signatures (OnnxTensorInfo).OnnxInterventionDiagnostics(experimental): the intervention identity of a session, a companion toOnnxSessionDiagnosticsbuilt by the model package that loaded it.ForOriginalGraph(session)reports interventions off;ForDerivedGraph(session, originalModelSha256, derivedGraphSha256, profileIds, interventionInputNames)reports the original (base) graph identity, the loaded derived graph identity (compared with the session's digest when it has one, and a mismatch throwsOnnxModelIntegrityException;LoadedModelSha256Sourcesays whether the session hashed the file (Computed), took the caller'sKnownModelSha256without re-reading it (Known), or had no digest (null)), the profile ids, and the graph inputs split intoInterventionInputsandModelInputs; a declared input the graph lacks throwsArgumentException. A derived intervention graph loads through the ordinaryCreate(path, options), andOnnxSessionOptionshas no intervention setting. Design: docs/interventions/architecture.md.OnnxRuntimeInfo:NativeVersion,ManagedVersion,AvailableProviders,CpuProviderName,CudaProviderName,TryInitializeProviderandConfigureLogging(below).- Exceptions:
OnnxRuntimeUnavailableException(native library missing or of another version),OnnxProviderUnavailableException(the requested provider cannot be used; never followed by a fallback),OnnxModelIntegrityException(SHA-256 mismatch). All three derive fromOnnxSessionException, so onecatch (OnnxSessionException)handles them. Invalid options throwArgumentException; a cancelled run throwsOperationCanceledExceptionand the session stays usable; a failing run throws ONNX Runtime'sOnnxRuntimeException.
Provider probe
OnnxRuntimeInfo.TryInitializeProvider(provider, deviceId, out problem) (or TryInitializeProvider(sessionOptions, out problem), which also passes OnnxSessionOptions.Cuda) checks without loading a model that the native library loads and matches the managed binding and, for CUDA, that the native package has the CUDA provider and that libonnxruntime_providers_cuda with its CUDA and cuDNN dependencies loads and accepts the options. It returns false with the reason that session creation would throw. It allocates no device memory, so a missing, busy or full device can still fail at session creation. On the CPU native package it returns false for CUDA with the package to reference instead. Status: experimental on CPU; untested on CUDA (no GPU run).
CUDA provider options
OnnxSessionOptions.Cuda (OnnxCudaProviderOptions) passes typed options to the CUDA provider; each null member keeps ONNX Runtime's default:
| Member | ONNX Runtime option | Values |
|---|---|---|
GpuMemoryLimitBytes |
gpu_mem_limit |
positive bytes |
ArenaExtendStrategy |
arena_extend_strategy |
NextPowerOfTwo (kNextPowerOfTwo), SameAsRequested (kSameAsRequested) |
CudnnConvolutionAlgorithmSearch |
cudnn_conv_algo_search |
Exhaustive, Heuristic, Default |
EnableCudaGraph |
enable_cuda_graph |
true or false; needs every node on CUDA, fixed shapes and I/O binding at fixed device addresses |
The options are validated on every build, CPU-only included, and ignored for a CPU session. use_tf32=0 is always set and is not an option. A non-default cuDNN algorithm search can change convolution results in the last bits; the qualified CUDA profiles were measured with the defaults, and no model package's parity claim covers other values. Diagnostics.ProviderOptions reports the options as ONNX Runtime returns them. Status: untested on CUDA (no GPU run with these options).
AdditionalProviderOptions passes further CUDA provider options as raw key-value strings, after the typed ones, for options the typed members do not cover. Starlings does not validate their names or values: it only rejects a blank key, a null value, or a key that Starlings or a typed member sets (device_id, use_tf32, gpu_mem_limit, arena_extend_strategy, cudnn_conv_algo_search, enable_cuda_graph), even when the typed member is null. ONNX Runtime rejects an unknown key or a bad value when the CUDA provider is created (OnnxProviderUnavailableException). No model package's parity claim covers options passed this way. Status: untested on CUDA.
OnnxSessionOptions.Validate() (and OnnxCudaProviderOptions.Validate()) is public: it runs the same checks as session creation, without a model or GPU, so a configuration can be checked at startup.
Logging, optimization level and the optimized graph
LogSeverity(defaultError, the level before this option existed) andLogIdset the session's ONNX Runtime log filter and logger id.OnnxRuntimeInfo.ConfigureLogging(sink, minimumSeverity = Warning, logId = null)sends ONNX Runtime's messages (OnnxRuntimeLogMessage: severity, category, log id, code location, message) to a delegate instead of standard error. ONNX Runtime takes a logging function only when it creates its process-wide environment, so call it once, before any other ONNX Runtime use in the process; afterwards it throwsInvalidOperationException. The sink runs on ONNX Runtime's threads, possibly concurrently; an exception it throws is dropped. Status: experimental.GraphOptimizationLevel(None,Basic,Extended,Layout,All; defaultAll, the level every profile was qualified at). Another level can change results in the last bits.OptimizedModelFilePathwrites the optimized graph to a file (never the model file itself); it can contain provider-specific nodes, so it loads with the provider it was written for. Status: experimental.ExecutionMode = Parallelruns independent nodes concurrently onInterOpNumThreadsthreads; in the default sequential mode ONNX Runtime creates no inter-op pool andInterOpNumThreadsis only recorded. Each node computes the same values in either mode. Status: experimental on CPU, untested on CUDA.
DirectML (Windows) and CoreML (macOS) are not offered: they need hardware this repository has no runner for.
The text encoders of Starlings.Models.E5, Starlings.Models.EmbeddingGemma2 and Starlings.Models.Bert share one internal implementation of right-padded packing, mean and CLS pooling with L2 normalization, and the chunk run (EncoderCore). It is not public API.
Native runtime
The package references only Microsoft.ML.OnnxRuntime.Managed (exactly 1.30.0). The application references exactly one native package of the same version:
| Deployment | Package |
|---|---|
| CPU | Microsoft.ML.OnnxRuntime 1.30.0 |
| CUDA, Linux x64 | Microsoft.ML.OnnxRuntime.Gpu.Linux 1.30.0 (needs the CUDA 13 runtime, cuBLAS and cuDNN 9 on the loader path; not shipped) |
Both ship libonnxruntime under the same file name. Without a native package, or with a different version, session creation fails with OnnxRuntimeUnavailableException naming the package to add.
Native package check
The package ships MSBuild targets in buildTransitive/, so every executable or test project that references it, directly or through a model package, checks its package graph after restore: warning STARLINGS001 when no ONNX Runtime native package is present, error STARLINGS002 when both Microsoft.ML.OnnxRuntime and a Microsoft.ML.OnnxRuntime.Gpu* package are (a GenAI native package counts through the ONNX Runtime package it brings). Class libraries and browser projects (browser-* runtime identifiers, or Starlings.Runtime.OnnxWeb in the graph) are not checked. Turn it off with <StarlingsOnnxRuntimeNativeCheck>false</StarlingsOnnxRuntimeNativeCheck>, for example when the application deploys its own ONNX Runtime library. CUDA build recipe, Linux GPU dependencies and Native AOT status: native runtimes guide.
Threads in containers
When IntraOpNumThreads is not set and a CPU quota or affinity mask limits the process (Environment.ProcessorCount below the host's online CPUs on Linux), the session uses Environment.ProcessorCount intra-op threads. ONNX Runtime's own default ignores a container's CPU quota and starts one thread per physical core of the host (147 threads, each with its own glibc malloc arena, for one CUDA e5 session in a 31-CPU container on a 256-CPU host). On an unconstrained host the ONNX Runtime default is kept. OnnxSessionDiagnostics.IntraOpNumThreads reports the count used. Evidence: CUDA soak growth.
Most applications use this package through a model package such as Starlings.Models.E5. Documentation: ai-dotnet/README.md.
Micro-batching (experimental)
Starlings.Runtime.Onnx.Batching.MicroBatcher<TItem, TResult> coalesces items queued by concurrent callers into batches for one batch function: at most MicroBatchingOptions.MaxRows items and MaxTokens padded token cells (rows x longest row, from a per-item token count), dispatched when full or when the oldest item has waited MaxWait; an optional group key keeps items that must not share a batch apart. One dispatcher thread runs one batch at a time with a token the batcher owns: a caller's cancellation removes its queued item, or cancels only its own task once dispatched (the batch token is cancelled only when every caller of the batch has cancelled). At most QueueCapacity items wait; further callers wait for a slot (backpressure). An exception from the batch function faults every caller of that batch, so callers validate items before queueing them. Disposal faults queued items with ObjectDisposedException, lets a running batch finish and waits for the dispatcher. The e5 and EmbeddingGemma 2 embedding generators and the GLiNER2 extractor expose it as an opt-in MicroBatching option.
Memory on Linux
When a session's native memory is released (after Dispose and the last in-flight run), the package calls glibc's malloc_trim(0) on Linux. glibc otherwise keeps the freed memory of the session (graph, initializers, ONNX Runtime's CPU arena, run buffers) resident in its heaps, so a process that loads and unloads models keeps up to about a model's worth of freed memory per malloc arena. On the hosted qualification runner the resident memory after unloading fell from about 1,480 to 400 to 500 MiB for EmbeddingGemma 2 text and from about 1,110 to 260 MiB for GLiNER2 without MALLOC_MMAP_THRESHOLD_ (evidence).
malloc_trimonly returns memory that is already free. It changes no allocator setting (unlikemalloptor theMALLOC_*environment variables) and touches no live allocation. It runs once per released session, not on the request path; it costs a few milliseconds per release, and the next load faults the returned pages in again. The opt-in trims after load and after each run (below) call the same function.It is a no-op on Windows and macOS, and on Linux without glibc (musl: the first release finds no
libc.so.6and turns it off;OnnxSessionDiagnostics.TrimNativeHeapOnReleasecan reporttruefor sessions created before that).Opt out per session with
OnnxSessionOptions.TrimNativeHeapOnRelease = false, or for the whole process (also for sessions a model package creates) with theAppContextswitchStarlings.Runtime.Onnx.DisableNativeHeapTrim, for example in the application's project file:<ItemGroup> <RuntimeHostConfigurationOption Include="Starlings.Runtime.Onnx.DisableNativeHeapTrim" Value="true" /> </ItemGroup>CPU memory arena:
OnnxSessionOptions.CpuMemoryArenaturns ONNX Runtime's CPU arena (enable_cpu_mem_arena) on or off;null, the default, keeps ONNX Runtime's default (on) for sessions created directly and lets a model package choose for the sessions it creates (EmbeddingGemma 2 turns it off on CPU). With the arena, ONNX Runtime keeps the CPU buffers of every run in regions it does not return while the session lives; their resident size can climb toward the region size over many runs, and regions of up to 32 MiB freed at release stay resident in glibc's per-thread heaps, whichmalloc_trimcannot return. Outputs are the same either way.OnnxSessionDiagnostics.CpuMemoryArenareports the effective setting. Evidence: EmbeddingGemma 2 hosted soak.Dedicated run threads: with
OnnxSessionOptions.DedicatedRunThreads = true, runs execute on a small process-wide set of long-lived threads owned by this package, and the caller blocks until its run completes. A thread is started only when every existing one is busy, up toEnvironment.ProcessorCount; further concurrent runs wait for a free one.null, the default, runs on the calling thread for sessions created directly and lets a model package choose (EmbeddingGemma 2 and e5 use them on CPU). Why: without ONNX Runtime's CPU arena, a run allocates and frees its buffers withmallocon the thread that calls it, and glibc serves each thread from its own malloc arena. The free memory at the top of a non-main arena stays resident:malloc_trimdoes not return it, andfreereturns it only above the dynamic trim threshold (up to 64 MiB). When runs come from a rotating set of thread-pool threads, more and more arenas each keep 20 to 60 MiB, and resident memory steps up over an hour. On the dedicated threads the same few arenas serve every run, so that memory is reused. Results are the same either way. Cancellation works the same (the token sets ONNX Runtime's terminate flag); a run started from a dedicated thread executes inline.OnnxSessionDiagnostics.DedicatedRunThreadsreports the setting. Evidence: EmbeddingGemma 2 soak reliability.Trims after load and after each run:
OnnxSessionOptions.TrimNativeHeapAfterLoadcallsmalloc_trim(0)once when the session has been created, andOnnxSessionOptions.TrimNativeHeapAfterRuncalls it on the thread that ran, after each run.null, the default, means off for sessions created directly and lets a model package choose (the EmbeddingGemma 2 image and video path turns both on for its CPU sessions). Why: creating a large session allocates and frees copies of the graph and its initializers, and glibc keeps that memory resident while the session lives (about 400 MiB for the EmbeddingGemma 2 vision and multimodal sessions, varying from load to load). A run without ONNX Runtime's arena leaves its freed buffers as free chunks in the running thread's heap (60 to 190 MiB per thread for EmbeddingGemma 2 images, rising in steps). The trim returns both; it cannot return the top chunk of a non-main heap. The run trim is part of the run (milliseconds) and the next run faults the pages in again, so it suits runs that take seconds, not short ones. TheAppContextswitch above turns both off.OnnxSessionDiagnostics.TrimNativeHeapAfterLoadandTrimNativeHeapAfterRunreport them. Evidence: EmbeddingGemma 2 image loaded soak.Transparent huge pages: with
/sys/kernel/mm/transparent_hugepage/enabledset toalways, the kernel's khugepaged collapses partly used 2 MiB ranges of the heaps into huge pages and fills in their absent pages, so resident memory can rise over time with no allocation behind it. A library cannot change that policy for its host;madvise(the Ubuntu default) orneveravoids it.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Microsoft.ML.OnnxRuntime.Managed (= 1.30.0)
NuGet packages (6)
Showing the top 5 NuGet packages that depend on Starlings.Runtime.Onnx:
| Package | Downloads |
|---|---|
|
Starlings.Models.EmbeddingGemma2
google/embeddinggemma-2 (revision 914f7f89) text and image embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers, profile gemma-bpe-embeddinggemma2: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right-padded batching, mean pooling over every token including the prompt, L2 normalization, Matryoshka sizes 768/512/256/128, explicit named prompts (never added silently), inputs above 8,192 tokens rejected. Images: image-only and interleaved text-and-image embeddings from already-decoded RGB (no native dependency; the upstream processor restated bit for bit, including PyTorch's uint8 antialiased bicubic resize), default budget 280 soft tokens, vision graph plus multimodal text graph; text-only input keeps the released text graph. Videos: video-only and interleaved text/image/video embeddings from decoded RGB frames at the checkpoint settings (1 fps, at most 32 frames, 140 soft tokens per frame, no timestamps; upstream sampling restated exactly; frames without timing rejected unless opted in), same graphs; decoding is the optional FFmpeg adapter. Audio is not supported. Ships the asset locks (Starling's ONNX graphs and the pinned tokenizer.json), not the weights; acquire them explicitly. Profiles embeddinggemma-2/onnx-fp32-cpu, onnx-fp32-cuda, image-fp32-cpu, image-fp32-cuda, video-fp32-cpu and video-fp32-cuda (Linux x64). Apache-2.0 weights; deployments must adhere to the Gemma Prohibited Use Policy. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.E5
intfloat/multilingual-e5-small (revision 614241f6) sentence embeddings on ONNX Runtime, from text through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer) or from token ids: right padding, mean pooling, L2 normalization. Ships the model's asset lock, not the weights; acquire them explicitly. Profiles multilingual-e5-small/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Gliner2
fastino/gliner2-base-v1 (revision f9634218, GLiNER2 2.0.0) English entity extraction with labels and a threshold on ONNX Runtime: GLiNER2's Python text semantics (normalization, word splitter, lower-casing) and schema prefix through the owned tokenizer (Starlings.Tokenizers: no Microsoft.ML.Tokenizers, no native tokenizer), one ONNX graph, GLiNER2's span decoding, offsets into the original text. Ships the asset lock, not the weights; acquire them explicitly. Profiles gliner2-base-v1/onnx-fp32-cpu and onnx-fp32-cuda. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Kev.Onnx
Kev decision models (jaredpalmer/kev-0.8b and jaredpalmer/kev-4b, Kev 1.0) on ONNX Runtime: one state and typed questions (noul, choice, score) in, a calibrated probability per option out, in one prefill pass without generating text. KevDecisionModel over the model definitions Starlings.Models.Kev (request rendering, delimiter escaping and token ids from the owned tokenizer Starlings.Tokenizers profile qwen2-bpe-kev), the Qwen3.5 backbone as an fp32 ONNX graph (the LoRA merged; ONNX Runtime GenAI builder without the LM head), the state run once and every question as its own row on the cached state, and Kev's pointer head. Ships the asset locks, not the weights. Profiles kev-0.8b/onnx-fp32-cpu and kev-4b/onnx-fp32-cpu, and onnx-fp32-cuda on the same bundles (TF32 off). The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
|
|
Starlings.Models.Lfm3
MODEL LICENSE: LFM Open License v1.0, not MIT or Apache-2.0. Commercial use of LFM3 is licensed only to entities below USD 10 million annual revenue (section 5); read LICENSE.txt in the package before use. LiquidAI/LFM3-1.7B text generation on ONNX Runtime (Liquid's ONNX export LiquidAI/LFM3-1.7B-ONNX, revision c680759a): the LFM3 chat template for text turns, a managed greedy decode loop over the export's step graph with its convolution and attention caches, and the owned tokenizer (Starlings.Tokenizers profile lfm3-bpe-chat). Ships the asset lock, not the weights; acquire them explicitly. The application adds the ONNX Runtime native package. Maturity is per model profile (Stable, Preview or Experimental): see the README, generated from the model registry. |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 0.5.0 | 273 | 11/3/2026 |
| 0.4.0 | 220 | 11/3/2026 |
| 0.3.0 | 247 | 10/19/2026 |
| 0.2.0 | 258 | 10/17/2026 |
| 0.1.0-preview.6 | 150 | 10/14/2026 |