SpawnDev.ILGPU.ML 5.4.0

<div align="center">

SpawnDev.ILGPU.ML

Neural network inference and training in C#, on any GPU, in the browser or on the desktop.

Compute kernels written in C# run as WGSL, GLSL, PTX, OpenCL C, WebAssembly or SIMD CPU code through SpawnDev.ILGPU. No ONNX Runtime. No native inference binaries. No server.

Live demo | Docs | Examples | Changelog | What is verified

<img src="https://raw.githubusercontent.com/LostBeard/SpawnDev.ILGPU.ML/master/SpawnDev.ILGPU.ML.Demo/wwwroot/screenshots/2026-10-21_TextTo3D-Rig-Animate-Astronaut-FLUX3-1024.gif" alt="A text prompt becomes a rigged, animated 3D astronaut, entirely in the browser" width="720">

_{One prompt, one browser tab, no server: Qwen3 writes the caption, FLUX 3 Image Klein draws it, TRELLIS.2 builds the textured mesh, SkinTokens rigs it, Kimodo clips animate it. All of it runs on the visitor's GPU.}

</div>

What it is

SpawnDev.ILGPU.ML is an inference and training engine for .NET whose every layer, from matrix multiply to a 5.6B-parameter image model, is a C# kernel that ILGPU compiles for the device in front of it. The same library, the same kernels and the same models run:

• in the browser (Blazor WebAssembly) on WebGPU, WebGL or WebAssembly, with nothing leaving the user's machine, and
• on the desktop (Windows, Linux, macOS) on CUDA, OpenCL or a SIMD CPU backend.

You hand it a model file. It detects the format, builds a graph, optimizes it and runs it on whatever accelerator you give it.

Highlights

• One codebase, six backends. WebGPU, WebGL, WebAssembly, CUDA, OpenCL and CPU (the Velocity SIMD backend by default on 64-bit desktop). Kernels are tested on all of them.
• Seven model formats, auto-detected. ONNX, TFLite, GGUF, SafeTensors, TensorFlow GraphDef, PyTorch checkpoints and Core ML, all lowered to one graph IR.
• 200+ ONNX operators, a graph optimizer (constant folding, fusion, strength reduction, dead-node elimination) and fused kernels for linear layers, attention, quantized matmul and more.
• Generative models, ported and checked. FLUX 3 Image Klein, FLUX.2 klein, SD-Turbo, TRELLIS.2 (text and image to textured 3D), SkinTokens and UniRig (auto-rigging), Kimodo (character animation). Stages are gated against the reference implementations.
• Language and speech. Quantized GGUF LLMs with KV-cache decode and TurboQuant cache compression, Whisper speech-to-text, Kokoro and ZipVoice text-to-speech with voice cloning.
• Training, not only inference. Backpropagation, SGD and Adam as GPU kernels, so a small network can be trained in a browser tab.
• Zero-copy by design. Pixels, activations and weights stay on the GPU. In the browser, bulk data streams from fetch and OPFS straight into GPU buffers without touching the .NET heap.
• Trim and AOT safe. The trim analyzer runs on every build of the library.

Try it

The live demo is a Blazor WebAssembly app. Everything runs on your GPU; models download once from models.spawndev.com and are cached in your browser.

Not every page is finished. Docs/DEMO_AND_MODEL_STATUS.md lists each demo and model as verified, partial or work in progress, with the test that proves it. If a statement here disagrees with that file, that file wins.

Install

dotnet add package SpawnDev.ILGPU.ML

Requires .NET 10. Targets the browser (Blazor WebAssembly), Windows, Linux and macOS. Depends on SpawnDev.ILGPU and SpawnDev.SpawnJS.

Quick start

1. Create an accelerator

The application owns the accelerator. Library code never disposes it.

// Desktop: the best device available (CUDA, then OpenCL, then CPU)
using var context = MLContext.CreateContext();
using var accelerator = context.GetPreferredDevice(preferCPU: false).CreateAccelerator(context);

// Blazor WebAssembly: WebGPU, falling back to WebGL, then Wasm
using SpawnDev.ILGPU;   // AllAcceleratorsAsync, CreatePreferredAcceleratorAsync

var builder = MLContext.Create();
await builder.AllAcceleratorsAsync();
var context = builder.ToContext();
var accelerator = await context.CreatePreferredAcceleratorAsync();

2. Run any model

The format (ONNX, TFLite, GGUF, SafeTensors, ...) is detected from the file's magic bytes.

using SpawnDev.ILGPU.ML;
using SpawnDev.ILGPU.ML.Tensors;

using var session = InferenceSession.CreateFromFile(accelerator, File.ReadAllBytes("squeezenet.onnx"));

// Inputs are named tensors; outputs come back as a disposable map of GPU-resident tensors you own.
using var input = OwnedTensor<float>.FromHost(accelerator, pixels, new[] { 1, 3, 224, 224 });
using var outputs = await session.RunOwnedAsync(new Dictionary<string, Tensor<float>>
{
    [session.InputNames[0]] = input,
});

float[] logits = await outputs[session.OutputNames[0]].ToHostAsync();

Large models should not be read into a byte[]. InferenceSession.CreateFromStreamAsync(accelerator, stream) takes any seekable stream (a file, an OPFS entry, a ranged HTTP stream) and sends ONNX and GGUF weights to the GPU tensor by tensor. Pipelines such as ClassificationPipeline, DepthEstimationPipeline, SpeechRecognitionPipeline (Whisper) and TextGenerationPipeline wrap a session with the pre- and post-processing for a task.

3. Generate images

ImageGenerator puts FLUX 3 Image Klein, FLUX.2 klein and SD-Turbo behind one call. Weights stream from the models host on first use and are cached on disk (OPFS in the browser).

using SpawnDev.ILGPU.ML.Hub;
using SpawnDev.ILGPU.ML.Pipelines;

using var http = new HttpClient { Timeout = Timeout.InfiniteTimeSpan };
var source = new HttpClientModelSource(http, FileModelStore.Default());

using var gen = new ImageGenerator(accelerator, source, (repo, file, progress) => source.OpenAsync(repo, file, progress));
await gen.LoadAsync(ImageEngine.Flux2);

var pictures = await gen.GenerateAsync(new ImageGenerationRequest
{
    Prompt = "a red fox in fresh snow",
    Engine = ImageEngine.Flux2,
    Size = 512,
    Seed = 1,
});
// pictures[0].Rgba is width x height x 4 bytes

4. Prompt to rigged, animated character

AssetGenerator chains the whole 3D pipeline on a desktop GPU: draw, cut out, build the mesh, rig, animate.

using SpawnDev.ILGPU.ML.Assets;

var (motions, ids) = AssetGenerator.MotionsFromDirectory("motions");
using var gen = new AssetGenerator(accelerator, source, motions);

var pictures = await gen.DrawAsync("a knight in plate armor, full body", new[] { 11, 12, 13, 14 });   // FLUX.2 klein
var (cutOut, _) = await gen.CutOutAsync(pictures[0]);                                                  // RMBG-1.4
var glb    = await gen.BuildModelAsync(cutOut, pictures[0].Width, pictures[0].Height);                 // TRELLIS.2
var rigged = await gen.RigAsync(glb);                                                                  // SkinTokens
var moves  = await gen.AnimateAsync(rigged, new[] { "walk", "dance", "wave" });                        // Kimodo

More: Docs/assetgen.md, Docs/scenes.md (a whole scene from one sentence).

What you can run

Pipelines, models and their verification status are tracked in Docs/DEMO_AND_MODEL_STATUS.md. "The loader reads format X" and "model Y runs end to end" are different claims, and that file keeps them apart.

Model formats

One API loads a model from any ecosystem. Every loader produces the same ModelGraph, so each operator, kernel, optimizer pass and backend works the same whatever the source.

The supported ONNX operator set is the single source of truth OperatorRegistry.BuiltinOpTypes, described in Docs/operators.md, including which operators are complete and which are partial.

Backends

MLContext.CreateCpuAccelerator(context) returns the Velocity accelerator where the context has one and the plain CPU accelerator otherwise. MLContext.CreatePlainCpuContext() gives the slow, exact reference accelerator that tests compare against.

How it works

  model file (.onnx .tflite .gguf .safetensors .pb .pt .mlmodel)
        |  magic-byte detection
        v
  format parser ---> ModelGraph (nodes, weights, shapes)
        |
        v
  GraphOptimizer     constant folding, identity elimination, linear / attention /
        |            layer-norm fusion, strength reduction, dead-node elimination
        v
  GraphCompiler ---> execution plan (200+ operators and fused ops)
        |
        v
  GraphExecutor      topological dispatch, buffer recycling, batched submission
        |
        v
  C# kernels  --- SpawnDev.ILGPU --->  WGSL | GLSL | Wasm | PTX | OpenCL C | SIMD CPU

• Kernels live in SpawnDev.ILGPU.ML/Kernels: tiled and register-blocked matmul, fused dequantizing GEMV for quantized LLMs, flash-style attention, convolution, normalization variants, RoPE, selective scan (Mamba), gated delta nets, mixture of experts, FWHT and TurboQuant, image pre- and post-processing, and the training backward passes.
• Decode fast paths. Autoregressive decode records its dispatch plan once and replays it per token (WebGPU command replay, CUDA graphs), so the host does almost nothing per token.
• Tensors are three small types: Tensor<T> (a shape-tracked view), OwnedTensor<T> (owns its GPU buffer) and TensorView<T> (a blittable struct kernels take directly).
• Model delivery is plain HTTP from models.spawndev.com, a static host with free egress and range requests, with Hugging Face as the automatic fallback. HubModelSource caches in OPFS in the browser; HttpClientModelSource with FileModelStore does it on disk with resumable downloads. The optional SpawnDev.ILGPU.ML.WebTorrent package adds peer-to-peer delivery behind the same IModelSource interface.

Performance

Measured on an RTX 4070 12 GB, models cached. Full methodology and the other numbers are in the linked docs.

Correctness comes first: kernels are checked against CPU references, vision and language models against ONNX Runtime outputs, and the generative models stage by stage against their reference PyTorch implementations (for example SkinTokens: 295 of 296 generated tokens identical, skin weights within 5.7e-5).

Examples

Self-contained projects under Examples/, each runnable with dotnet run --project Examples/<name>.

Repository layout

Documentation

Build and test

dotnet build SpawnDev.ILGPU.ML.slnx

Tests run through PlaywrightMultiTest, which drives the desktop backends and a real Chromium for the browser backends from one command:

dotnet test PlaywrightMultiTest/PlaywrightMultiTest.csproj -c Release

Scope a run with environment variables, for example PMT_LANES=cuda,webgpu for the fast backends or PMT_FILTER=Trellis2_ for one area. Heavy real-model tests are excluded by default and are opt-in by category.

The demo is published with dotnet publish SpawnDev.ILGPU.ML.Demo -c Release -p:DemoAot=true (AOT keeps the CPU-bound stages of the 3D pipeline fast in the browser).

Model licenses

The code in this repository is MIT. Model weights keep their own licenses, which travel with the files on the models host and are listed in each model's doc. Notably, FLUX 3 Image Klein weights are under the FLUX Kommunity License (non-commercial), FLUX.2 klein is Apache-2.0, TRELLIS.2, SkinTokens and UniRig are MIT, and the bundled Kimodo motion clips carry the terms in SpawnDev.ILGPU.ML.Demo/wwwroot/motions/NOTICE.md. Check them before commercial use.

Contributing

Issues and pull requests are welcome. A bug in this library, in SpawnDev.ILGPU or in SpawnDev.SpawnJS is fixed in that library, not worked around in a consumer, so a good bug report is a failing test or a minimal repro on a named backend. Run the tests for the lanes you touched before you open a pull request.

Support the project

SpawnDev is built by one developer and a small team of AI agents. If it is useful to you, sponsoring on GitHub directly funds more kernels, more verified models and more backends. Stars, issues and sharing what you build with it help just as much.

Credits

• ILGPU by Marcel Koester and contributors, the compiler that makes C# GPU kernels possible.
• SpawnDev.ILGPU, which adds the WebGPU, WebGL and Wasm backends and the Velocity CPU backend.
• SpawnDev.SpawnJS, typed C# access to the browser's APIs.
• The authors and licensors of the models ported here: Black Forest Labs (FLUX), Microsoft (TRELLIS.2), VAST AI Research (SkinTokens, UniRig), NVIDIA (Kimodo), OpenAI (Whisper), Stability AI (SD-Turbo), Alibaba (Qwen), and the many others named in the docs.

Created by Todd Tanner (@LostBeard) together with a crew of Claude AI agents: Riker, Data, Tuvok, Geordi and Seven. Live long and prosper. 🖖

License

MIT. SpawnDev.ILGPU.ML includes code derived from ILGPU under the University of Illinois Open Source License; see LICENSE.txt for the full text.