HartsyInference.Diffusion 1.0.0-alpha.9

HartsyInference

A pure C#/.NET AI inference engine for image generation, speech, vision, video, and interactive world models. No Python, no native runtime DLLs.

HartsyInference loads .safetensors and .gguf checkpoints directly and runs them on NVIDIA CUDA, cross-vendor Vulkan, or CPU SIMD, entirely in managed .NET. GPU kernels are PTX/SPIR-V shipped with the package and JIT-compiled at runtime; there are no C++ wrappers, no bundled native inference library, and no external Python process to manage. Just NuGet packages.

It is the non-LLM companion to dotLLM: together they form a complete AI stack in pure .NET.

⚠️ Alpha software

This is 1.0.0-alpha, an early, fast-moving preview. Use it to experiment, not in production.

• APIs will change without notice between alpha releases. Pin an exact version.
• Model coverage is broad but maturity varies. Many architectures are implemented and load/run end-to-end but are still being validated numerically against their reference implementations. Treat output quality per-model as "verify before you rely on it."
• No support guarantees, no semver stability until 1.0.0.
• The OpenAI-compatible server and CLI are not published as packages in this alpha; they live in the source repository. Publishing them is on the roadmap.

Found a bug or a mismatch against a reference? Please open an issue.

Install

One package pulls in the whole stack (all backends + every modality):

dotnet add package HartsyInference --prerelease

Or reference only the pieces you need (see Packages):

dotnet add package HartsyInference.Audio --prerelease
dotnet add package HartsyInference.Cpu   --prerelease

Requires .NET 8 or .NET 10.

Quick start: speech-to-text

The Whisper pipeline downloads a checkpoint from HuggingFace on first use and runs on whichever backend you pass:

using HartsyInference.Audio.Pipelines;
using HartsyInference.Core.Backends;
using HartsyInference.Cpu;          // or HartsyInference.Cuda / HartsyInference.Vulkan

using WhisperPipeline pipeline = await WhisperPipeline.LoadAsync("openai/whisper-base");
using IBackend backend = new CpuBackend();

WhisperOptions options = new() { Language = "en", WithTimestamps = false };
string text = pipeline.TranscribeWav(backend, "audio.wav", options);

Console.WriteLine(text);

Swap new CpuBackend() for new CudaBackend() or new VulkanBackend(); the pipeline is backend-agnostic. Audio pipelines that auto-download (WhisperPipeline, KokoroPipeline) share this LoadAsync convention, while image and video pipelines (StableDiffusion15Pipeline, WanVideoPipeline) are constructed from pre-loaded components. See the samples in the repo for image, video, and TTS walkthroughs.

What it can do

Checkpoints load directly from .safetensors / .gguf, including quantized weights (GGUF, MXFP4/8, NVFP4, block-scaled).

Coverage is wide because the engine shares a common core (tensors, schedulers, VAEs, text encoders, DSP) across architectures. Per-model numerical validation is ongoing; see the alpha note above.

Coming soon

HartsyInference is moving fast, and the roadmap is broad. On deck:

Track progress and releases on the GitHub repo.

Design pillars

Packages

Requirements

• .NET 8 or .NET 10 SDK

CUDA backend (NVIDIA, fastest)

• CUDA 12.x runtime
• NVIDIA GPU, compute capability 8.0+ (RTX 30xx/40xx, A100, H100)

Vulkan backend (NVIDIA / AMD / Intel, cross-vendor)

• Vulkan 1.3+ runtime (ships with the GPU vendor driver)
• GPU with FP16 compute (shaderFloat16), most discrete GPUs from 2019+

CPU backend

• Any x86-64 (AVX2+) or ARM64 (NEON) machine, no GPU required

Links

• Source & docs: https://github.com/kalebbroo/HartsyInference
• Issues: https://github.com/kalebbroo/HartsyInference/issues
• LLM companion: dotLLM

License

MIT © 2026 kalebbroo