HartsyInference.Core 2.0.0-alpha.389

HartsyInference

C#/.NET inference libraries for text, images, video, speech, music, vision, 3D, and world models. Load safetensors, GGUF, and supported PyTorch checkpoints with CUDA, Vulkan, or CPU backends. Inference does not require a Python runtime; reference-generation tooling may use Python.

Alpha: pin an exact package version. Model, checkpoint-format, feature, and backend coverage differ, and implementation does not imply real-weight verification — model status records verified paths and known gaps.

Get started

For a UI, use the SwarmUI backend extension. For a .NET application, install the meta-package (or select individual packages):

dotnet add package HartsyInference --prerelease

InferenceEngine owns loading, caches, placement, and generation:

using HartsyInference.Engine;
using HartsyInference.Engine.Dispatch;
using HartsyInference.Engine.Requests;

using InferenceEngine engine = new("cuda");
ImageResult image = await engine.Images.GenerateAsync(
    new ModelSpec { Requested = "sdxl", Modality = Modality.Image, LocalPath = "sdxl.safetensors" },
    new ImageRequest { Prompt = "a castle on a mountain at sunset", Steps = 25 });
// image.Rgb contains image.Width * image.Height * 3 RGB24 bytes.

Services include Images, Text, Video, Speech, Transcribe, Music, Vision, Mesh, World, Restore, VoiceConversion, Fx, and Embeddings. Inspect the service interfaces for their request/result contracts.

CLI and HTTP

dotnet run -c Release -f net10.0 --project src/HartsyInference.Cli -- \
  image "a castle on a mountain at sunset" --model-path sdxl.safetensors -b cuda
dotnet run -c Release -f net10.0 --project src/HartsyInference.API

The CLI provides catalog/download tools and modality commands; use --help for current options. The HTTP application wraps Engine with native routes and OpenAI-compatible routes. See endpoint implementations, server options, and deployment. The HTTP text path does not currently expose the LLM package's continuous-batching scheduler.

Image, video and music families take their defaults from a recipe: a request field left unset takes the recipe's value where the recipe declares one, and a recipe value of null leaves that choice to the pipeline or loader. Image and video results carry provenance (ParameterProvenance) for the settings planning resolves: each names its source, such as the recipe, the caller, or a normalization of the caller's value. GET /v1/native/recipes lists the image and video recipes, and POST /v1/native/recipes/describe answers for one model without loading weights; text, speech and transcription defaults still live in their loaders. See the recipe contract.

Video supports preflight at POST /v1/native/video/plan and SSE generation at POST /v1/native/video/stream. Plans govern compatibility, artifacts, defaults, and release gates before loading weights. H3 expansion features remain subject to their individual release gates.

Configuration

Settings live in one file, ~/.config/hartsyinference/settings.json, and are read and written through hartsy settings list | get | set. The engine reads no environment variables; see settings for the knobs and their scopes.

Requirements and backends

Libraries target .NET 8 and .NET 10; building/testing uses the .NET 10 SDK. CUDA requires an NVIDIA GPU supported by the shipped PTX (baseline sm_80, plus per-architecture variants where a kernel ships one), a compatible driver, and required CUDA userspace libraries. Turing cards (sm_75, such as the RTX 2060 SUPER) cannot run the shipped baseline: several kernels use cp.async or sm_80 bf16 instructions. The CUDA backend still constructs there with only the kernels built for that architecture, and auto selects Vulkan; use an Ampere-or-newer GPU for CUDA runs. Optional cuDNN paths have additional library requirements. Vulkan requires a compatible runtime and the features queried by the backend; AMD/Intel hardware validation remains open. CPU has no GPU dependency.

C# model code routes math through IBackend. CUDA uses disk-loaded PTX, Vulkan uses SPIR-V, and CPU uses SIMD with scalar fallbacks. CUDA kernels include compiled CUDA sources and legacy handwritten PTX. Vendor-library P/Invoke is used in GPU backends; this is not a ban on native driver/math libraries.

Community benchmarks

Run the same frozen workloads on your GPU and submit the complete evidence by PR. The benchmark guide covers setup, methodology and review; the explorer compares accepted runs on matching workloads. No historical scores are presented as verified community results. The image above is the checked-in dataset snapshot; it is replaced by the live explorer once Pages is activated for this repository.

Documentation

• Documentation map, contributing, agent instructions.
• Model support and numerical evidence.
• Open work, troubleshooting.
• Historical performance scoreboards — hardware, settings, dates, and external baselines.
• Multi-GPU configuration and settings.

Code and packages are MIT licensed. Model weights retain their publishers' licenses; consult the checkpoint's license, including the MiniMax-H3 license, before use or redistribution.