Nextended.Aspire.Hosting.LocalAI 10.1.14

Nextended.Aspire.Hosting.LocalAI

Self-hosted, OpenAI-compatible multimodal AI for .NET Aspire — the self-hosted counterpart to AddOllama for everything beyond text. Runs LocalAI as a single container resource that serves image generation, text-to-speech, speech-to-text, video generation, sound/music generation, chat and embeddings on one endpoint, with NVIDIA/AMD GPU support, gallery model management and an optional Open WebUI.

var builder = DistributedApplication.CreateBuilder(args);

var ai = builder.AddLocalAI("localai")                      // NVIDIA GPU + LocalAI 4.x image by default
    .WithDataVolume()                                       // persist model downloads
    .AddModel(KnownImageModel.Flux1Schnell)                 // image    -> /v1/images/generations
    .AddTextModel(KnownTextModel.Qwen3_8b)                  // chat/LLM -> /v1/chat/completions
    .AddTextToSpeechModel(KnownTextToSpeechModel.Kokoro)    // TTS      -> /v1/audio/speech
    .AddSpeechToTextModel(KnownSpeechToTextModel.WhisperBase) // STT    -> /v1/audio/transcriptions
    .WithOpenWebUI();                                       // dev-time UI (excluded from publish)

builder.AddProject<Projects.Web>("web")
    .WithLocalAI(ai);   // injects AI_API_BASE + IMAGE_MODEL / TEXT_MODEL / TTS_MODEL / STT_MODEL / VIDEO_MODEL / SOUND_MODEL / EMBEDDING_MODEL

Formerly Nextended.Aspire.Hosting.ImageGen (image-only). The container was always a full LocalAI server — this package now exposes the other modalities too. AddLocalAI/WithLocalAI replace AddImageGeneration/WithImageGeneration.

Version / image: the default tag is the standard non-AIO 4.x CUDA image (latest-gpu-nvidia-cuda-12) — that's what ships video generation and the ace-step sound backend. The all-in-one (-aio-) tags are frozen at v3.12.1 upstream (no video/sound); only pick one if you want the bundled default model set. Backends download on demand either way.

What the consumer gets

WithLocalAI(...) injects environment variables into the consuming resource. One base URL serves every modality; the default model per modality is injected only when you added one of that kind.

Back-compat: IMAGE_PROVIDER, IMAGE_API_BASE (and IMAGE_API_KEY) are still injected too, so existing image-only clients keep working after the rename with no code change.

Endpoints

All except /video are OpenAI-compatible, so any OpenAI client works unchanged. /video is LocalAI's own endpoint (there is no OpenAI video standard).

Chat / LLM, vision & embeddings

LocalAI is also a full LLM host — the AddOllama-style role for text, with 1000+ chat models in the gallery. Add them like any other modality; the string overload takes ANY gallery id:

ai.AddTextModel(KnownTextModel.Qwen3_8b)                    // enum -> gallery name
  .AddTextModel(KnownTextModel.Qwen3Vl8b)                   // vision-capable multimodal
  .AddTextModel("kimi-k2.7-code")                           // any of the 1000+ gallery LLMs by name
  .AddEmbeddingModel(KnownEmbeddingModel.NomicEmbedText);   // -> /v1/embeddings

Curated KnownTextModel picks (any other via string): Qwen3 (qwen3-0.6b…qwen3-32b, qwen3-30b-a3b, qwen3-coder-480b-a35b-instruct), Llama 3.x (meta-llama-3.1-8b-instruct, llama-3.3-70b-instruct), Gemma 3 (gemma-3-4b-it…-27b-it, vision), DeepSeek (deepseek-ai.deepseek-v3.2, deepseek-ocr), glm-4.7-flash, kimi-k2.7-code/kimi-k2.6, nousresearch_hermes-4-14b, vision qwen3-vl-{4,8,30}b, omni qwen3-omni-30b-a3b-instruct. KnownEmbeddingModel: bert-embeddings, nomic-embed-text-v1.5, bge-m3-colbert, granite-embedding-*, embeddinggemma-300m, qwen3-embedding-*.

Consume via the injected TEXT_MODEL/EMBEDDING_MODEL against {AI_API_BASE}/v1/chat/completions resp. /v1/embeddings (standard OpenAI shape). Browse all models in the LocalAI WebUI "Models" tab or at https://localai.io/gallery.html.

Generating from your app

Prefer calling the service server-side: if you set an ApiKey it must never reach the browser, and LocalAI is meant to live on a trusted network, not be exposed publicly.

Images

The endpoint speaks the OpenAI Images API and returns a base64 PNG.

// C# (from the consuming service)
var baseUrl = builder.Configuration["AI_API_BASE"]!;   // injected, e.g. http://localhost:5069
var model   = builder.Configuration["IMAGE_MODEL"]!;
var apiKey  = builder.Configuration["AI_API_KEY"];     // only if you set o.ApiKey

using var http = new HttpClient();
if (!string.IsNullOrEmpty(apiKey))
    http.DefaultRequestHeaders.Authorization = new("Bearer", apiKey);

var res = await http.PostAsJsonAsync($"{baseUrl}/v1/images/generations", new
{
    model, prompt = "a lighthouse at dawn, cinematic", size = "1024x1024", n = 1, response_format = "b64_json",
});
res.EnsureSuccessStatusCode();
using var doc = JsonDocument.Parse(await res.Content.ReadAsStringAsync());
var b64 = doc.RootElement.GetProperty("data")[0].GetProperty("b64_json").GetString();
await File.WriteAllBytesAsync("out.png", Convert.FromBase64String(b64!));

// TypeScript / JavaScript (Node, TanStack Start server fn, Next route handler, …)
// Keep this on the server — never ship AI_API_KEY to the browser.
const base   = process.env.AI_API_BASE!;   // injected by WithLocalAI
const apiKey = process.env.AI_API_KEY;      // optional
const auth   = apiKey ? { Authorization: `Bearer ${apiKey}` } : {};

const res = await fetch(`${base}/v1/images/generations`, {
  method: "POST",
  headers: { "Content-Type": "application/json", ...auth },
  body: JSON.stringify({
    model: process.env.IMAGE_MODEL, prompt: "a lighthouse at dawn", size: "1024x1024", n: 1, response_format: "b64_json",
  }),
});
if (!res.ok) throw new Error(`image backend ${res.status}: ${await res.text()}`);
const json = await res.json();
// OpenAI Images shape; extract defensively — some models/backends return a url instead.
const b64 = json.data?.[0]?.b64_json ?? json.images?.[0]?.b64_json;
const bytes = Buffer.from(b64, "base64");   // -> save, upload to storage, or return as data URI

Audio — text-to-speech & transcription

TTS returns audio bytes (not JSON); STT is a multipart upload returning { text }.

// Text-to-speech (OpenAI-compatible). response_format: wav | mp3 | aac | flac | opus.
const speech = await fetch(`${base}/v1/audio/speech`, {
  method: "POST",
  headers: { "Content-Type": "application/json", ...auth },
  body: JSON.stringify({ model: process.env.TTS_MODEL, input: "Willkommen bei promote.me", response_format: "mp3" }),
});
const audio = Buffer.from(await speech.arrayBuffer());   // -> save as .mp3 / stream to the client

// Speech-to-text (OpenAI-compatible, multipart/form-data).
const form = new FormData();
form.append("model", process.env.STT_MODEL!);
form.append("file", new Blob([audioBytes], { type: "audio/wav" }), "clip.wav");
const stt = await fetch(`${base}/v1/audio/transcriptions`, { method: "POST", headers: auth, body: form });
const { text } = await stt.json();

Video

POST /video is long-running (seconds to minutes, GPU-bound) and returns a URL (or base64) to the generated clip. Treat it as a job, not a request/response you block a UI on.

const res = await fetch(`${base}/video`, {
  method: "POST",
  headers: { "Content-Type": "application/json", ...auth },
  body: JSON.stringify({
    model: process.env.VIDEO_MODEL,
    prompt: "a neon city skyline at night, slow camera pan",
    width: 512, height: 512, num_frames: 16, fps: 8,
    response_format: "url",   // or "b64_json"
  }),
});
const json = await res.json();
const clipUrl = json.data?.[0]?.url;   // fetch / stream the mp4

curl {AI_API_BASE}/video -H "Content-Type: application/json" -d '{
  "model": "vllm-omni-wan2.2-t2v", "prompt": "A cat playing in a garden on a sunny day",
  "width": 512, "height": 512, "num_frames": 16, "fps": 8 }'

Other /video params: negative_prompt, start_image, end_image, input_reference, seconds, size, seed, cfg_scale, step.

WebUI note (LocalAI ≤ 3.12.x): in LocalAI's built-in WebUI the Video tab's model dropdown reverts to "select a model" and won't keep a vllm-omni (or other non-diffusers) video model selected — a known upstream WebUI bug (#8659, fixed by PR #8781, which added vllm-omni to the WebUI's video-usecase detection). It is a UI-only bug: driving POST /video via the API (as shown above, and as this package does) works regardless. Fix ships in LocalAI ≥ 4.0 — upgrade the image tag if you need the WebUI video picker.

Sound / music

POST /v1/sound-generation is LocalAI's ElevenLabs-compatible music/sound-effect endpoint (distinct from TTS on /v1/audio/speech). It takes model_id + text and returns audio bytes (wav/flac/mp3). Like video it is long-running / GPU-bound — treat it as a job. ACE-Step also accepts optional music metadata (lyrics, bpm, keyscale, duration_seconds, …).

const res = await fetch(`${base}/v1/sound-generation`, {
  method: "POST",
  headers: { "Content-Type": "application/json", ...auth },
  body: JSON.stringify({
    model_id: process.env.SOUND_MODEL,          // e.g. "ace-step-turbo"
    text: "an upbeat lofi hip-hop beat, mellow piano, 90 bpm",
    // optional ACE-Step music controls:
    // lyrics: "[Verse 1]\n…", bpm: 90, keyscale: "C major", duration_seconds: 30,
  }),
});
if (!res.ok) throw new Error(`sound backend ${res.status}: ${await res.text()}`);
const audio = Buffer.from(await res.arrayBuffer());   // -> save as .wav / stream to the client

curl {AI_API_BASE}/v1/sound-generation -H "Content-Type: application/json" -d '{
  "model_id": "ace-step-turbo", "text": "A funky Japanese disco track",
  "bpm": 120, "keyscale": "Ab major", "duration_seconds": 30 }' --output music.wav

Calling straight from the browser is possible for the OpenAI-compatible endpoints — but only in a trusted/dev setup: no API key (nothing to leak) and the backend reachable with permissive CORS. For anything user-facing, proxy through your own backend so you keep control of auth, rate limiting and prompt policy. (This server-proxy shape is exactly what the promote.me app uses in media-provider.server.ts — generateSpeech / transcribeAudio / generateVideo / generateSound.)

Using it from n8n

Because the endpoint is OpenAI-compatible, LocalAI is a drop-in backend for n8n's OpenAI, AI Agent and Embeddings OpenAI nodes — point them at {endpoint}/v1 with any key (default sk-local); pair it with AddOllama for the native Ollama nodes. When you host n8n via Nextended.Aspire.Hosting.N8n, wire the URL in and order startup:

var localai = builder.AddLocalAI("localai").AddTextModel(KnownTextModel.Qwen3_8b);

var n8n = builder.AddN8n("n8n")
    .WaitFor(localai)
    .WithEnvironment("OPENAI_API_BASE_URL",
        ReferenceExpression.Create($"{localai.Resource.HttpEndpoint}/v1"))
    .WithEnvironment("OPENAI_API_KEY", "sk-local");

n8n keeps node credentials in its own DB, so you still add the OpenAI credential once in the editor (Base URL http://localai:8080/v1, key sk-local) — or pre-seed it via WithImportCredentials. A full runnable example (LocalAI + Ollama + n8n) is in Tests/TestProjects/AiStack.AppHost.

Options

builder.AddLocalAI("localai", o =>
{
    o.Gpu = LocalAiGpu.Nvidia;               // None | Nvidia | Amd
    o.Image = "localai/localai";
    o.Tag = "latest-aio-gpu-nvidia-cuda-12"; // AIO = batteries included; slim tags also work
    o.HostPort = 5069;                       // fixed host port (optional)
    o.AioProfile = "gpu-8g";                 // force AIO profile (see GPU note below)
    o.ApiKey = "my-secret";                  // require a bearer key
});

Models

Each Add…Model call installs a model from the LocalAI gallery on startup and tags it with a modality, so WithLocalAI knows which default-model env var to inject. The typed enums are a curated convenience — any gallery model works via the string overload, and the gallery is always the source of truth.

• AddModel — text-to-image (KnownImageModel, gallery names, huggingface/OCI URIs, config URLs)
• AddTextToSpeechModel — TTS (KnownTextToSpeechModel or any gallery name)
• AddSpeechToTextModel — STT (KnownSpeechToTextModel or any gallery name)
• AddVideoModel — video (KnownVideoModel or any gallery name)
• AddSoundModel — sound / music generation (KnownSoundModel or any gallery name)
• AddHuggingFaceModel — any diffusers-format HuggingFace image repo (not in the gallery)

Image models (AddModel)

ai.AddModel(KnownImageModel.Flux1Schnell)          // enum -> gallery name
  .AddModel(KnownImageModel.Flux1DevUncensored)    // uncensored FLUX (GGML Q8)
  .AddModel("dreamshaper");                         // implicit string -> ImageModel

Audio models (AddTextToSpeechModel / AddSpeechToTextModel)

Text-to-speech (served on /v1/audio/speech):

Speech-to-text / whisper (served on /v1/audio/transcriptions):

ai.AddTextToSpeechModel(KnownTextToSpeechModel.Kokoro)
  .AddSpeechToTextModel(KnownSpeechToTextModel.WhisperBase)
  .AddTextToSpeechModel("vibevoice-cpp");   // any other gallery TTS model by exact name

Video models (AddVideoModel)

Served on POST /video. Weights are large (many GB) and generation is slow / GPU-bound — combine with WithDataVolume().

The video gallery moves fast. Install any other family by its exact gallery id:

ai.AddVideoModel(KnownVideoModel.Wan22TextToVideo)
  .AddVideoModel(KnownVideoModel.Ltx2)
  .AddVideoModel("some-new-video-id");   // any gallery id — browse https://localai.io/gallery.html

The built-in WebUI's Video tab can't select vllm-omni video models before LocalAI 4.0 — see the WebUI note under Video above. The API works either way.

Sound / music models (AddSoundModel)

Served on the ElevenLabs-compatible POST /v1/sound-generation (model_id + text; ACE-Step also takes lyrics/bpm/keyscale/duration_seconds). Multi-GB and GPU-bound — combine with WithDataVolume().

ai.AddSoundModel(KnownSoundModel.AceStepTurbo)
  .AddSoundModel("acestep-cpp-turbo");   // any gallery sound_generation model by exact name

HuggingFace diffusers image models (AddHuggingFaceModel)

Not in the LocalAI gallery — loaded from a HuggingFace repo by generating a model config that is bind-mounted into the container; LocalAI downloads the weights on startup. Includes curated, verified NSFW-capable SDXL models for adult platforms.

ai.AddHuggingFaceModel(KnownHuggingFaceImageModel.RealVisXL4)      // photorealistic, NSFW-capable
  .AddHuggingFaceModel(KnownHuggingFaceImageModel.NsfwGenV2)       // explicit content, unfiltered
  .AddHuggingFaceModel("mymodel", "SG161222/RealVisXL_V5.0");      // any repo id (string overload)

These run on LocalAI's python diffusers backend (first use downloads the backend + weights; SDXL-class models want ≥8 GB free VRAM). Sampler steps and the fp16 file-variant flag are set automatically per known model — turbo/lightning models use fewer steps, and repos that don't ship *.fp16.safetensors load their default-named weights (the wrong flag fails to load). For custom repos via the string overload, tune steps and pass f16: true only if the repo ships fp16-variant files:

ai.AddHuggingFaceModel(
    name: "my-turbo",
    hfRepo: "some/sdxl-turbo-repo",
    pipelineType: "StableDiffusionXLPipeline",   // default; fits SDXL-class models
    steps: 8,                                     // turbo/lightning want ~6-8
    f16: true);                                   // only if the repo publishes fp16 files

Changing/adding models on an existing stack: LocalAI imports each model config into its /models volume on first start and does not overwrite it later. After changing models (or this package), drop the volume so configs re-import: docker volume rm <name>-models.

Note: LocalAI gallery names (used with AddModel/AddTextToSpeechModel/…) and HuggingFace repos (used with AddHuggingFaceModel) are different namespaces. A gallery name like nsfw-v1 does not exist in the gallery — use AddHuggingFaceModel(KnownHuggingFaceImageModel.NsfwV1).

Browse everything at https://localai.io/gallery.html.

Behavior notes:

• No Add…Model → AIO images load their bundled default set (image-gen, chat, tts, …).
• With any Add…Model → only the models you added are downloaded/loaded (the AIO default set is overridden via the MODELS env var). Mix modalities freely in one service.
• WithDataVolume() persists /models and /backends — both downloads are multi-GB and only happen once.

GPU notes

• NVIDIA: the container gets --gpus all. Requirements: recent NVIDIA driver + Docker Desktop (WSL2) or NVIDIA Container Toolkit on Linux. Verify passthrough with: docker run --rm --gpus all ubuntu nvidia-smi
• Docker Desktop / WSL2 quirk: LocalAI's AIO images detect GPUs via lspci, which does not see the WSL2 GPU even when --gpus all works. This package therefore forces PROFILE=gpu-8g for NVIDIA + AIO automatically (override via AioProfile).
• First start downloads several GB (backends + models) — be patient, then use WithDataVolume() so it only happens once.

UIs

• LocalAI WebUI is built in — open the resource endpoint in the browser. It has real text-to-image, text-to-speech, transcription, video and sound-generation tabs, so it doubles as a quick way to try every modality without writing a line of client code. (Before LocalAI 4.0 the Video tab can't keep a vllm-omni video model selected — a UI-only bug, see the Video section; the POST /video API is unaffected.)
• SD.Next (for serious image work): WithSdNextUi() adds a full SD.Next studio — proper txt2img/img2img UI, model & LoRA management, Civitai/HuggingFace downloads. Runs its own GPU container with its own models (independent of LocalAI). Default image vladmandic/sdnext-cuda:latest, UI on port 7860.
• ACE-Step UI (music studio): WithAceStepUi() adds a local Suno-style music studio — ace-step-ui (song library, lyrics editor, stem separation, audio editor) plus the ACE-Step 1.5 server it requires, run in Gradio mode with --enable-api (the API surface the UI generates through). NOTE: the UI speaks ACE-Step's own Gradio/REST API — not LocalAI's /v1/sound-generation — so like SD.Next it runs its own model container with its own weights ({name}-checkpoints volume, ~18 GB on first start); models added via AddSoundModel are independent of it. Both containers are built from source on first run: the UI ships no image, and the server is built from the pinned ACE-Step tag v0.1.4 — the UI calls the Gradio endpoint with positional args, and its argument list matches exactly that signature (newer releases inserted parameters, shifting positions and breaking generation; GHCR offers no v0.1.4 image). If you change ApiGitRef/ApiTag, pin UiGitRef to a UI revision built against that server version. The generated server Dockerfile also pins torchcodec==0.10.0 (the release matching torch 2.10 on the CUDA-12.8 base) — v0.1.4 has no lockfile, so an unpinned build otherwise pulls a torchcodec built for CUDA 13 and cover/reference-audio generation fails to decode its input (plain text2music is unaffected).
• Open WebUI: WithOpenWebUI() adds a ghcr.io/open-webui/open-webui container. Overloads let you reuse an existing Open WebUI (e.g. the one from the Ollama integration) instead of a second one:

  var ollama = builder.AddOllama("ollama").WithOpenWebUI();  // Ollama's Open WebUI
  var ai = builder.AddLocalAI("localai").AddModel(KnownImageModel.Flux1Schnell)
      .WithOpenWebUI(useExistingIfFound: true);              // reuse it, add image models
  // or pass it explicitly:
  var ui = builder.Resources.OfType<OpenWebUIResource>().FirstOrDefault();
  if (ui is not null) ai.WithOpenWebUI(ui);
  

  Note Open WebUI's image generation is awkward — you chat with a text model and press the image button; selecting an image model as a chat model yields unimplemented (it hits /v1/chat/completions). Prefer the LocalAI WebUI or SD.Next for pure image generation.

var ai = builder.AddLocalAI("localai")
    .WithDataVolume()
    .AddHuggingFaceModel(KnownHuggingFaceImageModel.NsfwV1)
    .WithSdNextUi()      // full image studio on :7860
    .WithAceStepUi()     // Suno-style music studio (own ACE-Step 1.5 server)
    .WithOpenWebUI();    // optional, chat-first

All UIs are dev-time only (ExcludeFromManifest).