Darkspyre.Inference.Tensor.Embedding 11.19.3-rc.2

Darkspyre.Inference.Tensor.Embedding

Text embeddings from GGUF sentence encoders inside the application's process, through TensorSharp. The engine implements Common's IEmbeddingInference and returns unit-length vectors.

Install

dotnet add package Darkspyre.Inference.Tensor.Embedding

Register

Construct TensorEmbeddingInference over a TensorModelPackManager, or call AddTensorEngines() from Darkspyre.Inference.Tensor.Registry, which registers EmbeddingEngineProvider. Models arrive by import: the package has no built-in pack.

Example

await using var embeddings = new TensorEmbeddingInference(packs);
await embeddings.StartAsync();
await embeddings.LoadAsync(modelKey); // Loaded or Failed arrives through EngineStatusChanged.

EmbeddingResult result = await embeddings.EmbedAsync(new EmbeddingRequest
{
    ModelId = modelKey.ModelId,
    Kind = EmbeddingInputKind.Query,
    Inputs = [new TextEmbeddingInput("Where did the meeting move?")]
});

The engine formats queries itself; documents are embedded as given. A query request runs before waiting document requests.

Settings

EmbeddingSetup.CreateSettingsManager opens the engine's settings: the model with its install state, "Runs on", the fields that import a model, and the download, check, load and backend-check actions.

Limits

• A model must be a BERT embedding GGUF (mean, CLS or last-token pooling) with 1,024 dimensions and at least an 8,192-token context. A Qwen3-Embedding pack is reported as not supported.
• The engine runs on managed CPU and GGML CPU, and shares the process's GGML runtime through Darkspyre.Inference.Tensor.Ggml.
• Text only: a request with a media input fails with UnsupportedFeature before the model runs.
• An input over the limit fails the request with EmbeddingInputTooLongException, unless the request sets Truncate.
• A smaller requested size (256) keeps the vector's leading values and scales them back to unit length.
• Vectors with different ModelFingerprint values are not comparable.

The Darkspyre.Inference packages

This package is one library of the Darkspyre.Inference family. The family runs AI inference from .NET applications through typed, provider-neutral contracts.

• Darkspyre.Inference.Common holds the contracts every other package shares: requests, results, capabilities, failures and the provider setup model.
• Darkspyre.Inference.Connect.Common holds the conversation and embedding services. Each Connect provider package adds one provider, and AddLlmConnect() discovers every provider package the application references.
• The Tensor packages load and run models inside the application's own process.
• The UI.Settings packages provide the setup editors a host places in its own window or page.

Every package of the family carries the same version. Reference all of them at the same version: each package depends on the exact version of the family packages it uses.

The packages are licensed under the MIT license. LICENSE.md in each package holds the license text.