Darkspyre.Inference.Tensor.Common 11.19.3-rc.2

Darkspyre.Inference.Tensor.Common

The shared base of the Tensor engines, which run models inside the application's own process: model packs and their verified local store, the host-controlled model lifecycle, execution coordination and the Tensor capability interfaces (ITensorLlmInference, ITensorImageInference, ITensorDecisionInference, ITensorSpeechToTextInference and ITensorEmbeddingInference). Reference an engine package to run a model.

Install

dotnet add package Darkspyre.Inference.Tensor.Common

Register

The package needs no registration. The application builds a TensorModelCatalog from the pack definitions of the engines it ships and one TensorModelPackManager over its pack directory. Darkspyre.Inference.Tensor.Registry registers the shipped engines when the application wants them as services.

Example

// definition is a pack definition an engine package publishes.
var packs = new TensorModelPackManager(new TensorModelCatalog([definition]), packDirectory);

// Downloads, checks and stores the pack. Run it only when the person chooses to download the model.
await packs.InstallAsync(definition.Key, progress, cancellationToken);

An engine then follows the same lifecycle: StartAsync, LoadAsync, the capability's inference calls, UnloadAsync and StopAsync. Each call returns once the engine accepts it; EngineStatusChanged reports Loading, Loaded, Unloaded, Stopped or Failed with its error.

Settings

Each engine's CreateSettingsManager returns a TensorSettingsManager for the shared setup editor. Its settings choose the model and where it runs ("Runs on"), show the model's install state, and offer Download, Delete download, Check files, Load now and Check backend prerequisites. Selecting and saving never downloads or loads a model.

Limits

• Running a request never downloads weights or loads a model. The application installs packs and loads the model explicitly.
• The pack directory defaults to LocalApplicationData/Darkspyre Studios/Inference/Tensor; the application can supply another. Downloads are staged inside the pack directory, checked by length and SHA-256, and renamed into place.
• One engine instance holds one model at a time. StopAsync is final for an instance: create a new one to run again.
• Unload and stop wait for running requests; cancelling their token cancels those requests. Native work finishes before model memory is released.

The Darkspyre.Inference packages

This package is one library of the Darkspyre.Inference family. The family runs AI inference from .NET applications through typed, provider-neutral contracts.

• Darkspyre.Inference.Common holds the contracts every other package shares: requests, results, capabilities, failures and the provider setup model.
• Darkspyre.Inference.Connect.Common holds the conversation and embedding services. Each Connect provider package adds one provider, and AddLlmConnect() discovers every provider package the application references.
• The Tensor packages load and run models inside the application's own process.
• The UI.Settings packages provide the setup editors a host places in its own window or page.

Every package of the family carries the same version. Reference all of them at the same version: each package depends on the exact version of the family packages it uses.

The packages are licensed under the MIT license. LICENSE.md in each package holds the license text.