EvaluatedApplications.EvalApp.Neural 2.0.3

EvalApp.Neural

An always-learning neural concurrency tuner for EvalApp. Instead of the built-in heuristic, a holographic policy model (HoloFormer, d48/L2/K2/shifts8) tunes your resource-gate concurrency online, reacts in real time as conditions shift, and keeps learning from its own experience.

Version: 1.1.0
NuGet: dotnet add package EvaluatedApplications.EvalApp.Neural

Quick Start

Enable the neural tuner on your app with one line in the builder chain:

using EvalApp;
using EvalApp.Neural;

Eval.App("orders")
    .WithContext(ctx)
    .WithResource(ResourceKind.Cpu, Tunable.Between(1, 32, 4))
    .WithResource(ResourceKind.Of("db"), Tunable.Between(1, 24, 4))
    .WithNeuralTuning()               // <- one line: neural tuner replaces the heuristic
    .DefineDomain("orders", state)
        .DefineTask<Batch>("process")
            .ForEach(/* ... */)
            .Run(out var pipe)
        .Build();

The tuner loads a warm-start model and keeps learning online. It never freezes — it adapts as conditions shift.

Table of Contents

• Core Features
• Installation Methods
• Model Creation & Persistence
• Invariants & Gotchas
• When to Use It

Core Features

Builder Integration

Purpose: Plug the neural tuner into your EvalApp pipeline in place of the built-in heuristic.

using EvalApp;
using EvalApp.Neural;

var app = Eval.App("myservice")
    .WithContext(ctx)
    .WithResource(ResourceKind.Cpu, Tunable.Between(1, 16, 4))
    .WithNeuralTuning()  // Install neural tuner
    .DefineDomain("orders", state)
    .DefineTask<Task>("work")
        .ForEach(/* ... */)
        .Run(out var pipe)
    .Build();

// Expected behavior:
// - Tuner starts with embedded warm-start model
// - Learns online from each drain's outcome
// - Re-tunes gates in real time as load/conditions change

Invariants:

• Call .WithNeuralTuning() AFTER .WithResource() but BEFORE .DefineDomain().
• Requires the standard Eval.App() builder (throws InvalidOperationException otherwise).
• Tuning must be enabled (via .WithTuning() or global default) — the neural tuner plugs into the tuning slot.

Installation Methods

The package provides four ways to install the neural tuner, suited to different use cases:

1. Per-App Installation (.WithNeuralTuning())

Purpose: Enable the neural tuner on a single app's pipeline.

using EvalApp;
using EvalApp.Neural;

var app = Eval.App("myservice")
    .WithContext(ctx)
    .WithResource(ResourceKind.Cpu, Tunable.Between(1, 32))
    .WithNeuralTuning()  // This app learns and tunes its own gates
    .DefineDomain("work", state)
    .DefineTask<Batch>("process")
        .ForEach(/* ... */)
        .Run(out var pipe)
    .Build();

// Expected: Each app instance trains its own private model

Notes:

• Each call to .WithNeuralTuning() creates a separate model instance.
• Models do not persist across process restarts unless you manage it manually.
• Suitable for single-app scenarios or isolated testing.

2. Process-Wide Installation (.UseGlobally())

Purpose: Share one trained model across ALL apps in the process that have tuning enabled.

using EvalApp;
using EvalApp.Neural;

// Train one model across many real pipelines
var model = NeuralTuning.NewModel(seed: 1);

using (NeuralTuning.UseGlobally(model, eps: 0.6, lr: 0.03, learn: true))
{
    // Every app that calls .WithTuning() without an explicit tuner now uses the shared model
    var app1 = Eval.App("service-a")
        .WithResource(ResourceKind.Cpu, Tunable.Between(1, 32))
        .WithTuning()  // Implicitly uses the neural model installed globally
        .DefineDomain("work-a", state)
        // ...
        .Build();
    
    var app2 = Eval.App("service-b")
        .WithResource(ResourceKind.Db, Tunable.Between(1, 24))
        .WithTuning()  // Also uses the same shared model
        .DefineDomain("work-b", state)
        // ...
        .Build();

    // Run both apps; they train the same model from their combined experience
    // app1.Run();
    // app2.Run();
}
// Using block exits: prior tuner factory is restored

// Expected: One model, trained on experience from both apps

Parameters:

• sharedModel : HoloFormer — The model instance to use across all apps.
• eps : double = 0.6 — Exploration rate (0-1). Start at 0.6, decreases adaptively as signal quality improves.
• lr : double = 0.03 — Learning rate for gradient updates. Typical range 0.01–0.1.
• learn : bool = true — If false, model weights freeze after init (rarely used; frozen models transfer weakly).

Returns: IDisposable — Disposing restores the prior tuner factory (normally the heuristic).

Invariants:

• NOT thread-safe: drive workloads sequentially; do not run two apps concurrently under one using block.
• Exploration rate (eps) adapts automatically based on signal quality; do not change it mid-run.

3. Persistent Installation (.UseGloballyPersisted())

Purpose: Dogfood-train the neural tuner across real workloads, saving learned knowledge to disk for reuse.

using EvalApp;
using EvalApp.Neural;

var modelPath = "/local/models/tuner-state.bin";

using (NeuralTuning.UseGloballyPersisted(modelPath, eps: 0.6, lr: 0.03))
{
    // Load model from disk if it exists, else use warm-start, else cold
    // Every app that calls .WithTuning() implicitly uses and trains this one model
    
    var app = Eval.App("orders")
        .WithResource(ResourceKind.Cpu, Tunable.Between(1, 32))
        .WithResource(ResourceKind.Of("db"), Tunable.Between(1, 24))
        .WithTuning()  // Uses the persistent model
        .DefineDomain("orders", state)
        .DefineTask<Batch>("process")
            .ForEach(/* ... */)
            .Run(out var pipe)
        .Build();

    // app.Run();  // Trains the model
}
// Dispose: saves the trained model back to modelPath

// Next process run with the same modelPath loads the previously trained model

Parameters:

• path : string — File path where the model will be loaded from (if present) and saved to (on dispose).
• eps : double = 0.6 — Exploration rate (same as .UseGlobally()).
• lr : double = 0.03 — Learning rate (same as .UseGlobally()).

Fallback behavior:

• If file does NOT exist at path → loads embedded warm-start, else cold model.
• If file is corrupt or mismatched in format → silently falls back to cold (does not throw).
• On dispose → overwrites path with a fresh serialization (overwriting any stale file).

Returns: IDisposable — Saves the model on dispose, restores prior tuner factory.

Use case: Set up a shared path (e.g., in a git-ignored directory) and point every local runner at it:

export EVALAPP_NEURAL_PATH=/local/models/tuner.bin
dotnet run -- scenario-1
dotnet run -- scenario-2
dotnet run -- scenario-3

All three runs train the same model, accumulating experience across workloads.

4. Auto-Install from Environment Variable

Purpose: Automatically install the persistent tuner on ANY process that loads this DLL, if the environment variable is set.

// Bash/PowerShell: set once before running any app
export EVALAPP_NEURAL_MODEL="/home/user/models/tuner-state.bin"

# Now run any app that uses EvalApp. The neural tuner auto-installs on process load,
# trains against that shared model path, and saves on exit.
dotnet run -- myapp
dotnet run -- myservice

Mechanism: A [ModuleInitializer] fires when EvalApp.Neural.dll loads. If EVALAPP_NEURAL_MODEL environment variable is set, it calls UseGloballyPersisted(path) and registers a ProcessExit handler to dispose.

Invariants:

• If the env var is NOT set (default) → no-op; the built-in heuristic remains the default.
• Every process that loads the DLL while the var is set will auto-install — including test runs, unrelated tools, everything.
• Unset the env var when you do NOT want auto-install.

Model Creation & Persistence

Create a Fresh Model

Purpose: Create a new, untrained model for controlled experiments or offline analysis.

using EvalApp.Neural;
using PrismFormer;

// Fresh model, sized for the tuner's fixed vocabulary (d48/L2/K2/shifts8)
HoloFormer model1 = NeuralTuning.NewModel(seed: 1);
HoloFormer model2 = NeuralTuning.NewModel(seed: 2);  // Different seed = different random init

// Expected: model1 and model2 have the same shape but different initialized weights
// Both load the embedded warm-start if one is shipped, else start cold

Parameters:

• seed : int = 1 — Random seed for weight initialization. Ignored if warm-start is embedded (warm-start weights are deterministic).

Returns: HoloFormer — A model of fixed shape (d48/L2/K2/shifts8) with the tuner's vocabulary (16 concurrency buckets + 8 trajectory length + saturation + contention tokens).

Note: This is NOT guaranteed to be cold. It loads via the same warm-start-or-cold path as production:

• If embedded warmstart.bin exists and deserializes → returns that (warm).
• Else → returns a cold model with the given seed.

For controlled frozen-vs-learning comparisons, use the internal NeuralTuner.ColdModel(seed) instead.

Serialize and Deserialize Models

Purpose: Save a trained model to disk and restore it later.

using EvalApp.Neural;
using PrismFormer;

// Train a model
var model = NeuralTuning.NewModel(seed: 1);
using (NeuralTuning.UseGlobally(model, learn: true))
{
    // app.Run();  // Model learns
}

// Serialize to bytes
byte[] modelBytes = model.Serialize();
File.WriteAllBytes("/local/trained-tuner.bin", modelBytes);

// Later: deserialize and restore
byte[] saved = File.ReadAllBytes("/local/trained-tuner.bin");
HoloFormer restored = HoloFormer.Deserialize(saved);

// Use the restored model in production
using (NeuralTuning.UseGlobally(restored, learn: true))
{
    // app.Run();  // Continues learning from the saved state
}

// Expected: Restored model has identical weights to the saved model

Invariants:

• model.Serialize() → byte[] (binary encoding of weights, shape, and metadata).
• HoloFormer.Deserialize(bytes) → HoloFormer (restores exact model).
• Round-trip is bit-for-bit identical; safe for production hand-offs.

Format versioning: HoloFormer.Deserialize handles backward compatibility with older layouts automatically (see AlgFormer/CLAUDE.md).

Invariants & Gotchas

Shape and Vocabulary Are Fixed

The tuner's model shape is immutable and sized for EvalApp's concurrency vocabulary:

• Model shape: d48 (embedding dimension) / L2 (2 layers) / K2 (2 weight-tied iterations per layer) / shifts=8 (relation-banks).
• Concurrency buckets: B=16 (log-spaced, covering CPU-bound-low to I/O-bound-far-above-thread-count).
• Trajectory length: Hist=8 (last 8 (bucket, outcome) tokens per gate).
• Total vocabulary: 11 + 16 = 27 tokens (BOS, buckets, outcomes, saturation, contention states).

Impact:

• Changing B, Hist, or shape invalidates all saved models; they will not deserialize.
• If you need to adjust vocabulary, bump the package version and regenerate warmstart.bin via the test harness.
• Do NOT manually edit saved model files; they are binary.

The Neural Tuner Always Learns (Never Frozen)

Production paths (.WithNeuralTuning(), .UseGlobally(), .UseGloballyPersisted()) always pass learn: true implicitly.

using EvalApp;
using EvalApp.Neural;

// The tuner ALWAYS learns in production
Eval.App("myservice")
    .WithNeuralTuning()  // learn=true by default, no way to freeze
    .DefineDomain("work", state)
    // ...
    .Build();

Why: A frozen prior transfers poorly (~49% success rate in tests). An always-learning one reaches ~95% on unseen custom gates. Production should never freeze.

If you need a frozen model: Use the internal NeuralTuner.ColdModel() directly in tests (see EvalApp.Tests for examples).

Online Learning Has a Startup Ramp

When you start a new layer (L=2 models), the new layer begins with alpha=0 and ramps to 1 over ~50 training steps.

// First 5-10 real decisions: new L2 layer ramps in, preserving the model's learned function
// After ~50 steps: layer is fully active and contributing to tuning decisions

Mechanism: Each layer has alpha[l] which starts at 0 and linearly increases to 1.0 over AlphaRampSteps=50 training steps. This prevents abrupt weight changes when adding capacity.

Invariant: Alpha is NOT serialized. When you load a saved model and resume, the ramp resets (re-applies ramping if the model was upgraded). This is intentional: clients expect warm-start to continue learning, not replay a ramp.

Time-Paced Sampling Decouples Decision Rate from Drain Rate

Production tuners hard-code windowMs: 25 — they decide at most once per 25ms of wall time.

// Between decisions, the tuner:
// - Holds current concurrency (no change)
// - Accumulates reward from all drains in that window
// - Makes ONE denoised decision per window

Impact:

• A firehose of tiny fast drains gets one decision per 25ms, not thousands (saves overhead).
• Each decision averages rewards over the window (denoisification).
• Cost is independent of drain rate.

In tests: Synthetic tests use windowMs: 0 (per-drain decisions) for tight control.

Coordinate Descent for Coupled Gates

When an app has >1 gate, the tuner uses coordinate descent:

// For multiple coupled gates:
// 1. Credit the gate moved in the LAST run (only it changed, so global reward is a clean signal)
// 2. Move ONLY the gate with highest contention (bottleneck detection)
// 3. As it saturates, contention shifts to the next gate, and that one moves next

Why: Moving every gate every run with global reward is noise-fragile. Coordinate descent removes the runaway problem.

For single gates: Independent mode applies — credit and move the single gate each run.

Shift Count Must Be > 1

An internal invariant enforces shifts > 1 in model construction:

// This would throw ArgumentException:
// var badModel = NeuralTuner.BuildModel(dModel: 48, layers: 2, shifts: 1, seed: 1);
// Error: "shift count must be > 1 (no cross-channel routing at S≤1)"

// Why: At shifts=1, every relation-bank in HoloFormer degenerates to a pure diagonal (no learned routing)

Correctness: This is a theoretical requirement, not a tuning parameter. All shipped models use shifts=8.

Warm-Start Value Is Marginal at First Contact

The embedded warm-start beats a cold model by ~1% on first contact (~88% vs ~87%), but its real value is fast online adaptation:

// First decision: warm-start +1% edge
// After 10 decisions: both catch up (warm-start's true value: it adapted faster)
// On a shifting optimum: warm-start re-converges live; frozen cold stays stuck

Note: Don't oversell the warm-start; ship it because it doesn't hurt and enables live re-tuning.

Silent Fallback on Format Mismatch

Loading a stale or corrupt model does NOT throw:

using (NeuralTuning.UseGloballyPersisted("/path/to/old/tuner.bin"))
{
    // If the file is incompatible or corrupt:
    // - Deserialize throws → caught internally → falls back to cold model
    // - No exception bubbles to caller
    // - On dispose: overwrites the file with a valid, deserializable model
}

// Expected: Next process run loads the valid model, not the corrupt file

Belt-and-braces test: NeuralIntegrationTests.UseGloballyPersisted_CorruptFile_FallsBackSilently_AndOverwritesOnDispose pins this end-to-end.

No License Key Required

Nothing in the stack is license-gated. The neural tuner runs at full concurrency and full effect on any tier:

using EvalApp;
using EvalApp.Neural;

// Unlicensed, free-tier, or any other tier: neural tuning works identically
var app = Eval.App("myservice")
    .WithNeuralTuning()  // No license check; works on free tier
    .DefineDomain("work", state)
    // ...
    .Build();

EVALAPP_NEURAL_MODEL Is Process-Wide

The environment variable triggers auto-install for ANY process that loads the DLL:

export EVALAPP_NEURAL_MODEL=/shared/models/tuner.bin
dotnet test --filter FullyQualifiedName~SomeTest  # Tests also auto-install!

Gotcha: Test runs will also auto-install and train against that path. Unset it when running tests that should NOT use the neural tuner, or use a separate path for test runs.

Multiple Gates But Not Using .WithNeuralTuning()

If you have multiple gates and use the heuristic instead of the neural tuner:

var app = Eval.App("myservice")
    .WithResource(ResourceKind.Cpu, Tunable.Between(1, 32))
    .WithResource(ResourceKind.Of("db"), Tunable.Between(1, 24))
    // NO .WithNeuralTuning() → uses heuristic
    .DefineDomain("work", state)
    // ...
    .Build();

The heuristic does NOT use coordinate descent — it adjusts all gates every run, which is noisier on coupled workloads. This is where the neural tuner shines (see below).

When to Use It

Use .WithNeuralTuning() When:

• Multiple interacting gates (CPU, DB, disk, network together) — coordinate descent handles coupling.
• Noisy or drifting optimum — the tuner re-converges live; a frozen heuristic gets stuck.
• Concurrency is NOT simply core count — the tuner learns the workload's actual shape.

Stick with the Heuristic When:

• Single obvious CPU fan-out — the heuristic already gets it right (tuner will match, not hurt it).
• Extremely tight latency budget — the neural tuner adds ~0.4ms overhead per drain (negligible for most workloads, but measurable in microsecond-scale tasks).

Real-World Results

• Clean/lean pipelines (obvious optimum): Parity with the heuristic (no edge, no regression).
• Noisy/coupled real workloads (Sisyphus 3-gate DB+CPU+disk scenario): ~23–32% higher throughput.
• Non-stationary (shifting) optimum: Only the neural tuner re-converges; frozen modes collapse.
• First-contact knowledge transfer: The warm-start edges out cold; the real value is online re-adaptation.

Dependencies

• EvaluatedApplications.EvalApp (via ITuner, TuningState, tuner factory hook)
• EvaluatedApplications.AlgFormer (supplies HoloFormer and related model utilities)

License

Proprietary — this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key — every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.