EvaluatedApplications.AlgFormer.Gpu 1.16.1

AlgFormer.Gpu

Optional ILGPU/CUDA acceleration for the EvaluatedApplications.AlgFormer HoloFormer resonator-attention model (plus a deprecated twin for the older softmax-attention AlgFormer core). Every GPU kernel is gradchecked against the CPU model as the correctness oracle, fp32 on the GPU vs fp64 on the CPU, so results are float-close, not bit-identical. This page is the only documentation shipped with the package (binary-only NuGet, no separate docs site): every public type, method, and option is covered below with a compiling example.

Every code block below was pasted into a throwaway console project that ProjectReferences this package and built against the current public surface. Blocks marked "continuing from" reuse the Quick Start variables.

Table of contents

1. Quick start
2. Gotchas you must know
3. GpuDevice: device detection
4. GpuHoloTrainer: the production entry point
   • Core training
   • Inference
   • Sync & snapshotting
   • Fused on-GPU Adam (opt-in)
   • ADAM & EVE (opt-in)
   • Depth knobs
   • Throughput / diagnostics readouts
   • Relay / distributed-training helpers
5. GpuHoloModel: the low-level engine
   • Construction, params, lifecycle
   • Forward / inference
   • Backward / training
   • Fused on-GPU Adam (model level)
   • ADAM & EVE (model level)
6. Preference (DPO) training
7. Coherence gates, rising memory and atom preference on the GPU
8. Pleroma page transforms
9. GpuOps: standalone kernel verify/bench
10. Deprecated non-holo twin: GpuModel / GpuTrainer
11. Maturity notes
12. License

Quick start

dotnet add package EvaluatedApplications.AlgFormer.Gpu

using PrismFormer;
using PrismFormer.Gpu;

// Device auto-detect: a real CUDA GPU if present, otherwise ILGPU's own CPU accelerator backend running the
// SAME kernels, no branching needed in your code either way.
Console.WriteLine(GpuDevice.Describe);   // "GPU: <name> (<MB> MB)" or "CPU (no CUDA GPU)"

// A small CPU HoloFormer model, the GPU trainer wraps it and the CPU copy stays the source of truth.
var model = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);

using var trainer = new GpuHoloTrainer(model);

// batch is IReadOnlyList<(int[] Ctx, int Target)>
var batch = new (int[] Ctx, int Target)[]
{
    (new[] { 1, 2, 3 }, 4),
    (new[] { 5, 4, 3 }, 2),
};
double loss = trainer.TrainBatch(batch, lr: 5e-2);   // runs forward+backward on the GPU, one CPU Adam step, updates `model` in place

trainer.SyncToGpu();                                  // GPU is one step behind the just-updated CPU model, push it across
float[] logits = trainer.Logits(new[] { 1, 2, 3 });   // GPU inference

Gotchas you must know

• CUDA detection + CPU fallback is automatic and silent. GpuDevice.HasGpu == false on a box with no NVIDIA driver is a valid, expected outcome, not a bug, the exact same kernels run on ILGPU's CPU accelerator, just slower. Don't branch your own code on it; check it only for logging/diagnostics.
• The CPU model is always the source of truth. Everything you serve, save, or bleed/absorb reads the CPU HoloFormer/AlgFormer passed into the trainer's constructor, never the GPU buffers directly.
• Classic (default) path: GPU goes stale after every CPU-side weight change. TrainBatch/TrainBatchAllPos re-sync CPU→GPU automatically each call, but an out-of-band inference call (Logits/StackIterLogits) after a manual CPU edit needs an explicit trainer.SyncToGpu() first.
• Fused-Adam mode (FusedGpuAdam = true) inverts authority. The GPU holds the LIVE params for the duration of the run; the CPU model lags until you call SyncToCpu(). Call SyncToCpu() before reading/saving/serving the CPU model whenever fused mode is on. Do not call SyncToGpu() mid-run, that clobbers the live GPU params with the stale CPU copy; it's only for pushing an out-of-band CPU change (a bleed/absorb) back IN.
• SyncToCpu() syncs PARAMS ONLY. In fused mode the Adam m/v moments live on the GPU and are not touched by SyncToCpu. For an exact resume, also checkpoint SerializeFusedOptimizer()/RestoreFusedOptimizer() , restore params first, then optimizer state.
• Rebuild the trainer after the model's SHAPE changes (layer count, dims, e.g. after a HoloFormer "grow"). A new GpuHoloTrainer/GpuHoloModel is cheap to construct (only params re-upload); the existing one's device buffers are sized to the old shape and cannot be resized in place.
• fp32 on GPU, fp64 CPU oracle. Never expect bit-exact results between the two paths, use a loose tolerance (around 1e-4 to 1e-6) when comparing.
• d (the model's channel dimension) must be even, the resonator attention is complex-pair math (GpuHoloModel throws InvalidOperationException otherwise).
• Thread-safety: GpuDevice.Accelerator is ONE process-wide singleton, shared by every GpuHoloModel/ GpuHoloTrainer in the process. ILGPU documents accelerators as not thread-safe, never call GPU methods on this package's types concurrently from two threads. Build/train/infer serially (interleaved on one thread is fine; a background-thread snapshot read is explicitly NOT supported, see RefreshSnapshot below).
• Nothing in this package is license-gated or tiered. Every capability documented below, including the experimental ones, is available with no key and no "Pro" tier.
• GpuModel/GpuTrainer (the non-holo softmax-attention twin) are [Obsolete]. They still build, run, and gradcheck, but get no further work, use GpuHoloModel/GpuHoloTrainer for anything new.

GpuDevice: device detection

Static, thread-safe, lazy-cached: probes for a CUDA GPU once per process and caches the result.

using PrismFormer.Gpu;

Console.WriteLine(GpuDevice.Describe);
bool hasGpu = GpuDevice.HasGpu;
long memMb = GpuDevice.MemoryMb;
var accelerator = GpuDevice.Accelerator;   // null only if ILGPU/driver is missing entirely

// ... use the package ...

GpuDevice.Shutdown();   // at process exit

GpuHoloTrainer: the production entry point

GpuHoloTrainer wraps a CPU HoloFormer, runs batched forward+backward on the GPU, and keeps the CPU model as the checkpoint/serve source of truth. Construct one per model shape; rebuild it whenever the model's layer count or dimensions change.

public GpuHoloTrainer(HoloFormer cpu, int tokenBudget = 24576, double clip = 10.0)

tokenBudget seeds the adaptive GPU dispatch-size climb (it self-tunes from there); clip is the global gradient- norm clip applied on every Adam step (classic and fused paths alike).

Core training

// continuing from the Quick Start model/trainer/batch above
loss = trainer.TrainBatch(batch, lr: 5e-2);

var seqs = new int[][] { new[] { 1, 2, 3, 4, 5 }, new[] { 2, 3, 4, 5, 6 } };
double lossAllPos = trainer.TrainBatchAllPos(seqs, lr: 5e-2);

double lossCoop = trainer.TrainBatchCoop(batch, Array.Empty<(int[] Ctx, int Target)>(), lr: 5e-2);

Inference

trainer.SyncToGpu();
logits = trainer.Logits(new[] { 1, 2, 3 });
float[] stackLogits = trainer.StackIterLogits(new[] { 1, 2, 3 }, K: 2, alpha: new float[] { 1f, 1f });

Sync & snapshotting

using var snapshot = trainer.NewSnapshot();
trainer.RefreshSnapshot(snapshot);
float[] snapLogits = snapshot.StackIterForward(new[] { new[] { 1, 2, 3 } }, K: 1, alpha: new float[] { 1f, 1f })[0];

trainer.SyncToCpu();   // before saving/serving the CPU model

Fused on-GPU Adam (opt-in)

Default OFF (FusedGpuAdam = false): the classic path reads the full gradient back to the host and takes a CPU double-precision Adam step every batch, production-proven, and the CPU model is continuously authoritative. Setting FusedGpuAdam = true keeps the Adam m/v moments AND the param update resident on the GPU, removing the per-batch full-gradient readback, also production-proven (it's the mode the live HoloEngine trains with), but it inverts weight authority to the GPU for the run (see Gotchas).

var fusedModel = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);
using var fusedTrainer = new GpuHoloTrainer(fusedModel);
fusedTrainer.FusedGpuAdam = true;
fusedTrainer.StackK = 2;
double fusedLoss = fusedTrainer.TrainBatch(batch, lr: 5e-2);

fusedTrainer.SyncToCpu();                                    // REQUIRED before reading/saving fusedModel
byte[] optState = fusedTrainer.SerializeFusedOptimizer();     // checkpoint the GPU-resident Adam moments
fusedTrainer.RestoreFusedOptimizer(optState);                 // resume with momentum intact

long skipped = fusedTrainer.AdamStepsSkipped;

ADAM & EVE (opt-in)

A per-layer directional gradient gate that nudges Adam's step toward (or away from) the model's own decode-codebook usable range, layered on top of the fused Adam step. Live in production: this package ships it default OFF, but PrismStudio's HoloEngine has trained with it enabled (EveEnabled = true via FusedGpuAdam) as its active optimizer path since 2026-09-24, it is not an unused experiment. Requires FusedGpuAdam = true and StackK > 1 (throws NotSupportedException otherwise, since the gate needs the per-layer K path).

var eveModel = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);
using var eveTrainer = new GpuHoloTrainer(eveModel);
eveTrainer.FusedGpuAdam = true;
eveTrainer.StackK = 2;          // required: ADAM & EVE gates per-layer output
eveTrainer.EveEnabled = true;
eveTrainer.EveBandCap = true;   // or leave false for the softer EveNudge/EveDistanceScaled/EveInBoost gate
double eveLoss = eveTrainer.TrainBatch(batch, lr: 5e-2);

Depth knobs

These select which "depth" strategy TrainBatch/TrainBatchAllPos/Logits/StackIterLogits use underneath. IterCount and StackK are mutually exclusive, pick one.

trainer.StackK = 2;
trainer.IterAlpha = new float[] { 1f, 1f };   // length must equal the model's layer count

Throughput / diagnostics readouts

Console.WriteLine($"budget={trainer.TokenBudget} cpuShare={trainer.CpuShare} gpuWins={trainer.GpuWins} cpuWins={trainer.CpuWins}");

Relay / distributed-training helpers

var (shardGrads, shardLossSum, shardCount) = trainer.GpuGradsAllPos(new List<int[]> { new[] { 1, 2, 3, 4, 5 } });
byte[] wire = GpuHoloTrainer.GradBytes(shardGrads);

GpuHoloModel: the low-level engine

The lower-level forward/backward engine GpuHoloTrainer wraps. Use it directly only if you need gradients/logits without the trainer's batching, length-bucketing, or CPU-coop logic (e.g. a custom relay host, a research harness).

Construction, params, lifecycle

public GpuHoloModel(byte[] serialized)   // from HoloFormer.Serialize()

Throws InvalidOperationException if no accelerator is available at all, or if d is odd. Throws NotSupportedException if the serialized model uses an architecture flag this GPU path doesn't implement (bindFfn:true, normalize:false, unitary:true, or any unrecognized flag bit), this fails loudly at construction rather than silently computing the wrong math.

using var gpuModel = new GpuHoloModel(model.Serialize());
int maxCtx = gpuModel.MaxContext;
gpuModel.UpdateParams(model.Serialize());
gpuModel.UpdateTrainable(model.SerializeTrainable());
// ... gpuModel.Dispose() when done, or `using`

Forward / inference

float[][] fwd = gpuModel.Forward(new[] { new[] { 1, 2, 3 } });
float[][] stackFwd = gpuModel.StackIterForward(new[] { new[] { 1, 2, 3 } }, K: 2, alpha: new float[] { 1f, 1f });

Backward / training

All backward methods return flat gradients in "Pairs order" (Emb, Pos, C, then per-layer 7 banks), the same layout as HoloFormer.SerializeGradient.

var alpha = new float[] { 1f, 1f };   // length == layer count (2 here)
var (grads, bwdLoss) = gpuModel.Backward(new[] { new[] { 1, 2, 3 } }, new[] { 4 });
var (stackGrads, stackLoss) = gpuModel.StackIterBackward(new[] { new[] { 1, 2, 3 } }, new[] { 4 }, K: 2, alpha: alpha);
var (stackGrads2, stackLoss2) = gpuModel.StackIterBackward(new[] { new[] { 1, 2, 3 } }, new[] { 4 }, K: 2, alpha: alpha, out var layerOutNormSum);
var (apGrads, apLossSum, apCount) = gpuModel.StackIterBackwardAllPos(new[] { new[] { 1, 2, 3, 4, 5 } }, scoreP: 2, K: 2, alpha: alpha);

// L=1-only methods need an L=1 model:
var model1 = new HoloFormer(vocab: 16, shifts: 4, layers: 1, maxContext: 6, dModel: 8);
using var gpu1 = new GpuHoloModel(model1.Serialize());
var (iterGrads, iterLoss) = gpu1.IterBackward(new[] { new[] { 1, 2, 3 } }, new[] { 4 }, iters: 2, alpha: 0.5f);
var (thinkGrads, thinkLoss) = gpu1.ThinkBackward(new[] { 1, 2, 3 }, target: 4, nThoughts: 1);

Fused on-GPU Adam (model level)

The mechanism GpuHoloTrainer.FusedGpuAdam drives. Use these directly only if you're building a custom trainer.

gpuModel.BeginAccumulate();
gpuModel.Backward(new[] { new[] { 1, 2, 3 } }, new[] { 4 });   // armed: grads land in the accumulators, not returned
bool stepped = gpuModel.AdamStep(lr: 1e-2, scale: 1.0, clip: 1.0);
long gpuSkipped = gpuModel.SkippedSteps;

float[] flatParams = gpuModel.DownloadParams();
var (m, v, t) = gpuModel.DownloadOptimizerState();
gpuModel.UploadOptimizerState(m, v, t);

ADAM & EVE (model level)

The pure gate math and the fused-Adam-with-gate step, live in production via the trainer-level knobs above (GpuHoloTrainer.EveEnabled etc., see that section for the recommended way to use this); these are the primitives underneath.

var (floor, ceiling) = gpuModel.UsableRange(eps: 1e-3);
double gain = GpuHoloModel.EveGain(before: floor * 2, after: floor * 1.5, floor: floor, ceiling: ceiling);
double bandGain = GpuHoloModel.EveBandCapGain(before: floor * 2, after: floor * 1.5, ceiling: ceiling);

gpuModel.BeginAccumulate();
gpuModel.StackIterBackward(new[] { new[] { 1, 2, 3 } }, new[] { 4 }, K: 2, alpha: alpha, out _);   // seeds the probe
bool eveStepped = gpuModel.AdamStepEve(lr: 1e-2, scale: 1.0, floorNorm: floor, ceilNorm: ceiling, stackK: 2, alpha: alpha);

Preference (DPO) training

GPU log-probabilities and a signed-weight training step, at the same weight-tied depth the trainer trains at (StackK / IterAlpha). The DPO loss math itself stays on the host (HoloFormer.Dpo in the core package); these members supply the scores and apply the weighted gradient.

GpuHoloModel carries the matching low-level SequenceLogProbs(seqs, scoreFrom, K, alpha) and StackIterBackwardAllPosWeighted(seqs, scoreFrom, weights, K, alpha). unitary models are refused, as everywhere on the GPU path.

var prefModel = new HoloFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);
using var prefTrainer = new GpuHoloTrainer(prefModel);
prefTrainer.StackK = 2;

// Each sequence is prompt ++ answer; scoreFrom is where the answer starts.
var pair = new[] { new[] { 1, 2, 3, 4, 5 }, new[] { 1, 2, 3, 6, 7 } };   // chosen, rejected
var scoreFrom = new[] { 3, 3 };
double[] policyLogp = prefTrainer.SequenceLogProbs(pair, scoreFrom);        // log P(answer | prompt), <= 0

using var reference = prefTrainer.NewReference();                           // frozen copy of the current policy
double[] refLogp = prefTrainer.ReferenceLogProbs(reference, pair, scoreFrom);

// +1 raises the chosen answer's log-prob, -1 lowers the rejected one; scale = number of pairs
double[] nll = prefTrainer.TrainBatchAllPosWeighted(pair, scoreFrom, new[] { 1.0, -1.0 }, lr: 1e-3, scale: 1.0);

Coherence gates, rising memory and atom preference on the GPU

Each GPU twin reads its CPU model's switch live, so a switch set on the CPU model applies on the GPU. With a switch off, the kernels and their output are unchanged. All switches default to off.

• In-layer gate. GpuHoloTrainer, NewSnapshot() and NewReference() read HoloFormer.AttentionCoherenceGate and AttentionCoherenceC from the trainer's CPU model. Supported: StackIterForward, StackIterBackward, StackIterBackwardAllPos and its weighted form, SequenceLogProbs, and the fused atom stack. Refused with NotSupportedException: the legacy Forward and Backward, IterBackward, ThinkBackward, Eve (EveEnabled on the trainer, and AdamStepEve), the per-atom path with FusedAtoms = false, and the last-position atom methods.
• Atom memory gate. AtomFieldGpu.MemoryCoherenceGate and MemoryCoherenceC forward to the CPU field. Works with holographic and dense locality. Molecules throw InvalidOperationException, and shards throw NotSupportedException.
• Atom rising memory. AtomFieldGpu.MemoryRiseAmplitude(site) reads the device amplitude. Works with holographic and dense locality. Molecules throw InvalidOperationException, and shards throw NotSupportedException.
• Atom preference lane. AtomFieldGpu.SequenceLogProb, SequenceLogProbs and TrainStepAllPositionsWeighted mirror the CPU lane. They need the fused atom stack.

Verification. The in-layer gate (InLayerGateGpuTests, 17 cases), the atom memory gate (AtomFieldGpuGateTests, 5, and AtomFieldGpuGateLocalityTests, 9), atom rising memory (AtomFieldGpuRiseTests, 14) and the atom preference lane (AtomFieldGpuPreferenceTests, 10) were verified on the ILGPU CPU accelerator only. None of them has run on CUDA. The CUDA runs on this package cover the Pleroma transforms, the Newton pass and the timing of the fused atom step.

Pleroma page transforms

Exact batched transforms for Phasor's Pleroma key band, on the GPU. A "page" is N residues mod a prime q (a CompleteKeys key line); many pages are transformed in one call. The arithmetic is integer, so results are BIT-identical to the CPU, not float-close. q is 2^61 - 1 or any prime below 2^40; N is 2..4096; w must have order exactly N (the constructor throws ArgumentException otherwise). There is no Phasor dependency in this package: for a Pleroma store, recover w as RawCells[1] of Pleroma.FromCounts(shape, e_1).

Not thread safe: call serially. The ILGPU CPU backend is a correctness vehicle, 0.03 to 0.5x scalar Phasor; the speed-up is on CUDA. Measured on an RTX A3000: N = 3150 over 10,000 pages in 88-118 ms, 12-43x scalar Phasor.

const ulong q = (1UL << 61) - 1;
const int n = 110;                       // page length (a divisor of q - 1)

static ulong ModPow(ulong a, ulong e, ulong m)
{
    ulong r = 1;
    for (; e > 0; e >>= 1, a = (ulong)((UInt128)a * a % m)) if ((e & 1) == 1) r = (ulong)((UInt128)r * a % m);
    return r;
}

// w must have order exactly n: take a power of a generator candidate and let the constructor check it.
PleromaPageTransform? xform = null;
for (ulong g = 2; xform is null; g++)
{
    try { xform = new PleromaPageTransform(q, ModPow(g, (q - 1) / n, q), n); }
    catch (ArgumentException) { }
}
using (xform)
{
    var counts = new long[3 * n];                    // 3 pages of signed counts
    counts[5] = 7; counts[n + 9] = -2; counts[2 * n + 1] = 1;
    ulong[] cells = xform.Forward(counts, pages: 3);     // counts -> key-line cells
    long[] back = xform.Inverse(cells, pages: 3);        // cells -> counts, exact
    Console.WriteLine($"{xform.Backend} roundtrip exact: {back.AsSpan().SequenceEqual(counts)}");
}

PleromaNewtonGpu(vocab, radii) is the GPU twin of the Pleroma Cascade's Newton pass: Pass(sample, theta, wantH) returns the summed loss (nats), the gradient and, on request, the exact Hessian of the solved operator over token sequences (Loss(sample, theta) is the loss alone, for a line search). Gradient and Hessian are accumulated in fp64 from fp32 features, float-close to the fp64 CPU pass (relative error gate 1e-5). vocab is 2..110.

using var newton = new PleromaNewtonGpu(vocab: 27, radii: new[] { 0.5, 0.7, 0.85, 0.95 });
double[] theta = new double[newton.P];                    // [Re K..., Im K...], 880 values for 4 radii
var sample = new[] { new[] { 1, 2, 3, 4, 5 } };
var (nllSum, gradient, hessian) = newton.Pass(sample, theta, wantH: true);

GpuOps: standalone kernel verify/bench

Static helpers with no model required, gradcheck/benchmark harnesses for the two kernels every other type here is built on (the relation-bank op, and the fused Adam kernel). Useful for a CI smoke test or a quick sanity check on a new machine.

var (worstRelBank, gpuMs, cpuMs) = GpuOps.VerifyRelBank(BT: 64, d: 16, s: 4);
var (worstBankGrad, worstInputGrad) = GpuOps.VerifyRelBankBackward(BT: 32, d: 16, s: 4);
var (worstParam, worstM, worstV) = GpuOps.VerifyAdam(n: 256, t: 3);
// all three worst-error values should be well under 1e-4 on real CUDA (fp32-vs-fp64 closeness)

Deprecated non-holo twin: GpuModel / GpuTrainer

[Obsolete], still builds, runs, and gradchecks, but gets no further development. This is the GPU twin of the older softmax-attention AlgFormer core (not HoloFormer). Use GpuHoloModel/GpuHoloTrainer for anything new; migrate existing callers when convenient.

#pragma warning disable CS0618   // acknowledge the deprecation warning for this example
var cpuAlg = new AlgFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 8);
using var oldTrainer = new GpuTrainer(cpuAlg);
double oldLoss = oldTrainer.TrainBatch(batch, lr: 5e-2);

using var oldModel = new GpuModel(cpuAlg.Serialize());
float[][] oldFwd = oldModel.Forward(new[] { new[] { 1, 2, 3 } });
var (oldGrads, oldBwdLoss) = oldModel.Backward(new[] { new[] { 1, 2, 3 } }, new[] { 4 });
#pragma warning restore CS0618

Maturity notes

• Classic path (FusedGpuAdam = false, the default): production-proven, gradchecked against the CPU oracle, exercised by a live long-running trainer, with a drop-not-clip non-finite-gradient guard.
• Fused Adam (FusedGpuAdam = true): ALSO production-proven, it's what the live HoloEngine trains with, but it inverts weight authority to the GPU for the run (see Gotchas).
• ADAM & EVE (EveEnabled = true): LIVE IN PRODUCTION, this package ships it default OFF, but PrismStudio's HoloEngine has run it as the active optimizer path (via FusedGpuAdam) since 2026-09-24. Gradchecked against the CPU oracle in isolation (finite-difference, production shape) in addition to the live run.
• Known GPU/CPU architecture gaps fail LOUDLY (NotSupportedException at construction) rather than silently computing the wrong thing: a HoloFormer built with bindFfn:true, normalize:false, or unitary:true is refused by GpuHoloModel/GpuHoloTrainer: those three axes are not implemented on the GPU path.
• Nothing in this package is license-gated or tiered; every capability above is available with no key.

License

Proprietary, this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key, every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.