EvaluatedApplications.AlgFormer 3.7.0

AlgFormer

EvaluatedApplications.AlgFormer — the reusable phasor-transformer engine from PrismFormer: two param-for-param comparable attention cores (softmax and holographic), data-parallel training, incremental serving, checkpointing with in-place growth, a capacity-sizing module, a subword tokenizer, data-source adapters, and MQTT-based distributed/swarm training. Pure double[] + SIMD — no external ML runtime, no native dependency. This page is comprehensive: every public type and member is documented below, because this package ships binary-only (no source, no separate docs site).

dotnet add package EvaluatedApplications.AlgFormer

Table of contents

1. Quick start
2. Core invariants — read before using anything
3. Building a model
4. Training
5. Inference and serving
6. HoloFormer depth: weight-tied iteration and StackIter (+ Preference training (DPO))
7. Inspectors
8. Latent thought steps
9. ADAM & EVE — the per-layer gate
10. Coherence gates, rising memory and atom preference
11. Checkpointing
12. Growth in place
13. HoloShape, capacity-law sizing
14. PrismSpec, the frozen production shape
15. Weight-management utilities
16. Subword tokenizer
17. Data sources
18. Distributed / swarm training
19. Parallelism primitives
20. Self-check oracles (GradCheck family)
21. Exceptions
22. License

Quick start

using PrismFormer;

// A tiny model: 16 tokens, 4 relation shifts, 2 layers, up to 6 tokens of context, 16-wide, nothing frozen.
var model = new AlgFormer(vocab: 16, shifts: 4, layers: 2, maxContext: 6, dModel: 16, frozenPrefix: 0);

// Train on (context tokens, target token) pairs — data-parallel via EvalApp under the hood.
var data = new List<(int[] Ctx, int Target)>
{
    (new[] { 1, 2, 3 }, 4),
    (new[] { 2, 3, 4 }, 5),
};
var trainer = new PrismTrainer(model);
double loss = trainer.TrainEpoch(data, batchSize: 2, lr: 5e-2, shuffleSeed: 1);

// Inference.
int next = model.Predict(new[] { 1, 2, 3 });
double[] logits = model.LogitsFor(new[] { 1, 2, 3 });
int[] generated = model.Generate(new[] { 1, 2, 3 }, maxNewTokens: 5, temperature: 0.8, seed: 1);

Console.WriteLine($"loss={loss:F3} next={next} logits.Length={logits.Length} generated=[{string.Join(",", generated)}]");

HoloFormer is a drop-in for AlgFormer with the same constructor shape, same training/serve/checkpoint surface — swap the type name to switch attention cores.

Core invariants

Read this section before writing any code against the package — these are the mistakes the package itself has been bitten by.

• Shift count S must be > 1. S=1 collapses the relation-bank to a pure per-position diagonal map (y[i]=bank[i]·x[i], no cross-channel term) — attention breaks structurally. Go small on Dim/Layers instead of shrinking Shifts to 1.
• HoloFormer.Iters/IterAlphaServe/IterClean/IterTau are runtime knobs, NOT serialized. A resumed checkpoint always loads with these at their defaults — reset them yourself after Deserialize/Load.
• Arch flags (bindFfn/golden/normalize/unitary/growFromFront/goldenExact) must carry forward on every grow/upgrade. GrowLayers/GrowShifts/HoloUpgrade.* all do this correctly if you go through them; hand-rolling a rebuild and forgetting a flag silently reverts geometry to ctor defaults and scrambles trained banks.
• The embedding's frozen prefix is FrozenPrefix reals per row (0..Dim, can be partial). Step/ Accumulate zero gradient there; a number's exact value/signature is preserved. RestoreFrozen re-snaps it after any operation (e.g. weight averaging) that could introduce floating-point drift.
• Checkpoint formats throw CheckpointFormatException, never a bare EndOfStreamException, on a truncated/incompatible/future-version buffer. Catch this type specifically if you want to fall back to a fresh model on a bad checkpoint.
• Step returns bool — false means the whole batch was dropped because its gradient was non-finite (NaN/∞); the optimizer state and every parameter are left untouched. SkippedSteps on both cores counts this monotonically. Always check the return value (or at least watch SkippedSteps) in a long training run.
• Thread-safety: a single AlgFormer/HoloFormer instance is not safe for concurrent Step/ Accumulate calls, by design — data-parallel training shards accumulate into separate Grads buffers (see Training) and apply with one serialized Step. Serving (Predict/LogitsFor/ Generate/KvCache) is read-only and safe to call concurrently on an unchanging model, but a KvCache itself is per-generation, single-threaded state — never share one cache across threads.
• Production HoloFormer depth is the per-layer StackIter family (StackIterAccumulate/ StackIterLogitsCpu/PrimeIter/StepIter), not the single-layer Iters/IterAccumulate oracle (that one only works when Layers == 1).
• alpha arrays for StackIter are per-layer (double[], length must equal Layers) — pass a uniform array (every entry equal) to reproduce old scalar-alpha behaviour bit-for-bit; a newly grown layer should start its own entry at 0 and ramp to 1 while settled layers stay at 1.0.

Building a model

using PrismFormer;

var holo = new HoloFormer(vocab: 20, shifts: 6, layers: 2, maxContext: 8, dModel: 12, frozenPrefix: 0,
                           normalize: true, unitary: false);
Console.WriteLine($"Vocab={holo.Vocab} Dim={holo.Dim} Context={holo.Context} Shifts={holo.Shifts} " +
                   $"Layers={holo.Layers} FrozenPrefix={holo.FrozenPrefix} ParamCount={holo.ParamCount} " +
                   $"GrowFromFront={holo.GrowFromFront} Normalize={holo.Normalize}");

// opt-in snake init, then measure how straight the layer-to-layer trajectory is
var snake = new HoloFormer(vocab: 20, shifts: 6, layers: 6, maxContext: 8, dModel: 12, frozenPrefix: 0,
                            init: HoloInit.Snake, snakeGrowth: 0.5);
var st = snake.MeasureTrajectory(new[] { 1, 2, 3, 4, 5 });
Console.WriteLine($"stepFlip={st.StepSignFlip:P0} turn={st.MeanTurnDeg:F0} deg normMonotone={st.NormMonotone:P0}");

// opt-in zero-layers init: every layer starts as identity (Ro/Ao zero), inner banks random
var zero = new HoloFormer(vocab: 20, shifts: 6, layers: 6, maxContext: 8, dModel: 12, frozenPrefix: 0,
                           init: HoloInit.ZeroLayers);
Console.WriteLine($"zero-init OutputBankNorm(0)={zero.OutputBankNorm(0)}");

Using the real Phasor codec face (production seeding)

AlgFormer.Mini/HoloFormer.Mini and the production PrismSpec shape seed embeddings from the Phasor VSA codec (package EvaluatedApplications.Phasor, pulled in transitively) so a number token's embedding IS its exact value:

using Phasor;
using PrismFormer;

string[] vocabSymbols = { "0", "1", "2", "foo" };
var mini = AlgFormer.Mini(vocab: vocabSymbols.Length, embedSeed: id => PhasorCodec.Encode(vocabSymbols[id]));
Console.WriteLine($"mini dim={mini.Dim} frozenPrefix={mini.FrozenPrefix}");

Training

using PrismFormer;

var model = new AlgFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 5, dModel: 8, frozenPrefix: 0);
var g = model.NewGrads();
var scratch = model.NewScratch();
double loss = model.Accumulate(new[] { 1, 2, 3 }, 4, g, scratch);
bool applied = model.Step(g, lr: 1e-2);
Console.WriteLine($"loss={loss:F4} applied={applied} skipped={model.SkippedSteps}");

// Or let PrismTrainer shard + merge automatically:
var trainer = new PrismTrainer(model);
var grads = trainer.AccumulateBatch(new[] { (new[] { 1, 2, 3 }, 4) }, out var meanLoss);
Console.WriteLine($"meanLoss={meanLoss:F4}");

Inference and serving

using PrismFormer;

var model = new AlgFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 6, dModel: 8, frozenPrefix: 0);
var cache = model.NewCache();
double[] afterPrompt = model.Prime(cache, new[] { 1, 2, 3 });
double[] afterNext = model.Step(cache, 4);
Console.WriteLine($"cache length={cache.Length} logits lens: {afterPrompt.Length},{afterNext.Length}");

int[] generated = model.Generate(new[] { 1, 2, 3 }, maxNewTokens: 4, temperature: 0.7, seed: 7);
Console.WriteLine($"generated=[{string.Join(",", generated)}]");

HoloFormer depth

Weight-tied iteration gives depth via composition with zero new parameters: the same layer (or layer stack) is applied K times. Two entry families exist:

• Iters/IterAccumulate/IterClean — a single-layer (Layers == 1) oracle. Not the production path.
• StackIter* — per-layer weight-tied K-iteration, the production path. Every layer is applied K times with its own per-layer alpha ramp; layer l's K-iterated output feeds layer l+1.

using PrismFormer;

var holo = new HoloFormer(vocab: 12, shifts: 4, layers: 2, maxContext: 6, dModel: 8, frozenPrefix: 0);
var alpha = new double[holo.Layers];
Array.Fill(alpha, 1.0);   // every layer fully "on" — a settled model; a newly grown layer would start its own entry at 0

var g = holo.NewGrads();
double loss = holo.StackIterAccumulate(new[] { 1, 2, 3 }, 4, g, K: 2, alpha);
holo.Step(g, lr: 1e-2);

double[] served = holo.StackIterLogitsCpu(new[] { 1, 2, 3 }, K: 2, alpha);
Console.WriteLine($"trainLoss={loss:F4} servedLogits.Length={served.Length}");

// O(1)/token incremental serve at the same K:
var cache = holo.NewCache(stackK: 2);
holo.PrimeIter(cache, new[] { 1, 2, 3 }, stackK: 2, alpha);
double[] next = holo.StepIter(cache, 4, stackK: 2, alpha);
Console.WriteLine($"StepIter logits length={next.Length}");

// All-positions training over a window one token longer than Context (trains the final context position):
var seq = new[] { 1, 2, 3, 4, 5, 6, 1 };   // length 7 == Context(6)+1
var g2 = holo.NewGrads();
var (lossSum, count) = holo.StackIterAccumulateAllPos(seq, g2, scoreP: seq.Length, K: 2, alpha);
Console.WriteLine($"allPos meanLoss={(count > 0 ? lossSum / count : 0):F4} count={count}");

Preference training (DPO)

HoloFormer (CPU) has what a Direct Preference Optimization step needs, on the same weight-tied per-layer K-iteration forward the all-positions trainer uses, so train and score depth match.

using PrismFormer;

var policy = new HoloFormer(vocab: 32, shifts: 4, layers: 2, maxContext: 12, dModel: 16, frozenPrefix: 0);
var reference = policy.Clone();                      // frozen
var alpha = new[] { 1.0, 1.0 };
int[] prompt = { 1, 2, 3, 4 };
int[] chosen = prompt.Concat(new[] { 5, 6 }).ToArray();
int[] rejected = prompt.Concat(new[] { 7, 8 }).ToArray();
int from = prompt.Length;
const double beta = 0.1;

var g = policy.NewGrads();
var t = HoloFormer.Dpo(
    policy.SequenceLogProb(chosen, from, 2, alpha), policy.SequenceLogProb(rejected, from, 2, alpha),
    reference.SequenceLogProb(chosen, from, 2, alpha), reference.SequenceLogProb(rejected, from, 2, alpha), beta);
policy.StackIterAccumulateAllPosWeighted(new[] { chosen, rejected }, new[] { from, from },
    new[] { t.WeightChosen, t.WeightRejected }, g, K: 2, alpha);
policy.Step(g, lr: 1e-3, scale: 1);                  // scale = number of pairs in the batch
Console.WriteLine($"margin={t.Margin:F4} loss={t.Loss:F4}");

Inspectors

Read-only interpretability helpers — none affect forward/backward math or trainable state.

using PrismFormer;

var holo = new HoloFormer(vocab: 12, shifts: 4, layers: 2, maxContext: 6, dModel: 8, frozenPrefix: 0);
var alpha = new double[holo.Layers]; Array.Fill(alpha, 1.0);
int[] toks = { 1, 2, 3 };

double[][] boundaries = holo.InspectStackIter(toks, K: 2, alpha);
double[] decodedFinal = holo.DecodeFace(boundaries[^1]);
double[][] attn = holo.InspectAttention(toks, K: 2, alpha);
double[] retrieved = holo.InspectRetrieval(toks, K: 2, alpha);
Console.WriteLine($"boundaries={boundaries.Length} decodedFinal.Length={decodedFinal.Length} " +
                   $"attnRows={attn.Length} retrieved.Length={retrieved.Length}");

// Softmax core: the attention weights themselves, [layer][query][key]; each row sums to 1 over keys 0..query.
var soft = new AlgFormer(vocab: 12, shifts: 4, layers: 2, maxContext: 6, dModel: 8, frozenPrefix: 0);
double[][][] w = soft.AttentionWeights(toks);
Console.WriteLine($"layers={w.Length} row sum={w[0][2].Sum():F3} weight on a later key={w[0][0][2]}");

Latent thought steps

HoloFormer-only, Layers == 1 oracle: after the prompt, fold N self-generated continuous faces into the bundle before decoding — differentiable internal "babble" with no new parameters and no discrete sampling.

using PrismFormer;

var holo1 = new HoloFormer(vocab: 10, shifts: 3, layers: 1, maxContext: 8, dModel: 6, frozenPrefix: 0);
int n = holo1.ThoughtCount(promptLen: 3);
var g = holo1.NewGrads();
double loss = holo1.ThinkAccumulate(new[] { 1, 2, 3 }, 4, g, nThoughts: n);
int predicted = holo1.ThinkPredict(new[] { 1, 2, 3 }, n);
Console.WriteLine($"thoughts={n} loss={loss:F4} predicted={predicted}");

ADAM & EVE

HoloFormer only, opt-in, default OFF, byte-identical when unused. A per-layer Adam gate that nudges a layer's update toward a codebook-derived target norm range instead of letting magnitude drift unchecked. The CPU path documented here (Step with EveOptions, TrainEpochGated) is the reference; production training applies the same gate through the GPU-fused trainer in EvaluatedApplications.AlgFormer.Gpu.

using PrismFormer;

var holo = new HoloFormer(vocab: 12, shifts: 4, layers: 2, maxContext: 6, dModel: 8, frozenPrefix: 0);
var alpha = new double[holo.Layers]; Array.Fill(alpha, 1.0);

var (floor, ceiling) = holo.UsableRange();
double[] before = holo.LayerOutputNorm(new[] { 1, 2, 3 }, K: 2, alpha);

var g = holo.NewGrads();
holo.StackIterAccumulate(new[] { 1, 2, 3 }, 4, g, K: 2, alpha);

var eve = new HoloFormer.EveOptions
{
    Mode = HoloFormer.EveMode.BandCap,
    K = 1.0,
    LayerOutNorm = before,
    LayerInput = null,     // see note below — populate from the same forward in real use
    StackK = 2,
    Alpha = alpha,
};
Console.WriteLine($"floor={floor:F4} ceiling={ceiling:F4} before[0]={before[0]:F4} mode={eve.Mode}");

EveOptions.LayerInput must be the per-layer double[][][] input retained from the same forward that produced LayerOutNorm (an internal pairing the package doesn't expose a public accessor for yet) — in practice you drive Eve/BandCap through TrainEpochGated, which wires this correctly for you; building EveOptions by hand for anything but PerLayerClip needs that pairing done right or Step throws ArgumentException.

Coherence gates, rising memory and atom preference

Every switch in this section defaults to off (false). With a switch off, the model computes the same numbers as a model built without the option.

Atom shared-memory gate. AtomField.MemoryCoherenceGate (bool, default false) with AtomField.MemoryCoherenceC (double, default 1.0, the floor in units of the crosstalk rms). Each channel pair of the shared memory read is soft-thresholded by the crosstalk the other writers leave. Use it when many atoms share one medium and each reader should keep only its own signal. It needs Coupling = AtomCoupling.Memory, works with MemoryLocality in Holographic and Dense modes, and throws InvalidOperationException when MemoryMoleculeSize > 0.

In-layer gate. HoloFormer.AttentionCoherenceGate (bool, default false) with HoloFormer.AttentionCoherenceC (double, default 1.0). Each layer's holographic readout is soft-thresholded per channel pair by its derived crosstalk floor, before the layer's RMS norm. Use it when a bundle holds many items and a query should read out only its own. It works on a single HoloFormer and on each atom of a field. It is honoured by StackIterForward, StackIterAccumulate, StackIterAccumulateAllPos, the DPO methods, LogitsFor, Prime, Step, PrimeIter and StepIter. While it is on, the legacy K=1 Forward and Accumulate, the L=1 IterAccumulate oracle, ThinkAccumulate, Eve's layer replay and the LocalLoss, LocalRepair and Oph hooks throw NotSupportedException.

Both gates are runtime settings and are not serialized. A model trained gated and served ungated is a different function. Set the same values on every copy that is trained, loaded or served.

Rising memory. AtomField.MemoryRise (bool, default false) with MemoryRiseDelay (int, default 160), MemoryRiseGain (double, default 8) and MemoryRisePeg (int, default 0, meaning the atoms' context). Each atom keeps a second memory per site. A note enters at age MemoryRiseDelay with gain 1 and reaches gain MemoryRiseGain at age MemoryRisePeg. The learned read amplitude MemoryRiseAmplitude(site) starts at zero. Use it when notes from well before the current window should still reach the readers. It needs Coupling = AtomCoupling.Memory, works with MemoryLocality, and throws InvalidOperationException when MemoryMoleculeSize > 0 or when MemoryRisePeg does not exceed MemoryRiseDelay. A checkpoint saved with it on uses the MEM7 layout. A checkpoint saved with it off uses MEM6, and MEM6 loads with it off.

Atom preference lane. AtomField.SequenceLogProb(int[][] seqs, int scoreFrom) returns the summed log-probability of the reply under the softmax of the mean of the atom logits. A single int[] is fed to every atom. SequenceLogProbs is the batch form. TrainStepAllPositionsWeighted(seqs, scoreFrom, weights, lr, scale) adds weights[i] * grad(NLL) for each window and takes one Adam step. A positive weight raises that reply's log-probability and a negative weight lowers it. The step always uses the combined loss and returns each window's unweighted NLL. Pair it with HoloFormer.Dpo for preference training on a field. Each atom's window has the same length, 2 to Context + 1 tokens. The lane has no switch.

Checkpointing

Both cores support two independent save formats:

• Stream (Save/Load): writes/fills a 4-int header (v,d,s,layers) + parameters to/from an existing BinaryWriter/BinaryReader. Load requires an existing instance of the same shape; it never reconstructs one. Arch flags are not persisted here.
• Byte array (Serialize/Deserialize): fully self-describing (shape + arch flags + parameters), so Deserialize reconstructs a model of the exact right shape from nothing else. Prefer this unless you have a specific reason to use the stream pair.

using PrismFormer;

var model = new HoloFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 5, dModel: 8, frozenPrefix: 0);
byte[] bytes = model.Serialize();
var reloaded = HoloFormer.Deserialize(bytes);
Console.WriteLine($"roundtrip ok: Vocab={reloaded.Vocab} Layers={reloaded.Layers} ParamCount={reloaded.ParamCount}");

// Gradient transport:
var g = model.NewGrads();
model.Accumulate(new[] { 1, 2, 3 }, 4, g);
byte[] gradWire = model.SerializeGradient(g);
var gradBack = model.DeserializeGradient(gradWire);

// Optimizer state round-trip:
model.Step(g, lr: 1e-2);
byte[] optState = model.SerializeOptimizer();
bool restored = model.LoadOptimizer(optState);
Console.WriteLine($"optimizer restored={restored}");

// Malformed buffer -> typed exception, not a bare EndOfStreamException:
try { HoloFormer.Deserialize(new byte[] { 1, 2, 3 }); }
catch (CheckpointFormatException ex) { Console.WriteLine($"caught: {ex.Message[..Math.Min(40, ex.Message.Length)]}..."); }

Growth in place

Every grow returns a new model and leaves the original untouched; new capacity starts as an identity no-op (output unchanged) and trains from there.

using PrismFormer;

var model = new HoloFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 5, dModel: 8, frozenPrefix: 0);
var grown = model.GrowLayers(extra: 1, zeroOutputOnly: true);
Console.WriteLine($"layers {model.Layers} -> {grown.Layers}, new layer live-test norm={grown.OutputBankNorm(grown.Layers - 1):F6}");

var widened = HoloUpgrade.GrowShifts(grown, newShifts: 5, dim: grown.Dim, frozen: grown.FrozenPrefix);
Console.WriteLine($"shifts {grown.Shifts} -> {widened.Shifts}");

var deeper = HoloUpgrade.GrowContext(widened, newCtx: 12, dim: widened.Dim, frozen: widened.FrozenPrefix);
Console.WriteLine($"context {widened.Context} -> {deeper.Context}, shifts re-derived to {deeper.Shifts}");

HoloShape

Static class — model-shape sizing from measured over-bundling constants. Don't hand-pick Shifts; derive them here.

using PrismFormer;

int ctx = 64, dim = 32;
int shifts = HoloShape.ShiftsFor(ctx, dim);                      // the normal way to pick Shifts
bool collides = HoloShape.ShiftsCollide(shifts, dim, golden: true);
var derived = HoloShape.Derive(dim: 128, stackK: 2);
Console.WriteLine($"ShiftsFor({ctx},{dim})={shifts} collides={collides} " +
                   $"Derive(128) -> Shifts={derived.Shifts} Context={derived.Context} Layers={derived.Layers}");

PrismSpec

Static class — the one frozen production shape every swarm node must match exactly to merge gradients. Diagnostics/experiments can still build off-spec models directly via the constructors; those never join the swarm.

using PrismFormer;

var production = PrismSpec.NewModel();   // the real shape: Dim 256, Context 512, Layers 32, Shifts 64 — large; fine to construct, don't train it in a demo
Console.WriteLine($"Signature={PrismSpec.Signature}");
Console.WriteLine($"MeshCompatible(self)={PrismSpec.MeshCompatible(PrismSpec.Signature)}");
var parsed = PrismSpec.Parse(PrismSpec.Signature);
Console.WriteLine($"parsed.Layers={parsed?.Layers} CanUpgradeFrom(self)={(parsed != null && PrismSpec.CanUpgradeFrom(parsed))}");

Weight-management utilities

using PrismFormer;

var a = new AlgFormer(vocab: 8, shifts: 3, layers: 2, maxContext: 4, dModel: 6, frozenPrefix: 0, seed: 1);
var b = new AlgFormer(vocab: 8, shifts: 3, layers: 3, maxContext: 4, dModel: 6, frozenPrefix: 0, seed: 2);
a.PartialAverage(b, alpha: 0.1);     // only the shared (first 2 of 3) layers move
a.ResetToken(id: 0, seed: 99);
Console.WriteLine($"row0 after reset: [{string.Join(",", a.EmbRow(0).Select(x => x.ToString("F3")))}]");

Tokenizer

A fixed printable-ASCII character vocabulary, optionally extended with a deterministic committed list of subword n-grams (2..16 chars). With an empty subword list, encoding is byte-for-byte char-level.

using PrismFormer;

var chars = new CharVocab();
int[] charIds = chars.Encode("hi");
Console.WriteLine($"CharVocab.Total={CharVocab.Total} encoded=[{string.Join(",", charIds)}] decoded={chars.Decode(charIds)}");

var mined = SubwordBuilder.FromSegments(new[] { "the cat sat", "the cat ran" }, topBi: 4, topTri: 2, topQuad: 0, includeAllDigit2: false);
var sw = new SubwordVocab(mined);
int[] subIds = sw.Encode("the cat");
Console.WriteLine($"mined=[{string.Join(",", mined)}] subwordVocabSize={sw.Size} encodedLen={subIds.Length}");

Console.WriteLine($"SubwordTable.List.Count={SubwordTable.List.Count}");   // the package's own shipped list

Data sources

All implement IJobSource (long Count; (int[] Ctx, int Target) GetExample(long index)), so a distributed host can ship only example positions — every node reconstructs the same example from the same source deterministically.

using PrismFormer;

var corpus = new CorpusSource(ctx: 4, segments: new[]
{
    "the quick brown fox jumps over the lazy dog, again and again, for no particular reason at all",
    "jumps over the lazy dog while the quick brown fox watches patiently from across the yard",
});
var (ctxTokens, target) = corpus.GetExample(0);
Console.WriteLine($"CorpusSource.Count={corpus.Count} example0: ctxLen={ctxTokens.Length} target={target}");

var pairs = new PairSource(ctx: 16, pairs: new[] { ("hello", "world"), ("foo", "bar") });
Console.WriteLine($"PairSource.Count={pairs.Count} PairCount={pairs.PairCount} SkippedTooLong={pairs.SkippedTooLong}");

var mixed = new MixSource(corpus, pairs);
Console.WriteLine($"MixSource.Count={mixed.Count}");

var tempDir = Path.Combine(Path.GetTempPath(), "algformer-package-doc-gossip");
var inbox = new GossipInbox(tempDir);
int added = inbox.Add(new[] { ("hi", "there") });
Console.WriteLine($"GossipInbox added={added} count={inbox.Count}");
Directory.Delete(tempDir, recursive: true);

var chatPairs = GroupChat.Pairs("user: hello\nprism: hi there\nuser: how are you\n");
Console.WriteLine($"GroupChat.Pairs -> {chatPairs.Count} pair(s)");

CorpusSource's default tokenizer is the full shipped subword vocabulary (CharVocab, ~36.9k entries at the time of writing), not raw characters — a short segment can tokenize to fewer tokens than its character count, and a segment with encodedLength <= minCtx (default minCtx = ctx) is silently dropped. Use long-enough segments (or a small ctx) or pass your own ITokenizer/CharVocab() if you need character-level, predictable-length windows.

Swarm

Distributed/federated training over MQTT — both host and worker only make outbound connections to a broker, so it works through any home NAT with no port-forwarding. Two transports:

• MqttRelay — full model + full batch data crosses the wire every round. Simple, works for any IBatchTrainer consumer, but ships your training data.
• MqttJobRelay (MqttJob*) — only a manifest (once) + example positions cross the wire; every worker reconstructs its shard locally from the same IJobSource. No training data ever leaves the host.

A third, independent layer — SwarmChatter — is a ping-ranked peer-gossip mesh for exchanging training pairs, weight slices, and inference queries between peers (not gradient-merge training).

Construction and wiring compile and run without a broker; only Start/StartRelay/JoinRoom actually connect out:

using PrismFormer;

var model = new AlgFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 5, dModel: 8, frozenPrefix: 0);
var relayHost = new MqttRelayHost(model) { ChunkTarget = 2048, MinRoundIntervalMs = 250 };
Console.WriteLine($"relay host ready: ActiveWorkers={relayHost.ActiveWorkers} Hosting={relayHost.Hosting}");
// relayHost.StartRelay(...) would dial out to MqttRelay.Broker — only call this with a real broker reachable.

var source = new SyntheticJobSource(count: 1000, vocab: model.Vocab, ctxLen: 3);
var jobHost = new MqttJobHost(model, source, manifest: source.Manifest());
Console.WriteLine($"job host ready: ActiveWorkers={jobHost.ActiveWorkers}");

var local = new LocalJobTrainer(model, source);
double trainedLoss = local.TrainPositions(new long[] { 0, 1, 2, 3 }, lr: 1e-2);
Console.WriteLine($"local job-trainer loss={trainedLoss:F4}");

string roomCode = MqttRelay.MakeRoomCode(MqttRelay.NewRoom());
string? parsedRoom = MqttRelay.ParseRoomCode(roomCode);
Console.WriteLine($"room code round-trips: {parsedRoom != null}");

var chatter = new SwarmChatter("demo-room", serve: _ => (0.9, "ok"), onPair: (_, __) => { }, signature: "demo-sig")
{
    AnswerQuorum = 2,
    BleedFanout = 2,
};
Console.WriteLine($"chatter constructed: PeerCount={chatter.PeerCount} ReconnectDropped={chatter.ReconnectDropped}");
// chatter.Start() would dial out to the broker — only call this with a real broker reachable.

Parallelism

using PrismFormer;

var model = new AlgFormer(vocab: 10, shifts: 3, layers: 2, maxContext: 5, dModel: 8, frozenPrefix: 0)
{
    Map = PrismEval.Cpu,   // serve-time fan-out across cores; leave the default SequentialMap for training
};
double[] logits = model.LogitsFor(new[] { 1, 2, 3 });
Console.WriteLine($"PrismEval.Licensed={PrismEval.Licensed} logits.Length={logits.Length}");

Self-check oracles

Both cores ship finite-difference gradient-check oracles and round-trip/regression self-checks as public static methods — call them from your own harness to prove the math on your build/runtime, not just trust the package's own CI. Each uses a small, fixed, internal toy model and returns true/false (with the worst relative error via out double worstRel where applicable).

using PrismFormer;

bool ok1 = AlgFormer.GradCheck(out var rel1);
bool ok2 = HoloFormer.GradCheckStackIter(out var rel2);
bool ok3 = HoloFormer.GradCheckStackIterAllPos(out var rel3);
Console.WriteLine($"GradCheck ok={ok1} rel={rel1:E2}; StackIter ok={ok2} rel={rel2:E2}; AllPos ok={ok3} rel={rel3:E2}");

Exceptions

Other public surface worth knowing about

• DistNet (public static class) exists as wire-format plumbing shared by the two MQTT relay transports; every one of its members is internal (visible only to an InternalsVisibleTo-granted assembly). A normal NuGet consumer sees an empty class — nothing to call directly. Use MqttRelayHost/ MqttJobHost/MqttRelayWorker/MqttJobWorker instead; they already use it for you.
• Phasor dependency. AlgFormer/HoloFormer's dModel/frozenPrefix default from Phasor.PhasorLayout.Dim/FrozenReals, and PhasorCodec.Encode(string) is the standard embedSeed source for a production-shaped model — both come from the separate EvaluatedApplications.Phasor NuGet package, pulled in automatically as a dependency.

License

Proprietary — this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key — every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.