EvaluatedApplications.HoloDb 5.1.2

HoloDb

An embeddable holographic SQL database for .NET. It speaks a real SQL dialect, is transactional (ACID with a write-ahead log), and stores data in a columnar engine with compiled predicates, holo-native constant-time aggregates, and hash joins — plus an optional durable larger-than-memory tier (paged buffer-pool + write-ahead log, no memory-mapped files). It runs inside an EvalApp harness, so parallel scans execute as compiled, resource-gated pipelines.

This file is the complete public surface: every public type, method and SQL feature this package ships, each with a one-line purpose and a working example.

Proprietary — this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key — every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.

Table of contents

• Quick start
• Changes in 5.1.2
• Changes in 5.1.1
• Changes in 5.1.0
• Core concepts
• Opening a database
• SQL reference
  • DDL: CREATE TABLE, ALTER TABLE, CREATE INDEX
  • DML: INSERT, UPDATE, DELETE
  • Parameterized queries
  • SELECT: projection, DISTINCT, aggregates
  • JOIN
  • WHERE operators
  • GROUP BY / HAVING, ORDER BY, LIMIT
  • NEAREST — holographic similarity search
  • Transactions: BEGIN / COMMIT / ROLLBACK
• BulkLoad — fast typed-array ingest
• QueryResult — reading results
• Durable, larger-than-memory (paged) tables
• Dependency injection
• EvalApp pipeline integration
• Usage metering and the billing model
• Parallel fan-out primitive (HoloPar)
• Advanced / low-level engine types
• Invariants and gotchas
• Exceptions
• File layout
• License

Quick start

dotnet add package EvaluatedApplications.HoloDb

using HoloDb;

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE users (id INT PRIMARY KEY, age INT, name TEXT)");
db.Execute("INSERT INTO users VALUES (1, 30, 'Ada')");

var result = await db.ExecuteAsync("SELECT name FROM users WHERE age >= 18");
foreach (var row in result.Rows) Console.WriteLine(row["name"]);
// Ada

Changes in 5.1.2

A statement is refused when any part of it would be ignored, instead of running as something else. LEFT, RIGHT, FULL, CROSS and NATURAL joins, JOIN ... USING, comma joins, OFFSET, UNION, INTERSECT, EXCEPT, a second statement after ;, ROLLBACK TO and trailing text after a statement each throw a HoloException that names what is not supported. A table alias throws and asks for the full table name. NEAREST combined with a join, GROUP BY, HAVING, ORDER BY, DISTINCT or an aggregate, HAVING without GROUP BY, DISTINCT with an aggregate, LIMIT 0 on an aggregate-only query and a non-integer LIMIT are refused the same way. Statements that ran correctly before are unchanged.

Changes in 5.1.1

A table with a Pleroma plane answers an equi-join COUNT or SUM faster. A join whose two tables have not been written since the same join last ran is served from a result memo kept with the plane: the first call, and the first call after any write to either table, costs the same as before, and the memo never changes a result. Dimension GROUP BY after a join on an INT key encodes through the column's zone map.

Changes in 5.1.0

New SQL (see the DDL table): CREATE INDEX [name] ON t (cols) [INCLUDE (cols)] [USING SORTED|SUMMARY] [WITH (...)], DROP INDEX, SHOW INDEXES and EXPLAIN.

New diagnostic hook: Database.RouteTrace is a thread-static List<string>?. Set it to a list and the engine appends the access paths and the route each statement on that thread takes; set it back to null to stop. EXPLAIN reports the same decision without running the query.

using HoloDb;

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE s (id INT PRIMARY KEY, n INT)");
db.Execute("INSERT INTO s VALUES (1, 1)");
db.Execute("INSERT INTO s VALUES (2, 2)");

Database.RouteTrace = new List<string>();
db.Execute("SELECT count(*) FROM s WHERE n > 1");
foreach (var step in Database.RouteTrace) Console.WriteLine(step);
Database.RouteTrace = null;
// access:block-wise mask scan
// route:predicate count (block-wise byte-mask, no position list)

foreach (var r in db.Execute("EXPLAIN SELECT count(*) FROM s WHERE n > 1").Rows)
    Console.WriteLine($"{r["kind"]}: {r["route"]}");
// access: block-wise mask scan
// route: predicate count (block-wise byte-mask, no position list)

Behaviour that differs from 5.0.0:

• Older binaries refuse new files. A write-ahead log or snapshot that contains INCLUDE, USING, WITH, DROP INDEX, SHOW INDEXES or EXPLAIN text cannot be opened by an older binary (its parser throws). The snapshot is written in the new v4 form only when a table has an index name or a SQL-created plane; every other database stays readable by a 5.0.0 binary. Run Checkpoint() on the new binary before an older one opens the same file.
• Rollback. BEGIN no longer copies the database; a rollback replays a journal. After a rolled-back DELETE the table holds the same rows, but their order may differ from before the transaction.
• Text GROUP BY is ordinal (case-sensitive). 'Ada' and 'ada' are two groups, where 5.0.0 merged them. The coded group-by paths (single-table and after a join) emit groups in ascending code order; add ORDER BY when order matters.
• Plane routing. In Auto mode a GROUP BY with a WHERE goes to the engine below 140,000 rows (a fixed engine constant); PleromaMode.Always still uses the plane.
• UPDATE validates every SET value before changing any row, so a bad value leaves the table untouched.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE s (id INT PRIMARY KEY, tag TEXT)");
db.Execute("INSERT INTO s VALUES (1, 'Ada')");
db.Execute("INSERT INTO s VALUES (2, 'ada')");
db.Execute("INSERT INTO s VALUES (3, 'Ada')");
foreach (var r in db.Execute("SELECT tag, count(*) FROM s GROUP BY tag ORDER BY tag").Rows)
    Console.WriteLine($"{r["tag"]} {r["count(*)"]}");
// Ada 2
// ada 1

Measured against 5.0.0 on an idle 16-core machine (release gate EXP-PB13, two runs of each build, rows identical to SQLite): no benchmark cell is slower; single-row autocommit UPDATE/DELETE at 1M rows went from about 69 ms/67 ms to about 12 us/9 us, BEGIN from 66 ms to 200 ns, and the 10M-row invoicing queries are faster on every join and group-by shape.

Core concepts

• Columns are authoritative; holograms are derived. Every row lives in plain typed columnar storage (long[]/double[]/string[]), scanned via compiled predicates. A holographic vector (VSA/HRR bind+bundle, used for NEAREST similarity search) is computed FROM the columns, never the other way round — losing a hologram loses nothing durable. Exact answers (sums, counts) come from exact integer accumulators, not from holograms.
• ACID. Atomicity + Isolation: one global lock serialises mutations, and a pre-statement undo journal (each changed row is journaled as it changes, BEGIN copies nothing) is replayed on error or ROLLBACK; DDL inside a transaction uses a snapshot. Durability: committed statements are write-ahead-logged and fsync'd. Consistency: schema/type, NOT NULL on the key, and PRIMARY KEY uniqueness are enforced on write.
• Two storage shapes, one SQL surface. In-RAM tables (CREATE TABLE by default) and paged, durable, larger-than-memory tables (CreatePaged/OpenPaged, or whole-database HoloDbOptions.Paged) are driven through the same Execute(sql)/BulkLoad calls — see Durable tables for where their capabilities differ.
• Constant-time exact aggregates. Whole-table and single-column-grouped COUNT/SUM/MIN/MAX on INT columns read a maintained accumulator instead of scanning — flat cost as the table grows. REAL aggregates and grouped REAL SUM are a fast SIMD scan, not maintained (see gotchas).
• Not a toy dialect, but not all of SQL either. Real WHERE/GROUP BY/HAVING/ORDER BY/LIMIT/DISTINCT/a single INNER JOIN/subquery IN/transactions, but no multi-way joins, no UPSERT, no NULL literal, one PRIMARY KEY per table. See the SQL reference for the exact grammar.

Opening a database

public sealed class HoloDbOptions
{
    public string? LicenseKey { get; set; }              // accepted, currently informational — nothing is gated by tier
    public string? WalPath { get; set; }                  // null = pure in-memory, no durability
    public long CheckpointThresholdBytes { get; set; }    // default 8 MB; 0 disables automatic checkpoints
    public bool Paged { get; set; }                       // opt in to whole-database paged-default mode (below)
}

var db = new HoloDbService(new HoloDbOptions { WalPath = "app.wal" });

• HoloDbService is the recommended entry point: async-friendly, licensed (informational only), wraps the engine below.
  • Execute(sql) — run a statement synchronously, return a QueryResult.
  • ExecuteAsync(sql, ct = default) — the same, offloaded to the thread pool.
  • BulkLoad(table, cols, data, count) — see BulkLoad.
  • Checkpoint() — fold the write-ahead log into a fresh snapshot now (no-op in-memory).
  • CreatePaged/OpenPaged/PagedBulkLoad/ClosePaged — see Durable tables.
  • Close() — release every open paged table's file handles; call before re-opening the same WalPath in-process, or on shutdown.
  • .Database — the underlying Database for direct synchronous access (below).
  • .LicenseMode — Unlicensed/Licensed; informational, gates nothing today.
• Database (HoloDb namespace) is the engine HoloDbService wraps; use it directly if you don't need the async wrapper or licensing shim.
  • Database.Open(walPath = null, checkpointThresholdBytes = 8 MB, paged = false) — static factory. walPath = null is pure in-memory. Recovery replays the write-ahead log over the last snapshot, so recovery time is bounded by live data, not by write history.
  • Execute(sql), BulkLoad(...), Checkpoint(), Close() — same contracts as HoloDbService.
  • WalPath — the configured log path, or null for in-memory (diagnostics).
  • IsPagedDefault — true iff this instance actually ended up in whole-database paged mode.
  • TableColumns(table) — the table's Column list in schema order, e.g. for building typed BulkLoad arrays without hardcoding types.
  • LastRowsScanned, LastTableRows, LastRowBytes, LastLogicalBytes — static, per-thread usage counters for the most recently executed query on the calling thread; see metering.

var path = "app.wal";
var db = new HoloDbService(new HoloDbOptions { WalPath = path });
db.Execute("CREATE TABLE t (id INT PRIMARY KEY, v INT)");
db.Execute("INSERT INTO t VALUES (1, 100)");
db.Checkpoint();
db.Close();

var reopened = new HoloDbService(new HoloDbOptions { WalPath = path });
var r = reopened.Execute("SELECT v FROM t WHERE id = 1");
Console.WriteLine(r.GetLong(0, 0));
// 100
reopened.Close();

SQL reference

DDL: CREATE TABLE, ALTER TABLE, CREATE INDEX

Column types accept SQL-ish synonyms: INT/INTEGER, TEXT/VARCHAR/STRING, REAL/FLOAT/DOUBLE/DECIMAL/NUMERIC — all three map onto exactly three physical storage kinds (long, string, double).

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE users (id INT PRIMARY KEY, age INT, name TEXT)");
db.Execute("CREATE TABLE IF NOT EXISTS users (id INT PRIMARY KEY, age INT, name TEXT)"); // no-op
db.Execute("INSERT INTO users VALUES (1, 30, 'Ada')");
db.Execute("ALTER TABLE users ADD COLUMN vip INT DEFAULT 0");
db.Execute("UPDATE users SET vip = 1 WHERE id = 1");
Console.WriteLine(db.Execute("SELECT vip FROM users WHERE id = 1").GetLong(0, 0));
// 1

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE events (id INT PRIMARY KEY, category TEXT, ts INT)");
db.Execute("CREATE INDEX ON events(category)");
db.Execute("CREATE INDEX ON events(category, ts)");   // composite
db.Execute("INSERT INTO events VALUES (1, 'login', 100), (2, 'login', 200), (3, 'logout', 50)");
Console.WriteLine(db.Execute("SELECT count(*) FROM events WHERE category = 'login'").Rows[0]["count(*)"]);
// 2

DML: INSERT, UPDATE, DELETE

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE p (id INT PRIMARY KEY, name TEXT)");
db.Execute("INSERT INTO p VALUES (1, 'Ada')");
try { db.Execute("INSERT INTO p VALUES (1, 'Dup')"); }
catch (HoloException ex) { Console.WriteLine(ex.Message); }
// duplicate primary key 1
try { db.Execute("UPDATE p SET id = 2 WHERE id = 1"); }
catch (HoloException ex) { Console.WriteLine(ex.Message); }
// cannot UPDATE the primary key

Parameterized queries

Every SQL method (Database.Execute, HoloDbService.Execute/ExecuteAsync) has a parameterized overload: ? placeholders, substituted in order by the trailing arguments as correctly quoted/ escaped SQL literals (ADO.NET-style positional binding) — added 2026-10, customer finding F-05.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE p (id INT PRIMARY KEY, name TEXT)");
db.Execute("INSERT INTO p VALUES (?, ?)", 1L, "O'Brien");
var r = db.Execute("SELECT name FROM p WHERE id = ?", 1L);
Console.WriteLine(r.Rows[0]["name"]);
// O'Brien

A string argument has its embedded ' doubled automatically — the same rule raw SQL text already requires ('O''Brien' when you write the literal yourself: a single quote inside a string literal is escaped by doubling it, per Sql.cs's lexer). Parameters don't replace that rule, they apply it for you. ? only has placeholder meaning OUTSIDE a string literal — WHERE name = 'what?' is untouched. Supported argument types: string, bool, any integer type (long/int/ short/.../ulong), any floating-point type (double/float/decimal). A null argument throws HoloException (this grammar has no NULL concept), as does an argument-count mismatch or an unsupported CLR type (e.g. DateTime — there is no date column type; encode a date as an INT day/epoch number instead). Database.Bind(sql, args) is also exposed directly if you want the literal SQL text without executing it (e.g. to log it).

var db2 = new HoloDbService(new HoloDbOptions());
string text = Database.Bind("SELECT * FROM t WHERE id = ? AND name = ?", 1L, "Widget");
Console.WriteLine(text);
// SELECT * FROM t WHERE id = 1 AND name = 'Widget'

SELECT: projection, DISTINCT, aggregates

SELECT [DISTINCT] col [, col ...] | agg(col) | *
FROM t
[INNER JOIN t2 ON ...]
[NEAREST (col = val, ...)]
[WHERE ...]
[GROUP BY col [, col ...]]
[HAVING ...]
[ORDER BY col [ASC|DESC] [, ...]]
[LIMIT n]

Aggregate functions: COUNT, SUM, MIN, MAX, AVG (COUNT(*) and COUNT(col) both supported). Result columns for aggregates are named in lower-case agg(col) form, e.g. count(*), sum(amount) — not the SQL text as typed, never aliased. AVG over an INT column returns a double, never integer-truncated.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE sales (id INT PRIMARY KEY, region TEXT, amount INT)");
db.Execute("INSERT INTO sales VALUES (1,'EU',100),(2,'EU',50),(3,'US',10)");

var agg = db.Execute("SELECT region, sum(amount) FROM sales GROUP BY region HAVING sum(amount) > 20 ORDER BY region LIMIT 10");
foreach (var row in agg.Rows) Console.WriteLine($"{row["region"]}={row["sum(amount)"]}");
// EU=150

var distinctRegions = db.Execute("SELECT DISTINCT region FROM sales ORDER BY region");
foreach (var row in distinctRegions.Rows) Console.WriteLine(row["region"]);
// EU
// US

JOIN

One INNER JOIN ... ON a = b per query (JOIN/INNER JOIN are equivalent). Qualify a projected or join-condition column with table.column when both sides share a name.

Anything else is refused with a HoloException naming it, never answered as if it were an inner join: LEFT/RIGHT/FULL [OUTER] JOIN, CROSS JOIN, NATURAL JOIN, JOIN ... USING (...), and comma joins (FROM a, b). Table aliases (FROM orders o, FROM orders AS o) are refused too: write the table name (orders.id). The same refusal covers any text left over after a complete statement (OFFSET, UNION, ROLLBACK TO ..., a second statement after ;), so a clause the dialect does not have can never be silently dropped.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE users (id INT PRIMARY KEY, name TEXT)");
db.Execute("INSERT INTO users VALUES (1, 'Ada')");
db.Execute("CREATE TABLE orders (id INT PRIMARY KEY, userId INT)");
db.Execute("INSERT INTO orders VALUES (1, 1)");
var j = db.Execute("SELECT users.name FROM orders INNER JOIN users ON orders.userId = users.id");
foreach (var row in j.Rows) Console.WriteLine(row["users.name"]);
// Ada

WHERE operators

AND / OR / NOT / parentheses, comparisons (= != < > <= >=, <> is accepted as a synonym for !=), LIKE (% = any run of characters, _ = any one character, case-insensitive), IN (v, ...) / IN (SELECT ...) (a scalar subquery), BETWEEN lo AND hi, each with an optional leading NOT.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE p (id INT PRIMARY KEY, name TEXT, price INT)");
db.Execute("INSERT INTO p VALUES (1,'Widget',25),(2,'Gadget',75),(3,'Widgetpro',150)");

db.Execute("SELECT name FROM p WHERE name LIKE 'Widget%'");       // Widget, Widgetpro
db.Execute("SELECT name FROM p WHERE id IN (1,2)");                // Widget, Gadget
db.Execute("SELECT name FROM p WHERE price BETWEEN 50 AND 100");   // Gadget
db.Execute("SELECT name FROM p WHERE id IN (SELECT id FROM p WHERE price > 50)");   // Gadget, Widgetpro

GROUP BY / HAVING, ORDER BY, LIMIT

GROUP BY accepts one or more columns; HAVING may reference an aggregate (count(*), sum(col), …) or a grouped column. ORDER BY accepts multiple keys with independent ASC/DESC. LIMIT n caps the row count (used alone or with NEAREST's top-k). See the aggregates example above — the same clauses apply there.

These combinations are refused with a HoloException because the engine would otherwise ignore one of the clauses: HAVING without GROUP BY (filter rows with WHERE), DISTINCT together with an aggregate (use GROUP BY, or COUNT(DISTINCT col)), and LIMIT 0 on an aggregate-only query. NEAREST accepts only a projection, WHERE and LIMIT: with JOIN, GROUP BY, HAVING, ORDER BY (the rows already come back ranked; select score to read the ranking), DISTINCT or an aggregate it is refused.

NEAREST — holographic similarity search

SELECT ... FROM t NEAREST (col = val [, col = val ...]) [WHERE ...] [LIMIT k]

Ranks rows by cosine similarity of their holographic encoding to a partial key (any subset of columns, any mix of INT/TEXT/REAL) and returns the top k (LIMIT; omitted = all rows, ranked). No separate vector index to build or keep in sync — it is derived from the same columns the table already holds, incrementally updated on INSERT (O(1) per new row). Below ~8,000 rows ranking is exact (brute-force cosine over every row); at/above that it switches to an approximate, sub-linear LSH candidate search — both return the same shape of result, just traded for speed at scale. An UPDATE/DELETE/ALTER TABLE on the table forces the next NEAREST to rebuild its index (bucket removal isn't tracked incrementally).

Similarity is graded, not exact-match-only (fixed 2026-10, customer finding F-04): an INT/ REAL column scores a close value higher than a far one (e.g. querying price = 1000, 900 and 1100 rank right behind the exact match, well ahead of 1800), and a TEXT column scores a near-identical string higher than an unrelated one via character-trigram overlap (e.g. querying name = 'ACME LTD', 'ACME LTD.' ranks well ahead of 'BOLT SUPPLIES'). This grading is a well-behaved LOCAL/mid-range decay, not a global guarantee that EVERY pair orders strictly by distance — two values several orders of magnitude apart can occasionally tie or swap rank with each other (both score low either way); it reliably separates "close" from "far", not every far value from every farther one. Add score to the projection to see the ranking metric itself (cosine similarity, descending, 1.0 for an exact match):

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE products (id INT PRIMARY KEY, category TEXT, price INT)");
db.Execute("INSERT INTO products VALUES (1,'electronics',100),(2,'electronics',120),(3,'furniture',300)");

var near = db.Execute("SELECT id, category, price, score FROM products NEAREST (category = 'electronics', price = 110) LIMIT 2");
foreach (var row in near.Rows) Console.WriteLine($"id={row["id"]} category={row["category"]} price={row["price"]} score={row["score"]:F3}");
// id=2 category=electronics price=120 score=0.989
// id=1 category=electronics price=100 score=0.989

score is a pseudo-column, like an aggregate's agg(col) name — it only appears when you ask for it by name; SELECT * keeps returning just the table's own columns (no shape change for existing callers). It is not a real schema column, so it can't appear in WHERE/GROUP BY/etc.

Transactions: BEGIN / COMMIT / ROLLBACK

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE p (id INT PRIMARY KEY, name TEXT)");

db.Execute("BEGIN");
db.Execute("INSERT INTO p VALUES (1, 'Temp')");
db.Execute("ROLLBACK");
Console.WriteLine(db.Execute("SELECT count(*) FROM p").Rows[0]["count(*)"]);
// 0

db.Execute("BEGIN");
db.Execute("INSERT INTO p VALUES (1, 'Kept')");
db.Execute("COMMIT");
Console.WriteLine(db.Execute("SELECT count(*) FROM p").Rows[0]["count(*)"]);
// 1

A multi-statement transaction is a series of separate Execute calls on the same Database instance — the engine's internal lock releases between them, so if multiple callers share one Database concurrently, holding a transaction open across statements from different callers is the caller's own responsibility (serialise/gate it yourself). BulkLoad cannot run inside an explicit transaction (throws).

BulkLoad — fast typed-array ingest

int BulkLoad(string table, IReadOnlyList<string> cols, IReadOnlyList<Array> data, int count)

Appends count rows from typed column arrays (long[] for INT, double[] for REAL, string[] for TEXT) — one array per name in cols — bypassing the SQL parser entirely. Atomic (validates every key up front); throws on a duplicate primary key, same as INSERT. A column omitted from cols is filled with its type's zero value, exactly like a partial INSERT.

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE users (id INT PRIMARY KEY, age INT, name TEXT)");
var cols = new[] { "id", "age", "name" };
var data = new Array[] { new long[] { 10, 11 }, new long[] { 40, 22 }, new[] { "Bo", "Cy" } };
int n = db.BulkLoad("users", cols, data, 2);
Console.WriteLine(n);
// 2

QueryResult — reading results

Shape reference (abbreviated — bodies omitted, not literal source):

public sealed class QueryResult
{
    public int Affected { get; }                                       // rows affected (mutations) / returned (SELECT)
    public List<Dictionary<string,object>> Rows { get; }                // lazily materialised row-dict view; always safe
    public bool IsColumnar { get; }                                     // true for a plain, non-aggregated projection scan
    public int RowCount { get; }
    public IReadOnlyList<string> Columns { get; }
    public bool IsIntColumn(int col); public bool IsRealColumn(int col);
    public long GetLong(int row, int col); public double GetDouble(int row, int col); public string GetText(int row, int col);
    public object GetValue(int row, int col);                           // works for both columnar and row-dict results
    public static QueryResult FromRows(List<Dictionary<string,object>> rows, string[] columns);  // rebuild from wire data (HoloDb.Protocol)
}

.Rows (a case-insensitive Dictionary<string,object> per row) always works, for any query shape, and is the simplest way to read a result. IsColumnar is true only for a plain projection scan — SELECT col[, col...] FROM t [WHERE ...] [ORDER BY ...] [LIMIT ...] with no aggregate, GROUP BY, JOIN, DISTINCT, or NEAREST — in which case GetLong/GetDouble/ GetText read the underlying typed arrays with zero boxing, the fast path for large result sets. For every other query shape (aggregates, GROUP BY, JOIN, DISTINCT, NEAREST), use .Rows or GetValue (never GetLong/GetText/GetDouble directly — they assume columnar storage and will fail if it isn't there).

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE t (id INT PRIMARY KEY, name TEXT)");
db.Execute("INSERT INTO t VALUES (1, 'Ada'), (2, 'Bo')");
var r = db.Execute("SELECT id, name FROM t WHERE id >= 1 ORDER BY id");   // plain projection -> columnar
Console.WriteLine(r.IsColumnar);
// True
for (var i = 0; i < r.RowCount; i++) Console.WriteLine($"{r.GetLong(i, 0)}:{r.GetText(i, 1)}");
// 1:Ada
// 2:Bo

Durable, larger-than-memory (paged) tables

For data that outgrows RAM, create a paged table: stored on a buffer-pool + write-ahead-log substrate (plain FileStream-based, not memory-mapped), so it is durable and its data can exceed memory — cold pages demand-page back from disk. Whole-table aggregates and single-column GROUP BY count/sum stay O(1) regardless of table size. INSERT/UPDATE/DELETE/SELECT work through the normal Execute(sql) path once the table exists; transactions, primary-key uniqueness (with O(1) point lookup), a single INNER JOIN (COUNT/SUM only — no GROUP BY over a join, and no multi-way join), GROUP BY/HAVING, ORDER BY, LIMIT, DISTINCT, and NEAREST are all supported on a paged table. The primary-key index is kept in RAM (about 35 bytes/row, measured 34 to 39) — for a table far larger than RAM, that index, not the buffer pool, is the limiting resource.

• CreatePaged(createTableSql, dir, bufferFrames = 8192) — create a table backed by the paged store in dir (RAM budget = bufferFrames × 8 KB). Returns the table name.
• OpenPaged(dir, ifNotExists = false) — re-open a paged table previously created in dir, recovering committed state from its log. ifNotExists: if a table of that name is already registered on this Database, return its name instead of throwing.
• PagedBulkLoad(table, rows, commitEvery = 4096) — stream object?[] rows (schema column order) into a paged table, committing every commitEvery rows. Returns the row count.
• ClosePaged(table) — checkpoint and close one paged table's store.

var db = new HoloDbService(new HoloDbOptions());
var dir = Path.Combine(Path.GetTempPath(), "orders-paged");
db.CreatePaged("CREATE TABLE big (id INT PRIMARY KEY, v INT)", dir);
db.PagedBulkLoad("big", new object?[][] { new object?[] { 1L, 100L }, new object?[] { 2L, 200L } });
Console.WriteLine(db.Execute("SELECT sum(v) FROM big").Rows[0]["sum(v)"]);
// 300
db.ClosePaged("big");

var reopened = db.OpenPaged(dir);               // recovers from the log
Console.WriteLine(db.Execute($"SELECT count(*) FROM {reopened}").Rows[0]["count(*)"]);
// 2
db.ClosePaged(reopened);

Whole-database paged-default mode (HoloDbOptions.Paged = true, or Database.Open(walPath, ckpt, paged: true)) lets a plain CREATE TABLE/BulkLoad target the paged substrate automatically — no CreatePaged/PagedBulkLoad call needed. It only takes effect for a brand-new database (no existing WalPath/WalPath + ".snap" on disk); opening an existing legacy database with Paged = true silently falls back to the in-RAM engine — check Database.IsPagedDefault to see which mode you actually got.

var dir = Path.Combine(Path.GetTempPath(), "pageddefault-demo");
Directory.CreateDirectory(dir);
var walPath = Path.Combine(dir, "db.wal");
var pd = new HoloDbService(new HoloDbOptions { WalPath = walPath, Paged = true });
pd.Execute("CREATE TABLE t (id INT PRIMARY KEY, v INT)");
pd.Execute("INSERT INTO t VALUES (1, 42)");
Console.WriteLine(pd.Database.IsPagedDefault);
// True
Console.WriteLine(pd.Execute("SELECT v FROM t WHERE id = 1").Rows[0]["v"]);
// 42
pd.Close();

Pleroma aggregate plane and COUNT(DISTINCT)

An optional, derived summary of a table's small-domain columns, held in Phasor Pleroma stores. The columns stay the source of truth; the plane is rebuilt from them at any time and never answers approximately. Pick dimensions (INT or TEXT columns, used by WHERE, GROUP BY, COUNT(DISTINCT), MIN/MAX and join keys) and measures (INT columns whose SUM is served; exact past Int64 internally).

using HoloDb;

var db = Database.Open();
db.Execute("CREATE TABLE sales (id INT PRIMARY KEY, region TEXT, tier INT, qty INT)");
db.Execute("INSERT INTO sales (id, region, tier, qty) VALUES (1,'north',1,10),(2,'north',2,-4),(3,'south',1,7),(4,'east',3,5),(5,'south',2,9)");

// attach the plane: dimensions answer WHERE / GROUP BY / COUNT DISTINCT, measures answer SUM
db.EnablePleroma("sales", new PleromaPlaneOptions(new[] { "region", "tier" }, new[] { "qty" }));
db.PleromaMode = PleromaMode.Always;   // serve every shape the plane covers (the default, Auto, leaves O(1) accumulator shapes to the engine)

var r = db.Execute("SELECT COUNT(*), SUM(qty) FROM sales WHERE region = 'north' OR tier = 3");
Console.WriteLine($"{r.Rows[0]["count(*)"]} {r.Rows[0]["sum(qty)"]}");                 // 3 11

r = db.Execute("SELECT COUNT(DISTINCT region) FROM sales");
Console.WriteLine(r.Rows[0]["count(distinct region)"]);                                 // 3

db.Execute("DELETE FROM sales WHERE id = 4");                                           // writes keep the plane current
Console.WriteLine(db.Execute("SELECT COUNT(DISTINCT region) FROM sales").Rows[0]["count(distinct region)"]);   // 2

Console.WriteLine(db.PleromaVerify("sales") ?? "plane equals a rebuild from the columns");   // plane equals a rebuild from the columns
Console.WriteLine(db.PleromaSumExact("SELECT SUM(qty) FROM sales"));                    // 22

The same plane from plain SQL (so a remote client can ask for it too):

using HoloDb;

var db = Database.Open();
db.Execute("CREATE TABLE sales (id INT PRIMARY KEY, region TEXT, tier INT, qty INT)");
db.Execute("INSERT INTO sales (id, region, tier, qty) VALUES (1,'north',1,10),(2,'north',2,-4),(3,'south',1,7)");
db.Execute("CREATE INDEX by_region ON sales (region, tier) INCLUDE (qty) USING SUMMARY");

var show = db.Execute("SHOW INDEXES ON sales");
Console.WriteLine($"{show.GetValue(0, 1)} {show.GetValue(0, 2)} {show.GetValue(0, 3)}");   // by_region summary region,tier
db.PleromaMode = PleromaMode.Always;
Console.WriteLine(db.Execute("SELECT SUM(qty) FROM sales WHERE region = 'north'").Rows[0]["sum(qty)"]);   // 6
db.Execute("DROP INDEX by_region");

• Served by the plane: COUNT and SUM (with any AND/OR/NOT/LIKE/IN/BETWEEN over dimensions), GROUP BY on one or several dimensions with HAVING on count/sum, COUNT(DISTINCT dim), MIN/MAX of a dimension, and a single equi-join COUNT / SUM between two tables that both have planes.
• Declined (the engine answers): AVG, columns that are not plane dimensions or measures, a dimension past MaxDimBits distinct values, multi-dimension queries when the joint width exceeds MaxJointBits, joins with a WHERE or more than 4095 distinct keys. A declined query is never wrong, only unaccelerated.
• PleromaMode: Auto (default) leaves whole-table and single-column GROUP BY to the engine's own O(1) accumulators; Always serves every covered shape; Off bypasses the plane while it is still maintained.
• SQL form: CREATE INDEX sm ON sales (region, tier) INCLUDE (qty) USING SUMMARY builds the same plane, works through HoloDb.Server like any statement, is replayed from the write-ahead log on Open, survives Checkpoint() (the snapshot stores the definition and rebuilds the plane at load), and is undone by ROLLBACK. DROP INDEX sm removes it; SHOW INDEXES reports its size, served/declined counts and over-cap dimensions.
• Lifecycle: in-RAM tables only. A plane attached with EnablePleroma is not persisted: call it again after Database.Open (a SQL-created one is persisted). ROLLBACK restores it with the table. PleromaStats reports sizes and served/declined counts; PleromaVerify compares it with a fresh rebuild.
• Sums past Int64: the plane raises an error instead of wrapping; PleromaSumExact returns the exact Int128.
• COUNT(DISTINCT col) is a general SQL addition: it must be the only kind of select item (several COUNT(DISTINCT ...) items are allowed), with no GROUP BY or JOIN, and is not available on paged tables.
• Memory: the plane's cells are about 0.92 MB for a 14-bit joint whatever the row count (plus the same again in pending deltas before a flush), and attaching a plane added 1.9 to 4.6 MB of heap in the benchmark (measured at 100,000 to 1,000,000 rows).

Pleroma page store (the Pleroma as the data)

PleromaPageStore keeps a family of Phasor Pleroma pages of one shape (for example dense CompleteKeys pages, one per context) durably and larger than memory, with no columns behind them. Each page is saved through RawCells and RawCheckCells into HoloDb's paged buffer pool and write-ahead log; hot pages are live Pleroma objects in a byte-budgeted cache, cold pages load on demand.

using HoloDb;
using Phasor;

var shape = new HologramShape { KeyCapacity = 2, KeyCycle = 110, CompleteKeys = true };   // every page: 110 exact key counts
var dir = Path.Combine(Path.GetTempPath(), "plrstore-demo");
if (Directory.Exists(dir)) Directory.Delete(dir, true);

using (var store = PleromaPageStore.Open(dir, shape))
{
    // adds are journaled before they are applied; page 42 and page 7 are created on first use
    store.AddBatch(new[] { new PageAdd(42, 5, 3), new PageAdd(42, 77, -2), new PageAdd(7, 5, 1) });
    store.AppendEvent("context 42 seen"u8);                       // an opaque owner event, ordered with the batches
    Console.WriteLine($"{store.Counts(42)[5]} {store.Counts(42)[77]}");                 // 3 -2
}

using (var store = PleromaPageStore.Open(dir, shape))             // reopened: the pages come back exactly
{
    Console.WriteLine($"{store.Counts(42)[5]} {store.Counts(7)[5]}");                   // 3 1
    store.ReplayEvents((seq, bytes) => Console.WriteLine($"{seq} {System.Text.Encoding.UTF8.GetString(bytes)}"));   // 2 context 42 seen
    Console.WriteLine(store.Verify().Clean);                                            // True  (every record's check and parity group)
    Console.WriteLine(store.Stats().Pages);                                             // 2
}
Directory.Delete(dir, true);

• API: Open(path, shape, options), AddBatch(IReadOnlyList<PageAdd>) (returns the batch's sequence number), Get(pageKey) (a copy; null if absent), Read(pageKey, fn) (zero-copy), Counts(pageKey), Contains, PageKeys, KeysInRange(lo, hi) (children or prefix scan when the owner packs a parent into the high bits of the key), AppendEvent / ReplayEvents / TrimEvents, Checkpoint, Checkpoints, RestoreTo, Verify, Repair, Stats, Dispose. Page keys are non-negative longs.
• Concurrency: readers never take a store-wide lock. A read holds a shared lock on its page's stripe (1,024 stripes), so it waits only for a write to a page on the same stripe, or for Verify / Repair / RestoreTo. One writer at a time. A batch is applied page by page: a reader sees every page whole, but may see a batch applied to some pages and not yet to others. A stress test compares concurrent readers with a single-threaded oracle.
• Writes: every AddBatch is appended to the journal (fsynced unless SyncEveryBatch is false) before it is applied. Each page remembers the sequence number of the last batch applied to it, so crash recovery replays a logged add exactly once, even over pages the cache had already flushed. A torn journal tail is discarded, so a batch survives whole or not at all.
• Restore points: Checkpoint() returns a PleromaCheckpoint (id and journal sequence number). RestoreTo(id) returns the store to exactly that state, with no recount: pages written since go back, pages created since vanish, later points are dropped. It is copy-on-write: before a record is first overwritten after a restore point, its old bytes are appended to that point's undo file, so a restore point costs one before-image per page changed since, not a copy of the data file. MaxCheckpoints (default 8) bounds how many are kept. Events stay in the event journal (read them with ReplayEvents(afterSeq)).
• Event journal: AppendEvent stores an opaque record in the same sequence space as the batches; ReplayEvents returns them in order; TrimEvents(upToSeq) drops the absorbed prefix and sequence numbers never repeat.
• Redundancy: the store's shape carries a Phasor CheckPrime: every page keeps a mirror of its cells in a second prime, both copies are saved, and Pleroma.VerifyCheck proves a record intact (Verify runs it on every page; VerifyOnLoad runs it on every load, about 100 microseconds for a 110-key page). Records are grouped (ParityGroup, default 8) with one PleromaParity exact-sum record per group (one over the main cells, one over the check cells), and the members of a group sit on different 8 KB pager pages, so any one lost pager page, or any one damaged record per group, is rebuilt by Repair. A rebuilt page must pass its own check before it is written; a member and its group's parity both damaged are reported as unrecoverable, never rebuilt wrong.
• Cache: CacheBytes (default 64 MB) bounds the live pages (clock eviction); Stats reports pages, resident pages and bytes, hits, misses and hit rate, page flushes, the last sequence number and the restore points kept.
• Limits: one shape per store (reopening with another shape is refused); the shape must be CompleteKeys and admissible for a check prime; shapes with a continuous component, a relation face or a non-unit TimeScale cannot be saved; one process opens a store at a time.

Sparse pages for a large vocabulary

PleromaSparsePageStore stores Phasor PleromaSparsePage pages (a histogram over the whole 2^61 key cycle whose size follows the keys present) with the same journal, exactly-once replay, striped concurrent reads and cache. Pages have different sizes, so records come in capacity classes: class c holds pages of capacity up to 2^c in a record of one fixed size, laid out as header, capacity, cells, support count and RawKeys (several records to an 8 KB pager page for small classes, a chain of pages for large ones). When a page outgrows its class the next flush writes it into the larger class and appends a new directory entry (the directory is last-entry-wins). AddBatch(IReadOnlyList<SparseAdd>), Get, Read, Counts, KeysInRange, Verify (header hash, then support against cells), Checkpoint, Stats (pages per class, class changes). Parity groups, restore points and the event journal are in the dense store only.

Dependency injection

public static IServiceCollection AddHoloDb(this IServiceCollection services, Action<HoloDbOptions>? configure = null)

Registers a singleton HoloDbService. Needs only IServiceCollection (Microsoft.Extensions.DependencyInjection.Abstractions) — add the full Microsoft.Extensions.DependencyInjection package (or use ASP.NET Core / Generic Host, which already provides it) to get BuildServiceProvider().

var services = new ServiceCollection();
services.AddHoloDb(o => o.WalPath = null);
var provider = services.BuildServiceProvider();
var db = provider.GetRequiredService<HoloDbService>();
db.Execute("CREATE TABLE ping (id INT PRIMARY KEY)");
db.Execute("INSERT INTO ping VALUES (1)");
Console.WriteLine(db.Execute("SELECT count(*) FROM ping").Rows[0]["count(*)"]);
// 1

EvalApp pipeline integration

public static IAppBuilder WithHoloDb(this IAppBuilder builder, HoloDbService service)

Attaches a HoloDbContext (an EvalApp GlobalContext wrapping .Service) to a pipeline app, so a context-aware step (ContextPureStep<HoloDbContext, TDomain, T>) can reach global.Service inside TransformAsync without threading a connection through every record. WithContext/WithHoloDb hold one global per Eval.App (last call wins) — build one pipeline app per database you need to reach from steps.

var service = new HoloDbService(new HoloDbOptions());
service.Execute("CREATE TABLE users (id INT PRIMARY KEY, name TEXT)");
service.Execute("INSERT INTO users VALUES (1, 'Ada')");

Eval.App("Catalog")
    .WithHoloDb(service)
    .DefineDomain("Catalog")
        .DefineTask<LookupQuery>("LookupUser")
            .AddStep<LookupUserStep>("Lookup")
        .Run(out ICompiledPipeline<LookupQuery> pipeline)
    .Build();

var outcome = await pipeline.RunAsync(new LookupQuery(1));
if (outcome is PipelineResult<LookupQuery>.Success ok) Console.WriteLine(ok.Data.Name);
// Ada

// Top-level statements must precede type declarations in a Program.cs:
public sealed record LookupQuery(long Id, string Name = "");

public sealed class LookupUserStep : ContextPureStep<HoloDbContext, LookupUserStep.NoDomain, LookupQuery>
{
    public sealed class NoDomain : DomainContext { }
    protected override ValueTask<LookupQuery> TransformAsync(LookupQuery data, HoloDbContext global, NoDomain domain, CancellationToken ct)
    {
        var r = global.Service.Database.Execute($"SELECT name FROM users WHERE id = {data.Id}");
        return ValueTask.FromResult(data with { Name = (string)r.Rows[0]["name"] });
    }
}

Usage metering and the billing model

HoloDb.Analytics wraps a Database to report, per query, the WORK actually done versus what a per-scan warehouse (BigQuery, Athena) would bill for the same query on the same metric (bytes processed) — the spine of the metered analytics API this engine also runs behind.

Shape reference (abbreviated — bodies omitted, not literal source):

public readonly record struct QueryUsage(long RowsScanned, long TableRows, long RowBytes, long BilledBytes, double ElapsedMs)
{
    public long BytesActuallyRead { get; }   // RowsScanned * RowBytes — 0 for a maintained-accumulator aggregate
    public bool ConstantTime { get; }         // true iff RowsScanned == 0
    public double MarginHeadroom { get; }     // BilledBytes / BytesActuallyRead
}
public sealed record ScanPricing(string Name, double UsdPerTiB, long MinBytesPerQuery) { /* .BigQuery, .Athena, .All */ }
public sealed record HoloPricing(double UsdPerTiB = 0.625, long MinBytesPerQuery = 10L * 1024 * 1024) { public double CostUsd(QueryUsage u); public double DiscountVs(ScanPricing warehouse); }
public sealed class MeteredDatabase
{
    public MeteredDatabase(Database db, HoloPricing? pricing = null);
    public Database Inner { get; }
    public (QueryResult Result, QueryUsage Usage) Run(string sql);
    public double OurCostUsd(QueryUsage u);
    public double ScanCostUsd(QueryUsage u, ScanPricing engine);
}

var db = new HoloDbService(new HoloDbOptions());
db.Execute("CREATE TABLE t (id INT PRIMARY KEY, v INT)");
db.Execute("INSERT INTO t VALUES (1, 100), (2, 200)");
var metered = new MeteredDatabase(db.Database);
var (res, usage) = metered.Run("SELECT sum(v) FROM t");
Console.WriteLine(usage.ConstantTime);
// True  (served from the maintained accumulator, no scan)
Console.WriteLine(metered.OurCostUsd(usage) <= metered.ScanCostUsd(usage, ScanPricing.BigQuery));
// True

Database.LastRowsScanned/LastTableRows/LastRowBytes/LastLogicalBytes (static, per-thread) are the lower-level counters MeteredDatabase.Run reads; use them directly if you call Database.Execute yourself and just want the numbers without the pricing wrapper.

Parallel fan-out primitive (HoloPar)

public interface IParallelMap { void Map(int chunks, Action<int> body /* + compiler-filled caller info */); }
public static class HoloPar { public static IParallelMap Cpu { get; } public static bool Licensed { get; } }

The single parallel primitive the columnar query engine fans out through internally (never a raw Parallel.For/new Thread) — exposed publicly, but you don't need to call it: every query path that benefits from it already does. It runs on EvalApp's adaptive dispatch: each call site learns its own cost and runs inline when a fan-out would not pay. Licensed is always true and is not a license/tier check — nothing here is license-gated.

var sum = 0; var gate = new object();
HoloPar.Cpu.Map(4, i => { lock (gate) sum += i; });
Console.WriteLine(sum);
// 6

Advanced / low-level engine types

These are public types (the assembly ships them unmarked internal), but none of them is returned by, or needed to call, Database/HoloDbService/SQL — they are the engine's own building blocks. Documented here for completeness; prefer the SQL surface above for anything application-level.

var schema = new Schema(new[] { new Column("id", ColType.Int, true), new Column("tag", ColType.Text, false) });
var v1 = Holo.EncodeRow(schema, new Dictionary<string, object?> { ["tag"] = "red" });
var v2 = Holo.EncodeRow(schema, new Dictionary<string, object?> { ["tag"] = "red" });
Console.WriteLine(Holo.Similarity(v1, v2) > 0.99);
// True

Invariants and gotchas

• No UPSERT. INSERT on an existing primary key always throws; do DELETE + INSERT, or check existence first, for upsert semantics.
• One global reader-writer lock. SELECT takes the shared read lock (many run concurrently); every mutation/DDL/transaction statement takes the exclusive write lock (serialised). A lock-free "live metrics" fast path serves pure-accumulator reads (whole-table COUNT/SUM, a single-column GROUP BY) off a snapshot published after each write, so a streaming ingest never stalls a dashboard read.
• A multi-statement transaction is a series of separate Execute calls — the lock releases between them, so cross-statement isolation between different callers sharing one Database is the caller's own responsibility.
• NEAREST similarity is graded but not a global total order (2026-10 fix, F-04): close values reliably rank ahead of far ones at local/mid range; it is not guaranteed that EVERY pair of far values orders strictly by distance from each other (both score low either way). See the NEAREST section above.
• No AS aliasing (columns or tables), no NULL. Neither exists anywhere in the grammar today; a table alias in a FROM/JOIN throws. Projected columns always keep their raw/agg(col) name (read QueryResult by position if you want a different label); every column has a type-zero default (0/0.0/""), never a null state.
• Indexes are INT or TEXT only, never REAL.
• REAL columns never get a maintained accumulator (INT sums are exact Int128 running totals). Whole-table REAL SUM/MIN/MAX/AVG is a SIMD scan (still sub-millisecond at typical sizes); grouped REAL SUM scans too — the one place this engine doesn't maintain a running accumulator.
• Aggregate result columns are lower-case agg(col), e.g. count(*), sum(x) — never the SQL text as typed, never an alias.
• Paged tables have a narrower JOIN ceiling than in-RAM tables: one INNER JOIN, COUNT/SUM only, no GROUP BY over a join. This is exactly why HoloDbOptions.Paged/Database.Open(..., paged: true) is an opt-in, not the library default.
• BulkLoad cannot run inside an explicit transaction (throws).
• A legacy (non-paged) database's write-ahead log is literally SQL text, replayed by re-parsing it on the next Open. A build that understands a newer keyword (e.g. ALTER) writes that text into the log; an older binary's parser then throws on it and refuses to open the database until a newer binary checkpoints it first (Checkpoint() writes a fresh schema-only snapshot + empty log). Mixed binary versions sharing one on-disk legacy database must checkpoint on the newer binary before an older one opens that file. Paged tables have no such risk — their log is a generic, statement-agnostic physical redo log of raw pages.
• DebugType=none, no XML doc file, no PDB ship in the package — no IntelliSense doc text or symbol debugging from the NuGet package itself; this file is the documentation.
• HoloDbOptions.LicenseKey/Database's license plumbing is accepted but has no effect on which capabilities are available — nothing in this package is gated behind a tier or a license key.

Exceptions

Every error this engine raises deliberately (bad SQL, a schema violation, a duplicate key, an unsupported shape) is a HoloException : Exception with a plain message — catch that one type:

public sealed class HoloException : Exception { public HoloException(string message); }

File layout

• In-memory (WalPath = null): nothing touches disk; state does not survive the process.
• Legacy durable (WalPath set, Paged not set / existing legacy database): WalPath is an append-only text log of committed SQL; WalPath + ".snap" is the latest binary snapshot of all table data. Checkpoint() folds the log into a fresh snapshot and truncates the log.
• Paged / durable larger-than-memory (CreatePaged(sql, dir, ...), or whole-database Paged = true): one self-contained paged store per table — a buffer pool + a physical page-level write-ahead log — under its own directory (dir, or one subdirectory per table under WalPath + ".tables" in whole-database paged mode), auto-discovered on Open. No memory-mapped files anywhere in either storage shape.

License

Proprietary — this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key — every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.

HoloDbOptions.LicenseKey is accepted but has no effect on which capabilities are available.