EvaluatedApplications.HoloDb.Protocol 1.1.3

EvaluatedApplications.HoloDb.Protocol

The wire contract between a HoloDb server and HoloDb.Client: length-prefixed TCP framing plus a columnar-preserving binary codec. Requests and responses encode to/from typed column arrays (long[], double[], string[]), keeping the engine's columnar edge on the wire.

Use this package if you are implementing a HoloDb client or server. If you are writing an application that queries HoloDb, use EvaluatedApplications.HoloDb.Client instead — it handles the protocol for you.

Table of Contents

1. Quick Start
2. Protocol Basics
3. Wire Framing
4. Version Negotiation
5. Request Encoding (Client → Server)
6. Response Decoding (Client)
7. Request Decoding (Server)
8. Response Encoding (Server)
9. Exception Types
10. Wire Format Reference
11. Gotchas & Constraints

Quick Start

Install the package:

dotnet add package EvaluatedApplications.HoloDb.Protocol

Encode a client Hello handshake with version negotiation, send a query, and decode the result:

using HoloDb.Protocol;

// Version negotiation: client Hello includes the protocol version this build speaks
byte[] helloRequest = HoloProtocol.Hello("MyClient/1.0", "auth-token");
Console.WriteLine($"Hello frame: {helloRequest.Length} bytes, first byte (opcode): 0x{helloRequest[0]:X2}");

// Encode a query request
byte[] queryRequest = HoloProtocol.Exec("SELECT id, name FROM users WHERE id > 100");

// In real code, frame and send over a TCP stream:
// var stream = new NetworkStream(...);
// await Wire.WriteFrameAsync(stream, helloRequest, CancellationToken.None);
// await Wire.WriteFrameAsync(stream, queryRequest, CancellationToken.None);

// Decode a Hello response (when receiving from server):
// byte[] helloResponse = await Wire.ReadFrameAsync(stream, CancellationToken.None);
// string serverInfo = HoloProtocol.DecodeHelloAck(helloResponse); // throws on version mismatch

// Decode a query response:
// byte[] queryResponse = await Wire.ReadFrameAsync(stream, CancellationToken.None);
// QueryResult result = HoloProtocol.DecodeResult(queryResponse);
// Console.WriteLine($"Rows: {result.RowCount}");
// for (int r = 0; r < result.RowCount; r++)
// {
//     long id = result.GetLong(r, 0);        // first column, no boxing
//     string name = result.GetText(r, 1);    // second column
//     Console.WriteLine($"  {id}: {name}");
// }

Protocol Basics

Wire Opcodes (WireOp enum)

The first byte of every frame payload is an opcode indicating the message type:

Opcode Value	Name	Direction	Purpose
0x01	Hello	Client → Server	Handshake with client info and auth token
0x10	Exec	Client → Server	Execute SQL query or transaction command
0x11	BulkLoad	Client → Server	Load typed column arrays into a table
0x1E	Stats	Client → Server	Request server statistics
0x1F	Ping	Client → Server	Keep-alive probe
0x80	Affected	Server → Client	Mutation result (row count only)
0x81	Columnar	Server → Client	Query result with typed column arrays
0x82	Error	Server → Client	Error response (any opcode)
0x83	HelloAck	Server → Client	Handshake acknowledgment
0x84	Rows	Server → Client	Query result with materialised dict rows
0x85	Stats	Server → Client	Statistics response
0x86	VersionMismatch	Server → Client	Protocol version incompatibility (replaces HelloAck)
0x8F	Pong	Server → Client	Keep-alive response

Column type tags (private, internal use only):

• 0 = long (INT columns)
• 1 = double (REAL columns)
• 2 = string (TEXT columns)

Value tags in row-dict responses (private, internal use only):

• 0 = long (including int, upcast to long)
• 1 = double (REAL)
• 2 = string (TEXT)
• 3 = null (no value)

Wire Framing

The Wire class provides transport-agnostic length-prefixed framing over any duplex Stream (TCP NetworkStream, SslStream, test MemoryStream, etc.).

Wire.WriteFrameAsync(Stream s, byte[] payload, CancellationToken ct) — Send a frame

Purpose: Write a frame with length prefix and flush.

Wire format:

• 4 bytes: Int32 little-endian payload length (network byte order)
• N bytes: payload (first byte = opcode)

Example:

using HoloDb.Protocol;
using System.IO;

var payload = HoloProtocol.Hello("client", "token");
var stream = new MemoryStream();
await Wire.WriteFrameAsync(stream, payload, CancellationToken.None);

// Frame bytes: [04, 00, 00, 00] + [01, 01, ...] = len=4 in LE, then opcode 0x01
Console.WriteLine($"Frame length: {stream.Length} bytes");

Wire.ReadFrameAsync(Stream s, CancellationToken ct) — Receive a frame

Purpose: Read a frame with length prefix validation. Returns null on clean end-of-stream at a frame boundary; throws HoloProtocolException on truncated frame or out-of-range length.

Example:

using HoloDb.Protocol;
using System.IO;

var stream = new MemoryStream();

// Simulate receiving a frame
byte[] payload = HoloProtocol.Ping();
await Wire.WriteFrameAsync(stream, payload, CancellationToken.None);

// Read it back
stream.Position = 0;
byte[]? frame = await Wire.ReadFrameAsync(stream, CancellationToken.None);

if (frame != null && HoloProtocol.IsPong(frame))
    Console.WriteLine("Received Pong");
else if (frame == null)
    Console.WriteLine("Stream closed cleanly");
else
    Console.WriteLine($"Received frame, first byte: 0x{frame[0]:X2}");

Wire.MaxFrame constant

Purpose: Hard ceiling (256 MiB) to prevent a corrupt or hostile length prefix from forcing a giant allocation. Frames larger than this raise HoloProtocolException.

Example:

using HoloDb.Protocol;

Console.WriteLine($"Max frame size: {Wire.MaxFrame} bytes ({Wire.MaxFrame / 1024 / 1024} MiB)");

// Any length > MaxFrame will throw on ReadFrameAsync

Version Negotiation

The protocol includes a versioning mechanism to detect incompatibilities between client and server builds.

HoloProtocol.Version constant

Purpose: The protocol version this build speaks (currently 1).

using HoloDb.Protocol;

Console.WriteLine($"Protocol version: {HoloProtocol.Version}");

HoloProtocol.MinSupportedVersion constant

Purpose: The oldest peer version this build will accept. Today equals Version (exact match required). Lower it only when a specific older wire shape is proven decodable by the current code.

using HoloDb.Protocol;

Console.WriteLine($"Supported range: [{HoloProtocol.MinSupportedVersion}, {HoloProtocol.Version}]");

HoloProtocol.IsVersionSupported(byte peerVersion) — Check compatibility

Purpose: Returns true if peerVersion falls within [MinSupportedVersion, Version].

Example (client-side):

using HoloDb.Protocol;

byte serverVersion = 1;
if (HoloProtocol.IsVersionSupported(serverVersion))
    Console.WriteLine("Compatible");
else
    Console.WriteLine("Incompatible");

Example (server-side decision):

using HoloDb.Protocol;

// After decoding a client Hello
HoloProtocol.Request req = HoloProtocol.DecodeRequest(helloPayload);

if (HoloProtocol.IsVersionSupported(req.Version))
{
    // Send HelloAck
    byte[] ack = HoloProtocol.HelloAck("MyServer/1.0");
    await Wire.WriteFrameAsync(stream, ack, ct);
}
else
{
    // Send VersionMismatch
    byte[] mismatch = HoloProtocol.VersionMismatch(req.Version);
    await Wire.WriteFrameAsync(stream, mismatch, ct);
}

Request Encoding (Client → Server)

HoloProtocol.Hello(string clientInfo, string token) — Handshake

Purpose: Encode a Hello request with client identification and authentication token.

Wire format:

• Opcode: 0x01 (Hello)
• Byte: protocol version
• String: client info (length-prefixed UTF-8)
• String: auth token, typically empty or presented over TLS (length-prefixed UTF-8)

Example:

using HoloDb.Protocol;

byte[] hello = HoloProtocol.Hello("MyApp/2.1", "secret-token");
Console.WriteLine($"Hello payload: {hello.Length} bytes");
// First byte is 0x01 (Hello opcode)

HoloProtocol.Exec(string sql) — Execute SQL

Purpose: Encode an SQL statement (query, mutation, or transaction control).

Wire format:

• Opcode: 0x10 (Exec)
• String: SQL text (length-prefixed UTF-8)

Example:

using HoloDb.Protocol;

byte[] query = HoloProtocol.Exec("SELECT COUNT(*) FROM users");
byte[] insert = HoloProtocol.Exec("INSERT INTO users (name) VALUES ('Alice')");
byte[] begin = HoloProtocol.Exec("BEGIN");
byte[] commit = HoloProtocol.Exec("COMMIT");
byte[] rollback = HoloProtocol.Exec("ROLLBACK");

Console.WriteLine($"Query: {query.Length} bytes");
Console.WriteLine($"Insert: {insert.Length} bytes");

HoloProtocol.BulkLoad(string table, IReadOnlyList<string> cols, IReadOnlyList<Array> data, int count) — Load typed arrays

Purpose: Fast load of multiple typed column arrays into a table in a single atomic operation.

Wire format:

• Opcode: 0x11 (BulkLoad)
• String: table name (length-prefixed UTF-8)
• Int32: row count
• Int32: column count
• For each column:
  • String: column name (length-prefixed UTF-8)
  • Byte: column type tag (0=long, 1=double, 2=string)
  • Type-specific data (see details below)

Column encoding:

• Long column: tag 0x00, then count × Int64 values (little-endian)
• Double column: tag 0x01, then count × float64 values (IEEE 754, little-endian)
• String column: tag 0x02, then count × length-prefixed UTF-8 strings. A null element is refused with an ArgumentException naming the column and row (BulkLoad has no NULL mask). Send NULL rows through Exec with an INSERT that writes NULL for that column.

Example:

using HoloDb.Protocol;

// Create three columns of data
var ids = new long[] { 1, 2, 3 };
var prices = new double[] { 9.99, 19.99, 29.99 };
var names = new string[] { "Widget", "Gadget", "Doohickey" };

byte[] bulkLoad = HoloProtocol.BulkLoad(
    "products",
    new[] { "id", "price", "name" },
    new Array[] { ids, prices, names },
    count: 3
);

Console.WriteLine($"BulkLoad frame: {bulkLoad.Length} bytes");

Validation (client-side, before bytes are sent):

• If count < 0: throws ArgumentOutOfRangeException
• If data.Count != cols.Count: throws ArgumentException
• If any column array is shorter than count: throws ArgumentException

HoloProtocol.Ping() — Keep-alive

Purpose: Encode a keep-alive probe.

Wire format:

• Opcode: 0x1F (Ping)

Example:

using HoloDb.Protocol;

byte[] ping = HoloProtocol.Ping();
// Just 1 byte: [0x1F]

HoloProtocol.StatsRequest() — Request server statistics

Purpose: Encode a stats request.

Wire format:

• Opcode: 0x1E (Stats)

Example:

using HoloDb.Protocol;

byte[] statsReq = HoloProtocol.StatsRequest();
// Just 1 byte: [0x1E]

Response Decoding (Client)

HoloProtocol.DecodeHelloAck(byte[] payload) — Decode server handshake

Purpose: Decode a server Hello response. Validates version negotiation. Returns server info string.

Throws:

• HoloVersionMismatchException if server version is out of range or VersionMismatch opcode received
• HoloRemoteException if server sent an error (opcode 0x82)
• HoloProtocolException if payload is malformed

Wire format (HelloAck):

• Opcode: 0x83 (HelloAck)
• Byte: server's protocol version
• String: server info (length-prefixed UTF-8)

Wire format (VersionMismatch rejection):

• Opcode: 0x86 (VersionMismatch)
• Byte: server's protocol version
• Byte: server's minimum supported version
• Byte: client version (echoed back for logging)

Example:

using HoloDb.Protocol;
using System.IO;

var stream = new MemoryStream();

// Simulate server sending HelloAck
byte[] serverPayload = HoloProtocol.HelloAck("MyServer/1.0");
await Wire.WriteFrameAsync(stream, serverPayload, CancellationToken.None);

// Client decodes it
stream.Position = 0;
byte[] frame = await Wire.ReadFrameAsync(stream, CancellationToken.None);

try
{
    string serverInfo = HoloProtocol.DecodeHelloAck(frame);
    Console.WriteLine($"Server: {serverInfo}");
}
catch (HoloVersionMismatchException ex)
{
    Console.WriteLine($"Version mismatch: client v{ex.ClientVersion}, server v{ex.ServerVersion}");
}

HoloProtocol.DecodeResult(byte[] payload) — Decode a query response

Purpose: Decode a server query response into a QueryResult. Handles three result shapes: columnar arrays (fast), materialised dict rows (flexible), or affected row count (mutation).

Throws:

• HoloRemoteException if server sent an error
• HoloProtocolException if payload is malformed

Wire format (Columnar):

• Opcode: 0x81 (Columnar)
• Int32: row count
• Int32: column count
• For each column:
  • String: column name
  • (Byte tag + typed array data — see BulkLoad column encoding)

Wire format (Rows — materialised dict):

• Opcode: 0x84 (Rows)
• Int32: row count
• Int32: column count
• For each column: String column name
• For each row:
  • For each column: tagged value (tag=0 long, 1 double, 2 string, 3 SQL NULL; then data unless NULL)
  • A NULL cell decodes as DBNull.Value in Rows[i][col]; QueryResult.IsNull(row, col) is true for it. A NULL is never sent in columnar form.

Wire format (Affected — mutations):

• Opcode: 0x80 (Affected)
• Int32: affected row count

Example:

using HoloDb.Protocol;
using System.IO;

// Simulate server sending a columnar result
var ids = new long[] { 1, 2, 3 };
var names = new string[] { "Alice", "Bob", "Charlie" };
byte[] serverPayload = HoloProtocol.Result(
    HoloDb.QueryResult.Columnar(
        new[] { "id", "name" },
        new Array[] { ids, names },
        rowCount: 3
    )
);

// Client receives and decodes it
var stream = new MemoryStream();
await Wire.WriteFrameAsync(stream, serverPayload, CancellationToken.None);
stream.Position = 0;
byte[] frame = await Wire.ReadFrameAsync(stream, CancellationToken.None);

var result = HoloProtocol.DecodeResult(frame);
Console.WriteLine($"Rows: {result.RowCount}, Columns: {result.Columns.Count}");

for (int r = 0; r < result.RowCount; r++)
{
    long id = result.GetLong(r, 0);       // No boxing
    string name = result.GetText(r, 1);
    Console.WriteLine($"  {id}: {name}");
}

HoloProtocol.IsPong(byte[] payload) — Check for Pong

Purpose: Returns true if payload is a Pong response (opcode 0x8F).

Example:

using HoloDb.Protocol;

byte[] response = HoloProtocol.Pong();
if (HoloProtocol.IsPong(response))
    Console.WriteLine("Received Pong");

HoloProtocol.DecodeStats(byte[] payload) — Decode statistics

Purpose: Decode a Stats response. Throws on error or malformed payload.

Returns: Tuple of (serverInfo string, counters Dictionary<string, long>)

Wire format:

• Opcode: 0x85 (Stats)
• String: server info
• Int32: counter count
• For each counter:
  • String: counter name
  • Int64: counter value

Example:

using HoloDb.Protocol;

byte[] statsPayload = HoloProtocol.StatsResponse(
    "MyServer/1.0",
    new[] { ("queries", 1000L), ("writes", 500L) }
);

var (info, counters) = HoloProtocol.DecodeStats(statsPayload);
Console.WriteLine($"Server: {info}");
foreach (var (name, value) in counters)
    Console.WriteLine($"  {name}: {value}");

Request Decoding (Server)

HoloProtocol.DecodeRequest(byte[] payload) — Decode client request

Purpose: Decode any client request (Hello, Exec, BulkLoad, Ping, Stats) into a unified Request object. Server-side only.

Returns: HoloProtocol.Request object with fields populated per opcode.

Throws:

• HoloProtocolException for malformed/unknown opcode

Example (server dispatch loop):

using HoloDb.Protocol;
using System.IO;

var stream = new MemoryStream();

// Client sends a Hello
byte[] clientHello = HoloProtocol.Hello("client/1.0", "token");
await Wire.WriteFrameAsync(stream, clientHello, CancellationToken.None);

// Server receives and decodes
stream.Position = 0;
byte[] frame = await Wire.ReadFrameAsync(stream, CancellationToken.None);
var req = HoloProtocol.DecodeRequest(frame);

Console.WriteLine($"Request opcode: {req.Op}");
if (req.Op == WireOp.Hello)
{
    Console.WriteLine($"Client: {req.ClientInfo}, version: {req.Version}");
    
    if (HoloProtocol.IsVersionSupported(req.Version))
    {
        byte[] ack = HoloProtocol.HelloAck("server/1.0");
        await Wire.WriteFrameAsync(stream, ack, CancellationToken.None);
    }
    else
    {
        byte[] mismatch = HoloProtocol.VersionMismatch(req.Version);
        await Wire.WriteFrameAsync(stream, mismatch, CancellationToken.None);
    }
}

HoloProtocol.Request class — Decoded request fields

Purpose: Holds the decoded fields of any client request.

Properties:

• WireOp Op — the opcode (Hello, Exec, BulkLoad, Ping, Stats)
• byte Version — protocol version (Hello only, else 0)
• string ClientInfo — client identification (Hello only, else "")
• string Token — auth token (Hello only, else "")
• string Sql — SQL text (Exec only, else "")
• string Table — table name (BulkLoad only, else "")
• string[] Cols — column names (BulkLoad only, else empty array)
• Array[] Data — column arrays (BulkLoad only, else empty array). Each is long[], double[], or string[].
• int Count — row count (BulkLoad only, else 0)

Response Encoding (Server)

HoloProtocol.HelloAck(string serverInfo) — Acknowledge handshake

Purpose: Encode a successful Hello response.

Wire format:

• Opcode: 0x83 (HelloAck)
• Byte: server's protocol version
• String: server info (length-prefixed UTF-8)

Example:

using HoloDb.Protocol;

byte[] ack = HoloProtocol.HelloAck("MyServer/1.0");
Console.WriteLine($"HelloAck frame: {ack.Length} bytes");

HoloProtocol.VersionMismatch(byte clientVersion) — Reject incompatible version

Purpose: Encode a version rejection in response to an incompatible client Hello.

Wire format:

• Opcode: 0x86 (VersionMismatch)
• Byte: server's protocol version
• Byte: server's minimum supported version
• Byte: client version (echoed back for logging)

Example:

using HoloDb.Protocol;

// Server received a Hello with an unsupported version
byte clientVersion = 99;
byte[] rejection = HoloProtocol.VersionMismatch(clientVersion);
Console.WriteLine($"Sent VersionMismatch to client v{clientVersion}");

HoloProtocol.Affected(int n) — Encode mutation result

Purpose: Encode a result for INSERT, UPDATE, DELETE, or other mutation that only reports row count.

Wire format:

• Opcode: 0x80 (Affected)
• Int32: row count (little-endian)

Example:

using HoloDb.Protocol;

byte[] result = HoloProtocol.Affected(42);
Console.WriteLine($"Affected frame: {result.Length} bytes");

HoloProtocol.Error(string message) — Encode error response

Purpose: Encode an error response for any request. Client receives this as HoloRemoteException.

Wire format:

• Opcode: 0x82 (Error)
• String: error message (length-prefixed UTF-8)

Example:

using HoloDb.Protocol;

byte[] errorResp = HoloProtocol.Error("table 'users' not found");
Console.WriteLine($"Error frame: {errorResp.Length} bytes");

HoloProtocol.Pong() — Respond to Ping

Purpose: Encode a Pong keep-alive response.

Wire format:

• Opcode: 0x8F (Pong)

Example:

using HoloDb.Protocol;

byte[] pong = HoloProtocol.Pong();
// Just 1 byte: [0x8F]

HoloProtocol.Result(QueryResult q) — Encode query result

Purpose: Encode a QueryResult from the HoloDb engine. Automatically selects the best wire shape: columnar arrays (fast), materialised dict rows (flexible), or affected-only count (mutation).

Encoding rules:

• If QueryResult.IsColumnar: encode as Columnar (opcode 0x81) with typed arrays — the fast path.
• Else if result has columns (dict rows): encode as Rows (opcode 0x84) with materialised dicts and value tags.
• Else: encode as Affected (opcode 0x80) with row count only.

Example:

using HoloDb.Protocol;

// Columnar result (fast path)
var ids = new long[] { 1, 2 };
var names = new string[] { "Alice", "Bob" };
var colResult = HoloDb.QueryResult.Columnar(
    new[] { "id", "name" },
    new Array[] { ids, names },
    rowCount: 2
);
byte[] encoded = HoloProtocol.Result(colResult);
Console.WriteLine($"Columnar result frame: {encoded.Length} bytes");

// Materialised result (dict rows)
var rows = new List<Dictionary<string, object>>
{
    new() { { "id", 1L }, { "name", "Alice" } },
    new() { { "id", 2L }, { "name", "Bob" } }
};
var rowResult = HoloDb.QueryResult.FromRows(rows, new[] { "id", "name" });
encoded = HoloProtocol.Result(rowResult);
Console.WriteLine($"Rows result frame: {encoded.Length} bytes");

// Mutation result (affected count only)
var affectedResult = HoloDb.QueryResult.Ok(affected: 5);
encoded = HoloProtocol.Result(affectedResult);
Console.WriteLine($"Affected result frame: {encoded.Length} bytes");

HoloProtocol.StatsResponse(string serverInfo, IReadOnlyList<(string name, long value)> counters) — Encode statistics

Purpose: Encode a Stats response.

Wire format:

• Opcode: 0x85 (Stats)
• String: server info (length-prefixed UTF-8)
• Int32: counter count
• For each counter: String name + Int64 value

Example:

using HoloDb.Protocol;

var counters = new[] { ("queries", 1000L), ("writes", 500L) };
byte[] statsResp = HoloProtocol.StatsResponse("MyServer/1.0", counters);
Console.WriteLine($"Stats frame: {statsResp.Length} bytes");

Exception Types

HoloProtocolException — Malformed frame

Thrown when the wire framing or payload is malformed, truncated, or hostile (e.g., bad opcode, truncated length prefix, oversized length claim).

When to catch: Security boundary. Indicates a bug or attack, not a normal error condition. Log it, close the connection, and do not retry automatically.

Example:

using HoloDb.Protocol;

try
{
    byte[] badPayload = new byte[] { 0xFF, 0xFF }; // bad opcode
    HoloProtocol.DecodeResult(badPayload);
}
catch (HoloProtocolException ex)
{
    Console.WriteLine($"Protocol error: {ex.Message}");
}

HoloRemoteException — Server-side error

Thrown when the server returns a well-formed Error response (opcode 0x82). The message is the server's exception text.

When to catch: Application logic. The server rejected the request (bad SQL, auth failure, constraint violation, etc.). The message is for the user.

Example:

using HoloDb.Protocol;

byte[] errorPayload = HoloProtocol.Error("duplicate primary key");

try
{
    HoloProtocol.DecodeResult(errorPayload);
}
catch (HoloRemoteException ex)
{
    Console.WriteLine($"Server error: {ex.Message}");
}

HoloVersionMismatchException — Protocol version skew

Thrown when the client and server speak incompatible protocol versions. A clean, actionable error distinct from both HoloProtocolException (wire corruption/attack) and HoloRemoteException (application-level rejection).

Properties:

• byte ClientVersion — this client's protocol version
• byte ServerVersion — server's protocol version (from HelloAck or VersionMismatch opcode)

When to catch: Deployment issue. Surface to an operator ("client vX vs server vY, upgrade/downgrade one side") rather than blindly retry, which will fail identically every time.

Example:

using HoloDb.Protocol;

byte[] mismatchPayload = HoloProtocol.VersionMismatch(clientVersion: 1);

try
{
    HoloProtocol.DecodeHelloAck(mismatchPayload);
}
catch (HoloVersionMismatchException ex)
{
    Console.WriteLine(
        $"Version incompatibility: " +
        $"client speaks v{ex.ClientVersion}, " +
        $"server speaks v{ex.ServerVersion}"
    );
}

Wire Format Reference

Frame Structure (Transport Layer)

Every frame sent over the wire has this structure:

[0-3]   Int32 (LE): Payload length in bytes (0 to MaxFrame)
[4+]    Bytes: Payload (payload[0] = opcode)

Example (Hello with client info "test", token "secret"):

Length:   [0x21, 0x00, 0x00, 0x00]  // 33 bytes in little-endian
Opcode:   [0x01]                     // Hello
Version:  [0x01]                     // version 1
Info:     [0x04, 0x74, 0x65, 0x73, 0x74]  // length-prefixed UTF-8 "test"
Token:    [0x06, 0x73, 0x65, 0x63, 0x72, 0x65, 0x74]  // length-prefixed UTF-8 "secret"

Length-Prefixed Strings

Strings are encoded as UTF-8 with a length prefix (following BinaryWriter convention):

• For strings up to 16383 bytes: 1-byte length prefix (0x00–0x7F for short strings)
• For longer strings: 2-byte or 3-byte length with 7-bit encoding

This is handled transparently by BinaryWriter and BinaryReader.

Integer Encoding

All integers are little-endian (standard for .NET):

• Int32 (4 bytes): row count, column count, affected rows, counter count
• Int64 (8 bytes): long column values, counter values
• double (8 bytes, IEEE 754): double column values

Column Arrays (Bulk Data)

In BulkLoad and result messages, columns are encoded with a type tag and raw data:

[Type Tag]     Byte: 0=long, 1=double, 2=string
[Element 0]    Type-specific bytes
[Element 1]    Type-specific bytes
...
[Element N-1]  Type-specific bytes

For long[] at count=3, value [100, 200, 300]:

[0x00]                           // tag = long
[0x64, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]  // 100 in LE Int64
[0xC8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]  // 200 in LE Int64
[0x2C, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]  // 300 in LE Int64

Materialised Dict Rows

When results can't be columnar (e.g., JOIN/GROUP BY), the server sends rows as tagged values:

[Value 1, Col 1]  Tag byte + data (0=long+8B, 1=double+8B, 2=string+len+UTF8, 3=null)
[Value 1, Col 2]  Tag byte + data
...
[Value 2, Col 1]  Tag byte + data
...

For a row {id: 1, name: "Alice", email: null}:

[0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]  // tag=long + 1
[0x02, 0x05, 0x41, 0x6C, 0x69, 0x63, 0x65]              // tag=string + len 5 + "Alice"
[0x03]                                                    // tag=null (SQL NULL, no data) -> DBNull.Value

Gotchas & Constraints

Endianness

All integers and floats are little-endian (standard for .NET). Cross-language clients (e.g., Java, Go, Python) must byte-swap on big-endian platforms. Both ends are .NET today, so this is transparent.

Max Frame Size

Wire.MaxFrame = 256 MiB. Any frame larger than this raises HoloProtocolException. Design a query/BulkLoad to produce smaller responses (e.g., paginate, use LIMIT), or split bulk loads into batches.

Partial Reads

Wire.ReadFrameAsync reads the ENTIRE frame, including the length prefix. If the stream is interrupted mid-frame, it throws HoloProtocolException("unexpected end of stream mid-frame"). Partial reads are not exposed; the caller must reconnect and retry.

Bounded Allocation

Server-side decode (DecodeRequest for BulkLoad) uses bounds checking to prevent a tiny hostile frame from claiming billions of rows or columns. Declared count vs actual payload bytes is checked before any new statement.

String Encoding

All strings are UTF-8. The empty string "" is a text value (tag 2); SQL NULL is its own tag (3, no payload) and decodes as DBNull.Value, never as "".

Transaction Semantics

Transactions are NOT separate opcodes. BEGIN, COMMIT, and ROLLBACK are sent as ordinary Exec SQL strings. The server handles transaction affinity per connection.

Column Types

Only three column types cross the wire:

• Long (long[], INT in SQL) — 8 bytes, signed, little-endian Int64
• Double (double[], REAL in SQL) — 8 bytes, IEEE 754 double, little-endian
• String (string[], TEXT in SQL) — length-prefixed UTF-8

No other types (BLOB, DATE, BOOLEAN, etc.) are supported. Custom types must be serialized to string or integer.

Request/Response Pairing

There is no request ID or multiplexing. One client connection = one logical sequence: send a request, wait for a response, repeat. Pipelining multiple requests without reading responses is not supported; responses would be corrupted by this serialization assumption.

Bulk Load Atomicity

A BulkLoad either fully completes or not at all (atomic). If it fails (e.g., duplicate key), the entire load is rolled back and an Error response is sent.

No Compression

Frames are sent raw. Compression (if needed) is a transport/application concern outside this protocol.

Obsolete / Experimental Features

None currently. The protocol is stable (v1, frozen wire format).

---

Version: Check the live NuGet listing for the exact number — this file is a secondary reference, not the source of truth; the package's own <Version> in HoloDb.Protocol.csproj is.

Last updated: 2026-10-02

License

Proprietary — this package ships the compiled library only; see LICENSE for the full terms. Installing and using unmodified copies for internal evaluation and non-commercial development or testing is licensed; commercial, production, or revenue-generating use requires a separate commercial licence from the owner. No feature on this page is gated behind a tier or a licence key — every capability described works identically with or without one. Copyright (c) 2026 Dongyang Stephen Chen, trading as Evaluated Applications. All rights reserved.