MiniBson.Source 2.0.0

MiniBson

MiniBson is a small BSON library for .NET. It has a forward-only reader, a forward-only writer, and source-generated serialization. It uses no reflection at run time. The runtime library works with trimming and Native AOT.

Features

• Source-generated serialization for your models
• Low-level BsonReader and BsonWriter APIs
• Writes to any IBufferWriter<byte>, and reads from a ReadOnlySpan<byte>, a ReadOnlyMemory<byte>, or a ReadOnlySequence<byte>
• No Stream anywhere in the API, so a PipeWriter or a PipeReader needs no adapter
• No reflection at run time
• netstandard2.0 and net8.0 targets
• No dependency on net8.0, and only System.Memory on netstandard2.0
• An assembly NuGet package and a source-only NuGet package

Installation

For most applications, use the regular package:

dotnet add package MiniBson

To compile MiniBson into your own assembly, use the source-only package:

dotnet add package MiniBson.Source

The source-only package makes the MiniBson types internal. Thus they do not become part of your public API, and they do not collide with a second copy in a different assembly. If you need public types, set MiniBsonPublic to true.

Both packages include the source generator.

Source-generated serialization

Declare a partial context. Register each type that can be a top-level value:

using MiniBson;

public sealed class Person
{
    public string Name { get; set; } = string.Empty;
    public int Age { get; set; }
    public string[] Tags { get; set; } = [];
}

[BsonSerializable(typeof(Person))]
public partial class AppBsonContext
{
}

The context uses a BsonReader or a BsonWriter. You keep the ownership of the buffers:

var context = new AppBsonContext();
var original = new Person
{
    Name = "Ada",
    Age = 37,
    Tags = ["compiler", "math"]
};

var output = new ArrayBufferWriter<byte>(context.GetSerializedSize(original));
context.Serialize(original, new BsonWriter(output));

var reader = new BsonReader(output.WrittenSpan);
var copy = (Person?)context.Deserialize(ref reader, typeof(Person));

The destination is any IBufferWriter<byte>, including ArrayBufferWriter<byte> and PipeWriter. On netstandard2.0, where System.Buffers.ArrayBufferWriter<T> does not exist, MiniBson supplies MiniBson.Polyfills.ArrayBufferWriter<T> with the same shape.

Each context has these methods:

void Serialize(object input, BsonWriter writer);
object? Deserialize(ref BsonReader reader, Type type);
int GetSerializedSize(object input);

Deserialize takes the reader by reference, because BsonReader is a ref struct. A copy would read the same bytes a second time.

GetSerializedSize returns the number of bytes that Serialize writes, but it writes no bytes itself. Use it for these tasks:

• Allocate a buffer before you serialize.
• Put a length in front of a message before you send it.
• Reject a value that is too large, before you encode it.

var size = context.GetSerializedSize(person);
if (size > MaxMessageBytes)
    throw new InvalidOperationException($"{size} bytes exceeds the limit.");

WriteFrameHeader(pipe, size);
context.Serialize(person, new BsonWriter(pipe));

The number is exact. It is not an estimate. It is the same number that Serialize computes for itself, and the writer throws an exception if the two numbers disagree. This is true only if each property returns the same value two times. See Document lengths.

Give the number to the constructor of ArrayBufferWriter<byte>. It then allocates one buffer of that size and does not grow.

Model behavior

• The top-level dispatch uses the exact runtime type. Register each concrete type that you give to Serialize or Deserialize.
• MiniBson also writes code for the types of your properties. Such a type is not a valid top-level value until you register it.
• MiniBson serializes each public instance property that it can read, and it uses the C# name. It includes an inherited property. If a derived property hides a name, MiniBson uses the derived property.
• A deserializer matches the elements by name. It skips an element that it does not know. A property with no element keeps its default value.
• MiniBson writes an enum as a number. A new name for a member is safe, but a new number for a member changes the wire format.
• MiniBson writes a null reference as BSON Null and reads it back as null. The nullable annotation on the property does not change the wire format.

Document lengths

A BSON document starts with its total length, and an IBufferWriter<byte> does not return a byte that it gave out. Thus the generated code computes the length of each document before it starts that document. This is the same for every destination.

This costs one more walk of your object graph, and it adds one rule for your models: a property must return the same value two times. The measure pass and the write pass read the object graph separately. If a property gives a different value to each pass, the computed length is wrong. These properties are examples:

• A computed property that returns a new array each time.
• A property on an object that a different thread changes at the same time.

WriteEndDocument() finds the disagreement and throws an InvalidOperationException. It writes no bad document. This test runs on every write. Thus there is no destination where such a property passes without an error.

When the test throws, the destination can already hold some bytes. Discard them. Do not use them. ArrayBufferWriter<byte>.Clear() does this.

Supported model types

Model limitations

• MiniBson does not support a collection such as List<T> or Dictionary<TKey, TValue>. Use an array.
• MiniBson does not support a multidimensional array or a jagged array.
• MiniBson does not support decimal, because it has no Decimal128 mapping.
• A class that is not a record needs a parameterless constructor that MiniBson can use. Each property also needs a public set or init accessor.
• A record must be positional. Its constructor must accept each property in the generated order.
• A context must be a partial class. MiniBson ignores a context that is not partial, and it gives no diagnostic.

A property that MiniBson does not support gives the compiler error MINIBSON001. The error points at that property:

error MINIBSON001: MiniBson cannot serialize 'Order.Total': type 'decimal' is not supported

A different severity for the diagnostic does not add support. The generated code contains a fallback that throws a NotSupportedException. Thus such a property cannot give an empty value with no error.

Low-level reader and writer

Use the low-level API if you need direct control of the BSON document. Use it also if you do not want model types.

Write a document

Each document needs its length first. BsonSize computes it:

var tagsLength = BsonSize.ArrayOverhead(2)
    + BsonSize.String("compiler") + BsonSize.String("math");

var length = BsonSize.DocumentOverhead
    + BsonSize.Element("name") + BsonSize.String("Ada")
    + BsonSize.Element("age") + BsonSize.Int32
    + BsonSize.Element("active") + BsonSize.Boolean
    + BsonSize.Element("tags") + tagsLength;

var output = new ArrayBufferWriter<byte>(length);
var writer = new BsonWriter(output);

writer.WriteStartDocument(length);
writer.WriteString("name", "Ada");
writer.WriteInt32("age", 37);
writer.WriteBoolean("active", true);

writer.WriteStartArray("tags", tagsLength);
writer.WriteString("compiler");
writer.WriteString("math");
writer.WriteEndArray();

writer.WriteEndDocument();

byte[] bson = output.WrittenSpan.ToArray();

To avoid this arithmetic, use the source generator. See Source-generated serialization.

Read a document

var reader = new BsonReader(bson);
reader.ReadStartDocument();

while (reader.Read())
{
    switch (reader.CurrentName)
    {
        case "name":
            Console.WriteLine(reader.ReadString());
            break;
        case "age":
            Console.WriteLine(reader.ReadInt32());
            break;
        case "tags":
            reader.ReadStartArray();
            while (reader.Read())
            {
                Console.WriteLine(reader.ReadString());
            }
            reader.ReadEndDocument();
            break;
        default:
            reader.Skip();
            break;
    }
}

reader.ReadEndDocument();

ReadEndDocument() closes the current document or the current array.

The reader takes its input in four forms:

var reader = new BsonReader(bson);                  // byte[]
var reader = new BsonReader(memory);                // ReadOnlyMemory<byte>
var reader = new BsonReader(span);                  // ReadOnlySpan<byte>
var reader = new BsonReader(sequence);              // ReadOnlySequence<byte>, e.g. from a PipeReader

The full document must be in memory. With a PipeReader, read the four-byte length first, wait for that number of bytes, and give the reader that slice. BytesConsumed gives the end of the document, so you can slice at that point and read the next one.

BsonReader is a ref struct, the same as Utf8JsonReader. It cannot cross an await, a lambda cannot capture it, and a class cannot hold it in a field. Pass it as ref BsonReader.

Every failure that the input causes is an InvalidDataException: a truncated document, a length that cannot be correct, a name with no terminator, and a read of the wrong type for the current element. Thus one catch covers a parse of bytes that you do not trust. An InvalidOperationException means the calling code is wrong, such as a Read() before ReadStartDocument().

Supported BSON values

An array is a document on the wire. Thus ReadEndArray and ReadEndDocument are one method with two names. Use the name that agrees with your write code.

Skip() also accepts each deprecated type in the specification: Undefined, DBPointer, Symbol, JavaScriptWithScope, Decimal128, MinKey, and MaxKey. This is true even when there is no accessor for the value. Generated deserializers skip each element that they do not know. Thus a document with one of these types stays readable.

ReadBinary() returns a ReadOnlySpan<byte> that points into your input. ReadBinaryMemory() returns a ReadOnlyMemory<byte> slice of it. Neither makes a copy, except in two cases. A reader from a plain ReadOnlySpan<byte> has no memory behind it to slice, so ReadBinaryMemory copies there. A value that lies across two segments of a sequence also goes into a new array. ReadBinaryArray() always copies. Use it when the value must live longer than the input.

Document lengths and BsonSize

A BSON document starts with its total length. An IBufferWriter<byte> does not return a byte that it gave out, so the writer cannot write that length later. Thus each document needs its length at the start:

writer.WriteStartDocument(length);

WriteStartDocument(string, int), WriteStartArray(string, int), WriteStartNestedDocument(int), and WriteStartNestedArray(int) take a length in the same manner.

The length is the length of the complete document. It includes the four-byte prefix and the null byte at the end. BsonSize gives you the parts. Each helper there agrees with one writer method.

If your length does not agree with the bytes that the writer wrote, WriteEndDocument() throws an InvalidOperationException. It writes no bad document.

A generated serializer computes all of this for you. See Document lengths.

Output and buffering

BsonWriter holds a buffer from the destination and commits it with Advance. Two rules follow:

• Do not write to the same IBufferWriter<byte> yourself while a document is open. The writer holds a buffer from it.
• WriteEndDocument() on the top-level document commits each byte. Thus the destination always holds a complete document, and you need no call of your own. Use Flush() for a document that you do not finish.

The writer asks the destination for adjacent bytes only for a scalar or the digits of an array index. That is twelve bytes at the most. A longer value fills a buffer, commits it, and takes another one. A destination that gives one byte at a time works.

ArrayBufferWriter<byte> is the destination for a document that you keep in memory. Construct it with the number from GetSerializedSize, and it allocates one time and does not grow. Read the result from WrittenSpan or WrittenMemory, and call Clear() to reuse it. A later write makes a span or memory from an earlier call invalid.

How to contribute

See AGENTS.md for the build commands, the reader and writer invariants, the generator constraints, the testing rules, and the package commands.

Acknowledgments

Claude helped to make a large part of this project.

License

MiniBson uses the MIT License.