FourSer.Gen 0.0.272

A high-performance .NET 9 source generator that automatically creates serialization and deserialization code for binary data structures using attributes and conventions.

Overview

This project provides a compile-time source generator that creates efficient binary serialization methods for your classes and structs. It's designed for scenarios where you need fast, low-allocation binary serialization, such as network protocols, file formats, or inter-process communication.

Features

• High-Performance Serialization: Zero-allocation serialization and deserialization using Span<byte> and ReadOnlySpan<byte>.
• Compile-Time Code Generation: Eliminates runtime reflection, ensuring maximum performance.
• Wide Type Support: Supports all primitive types, strings, and a comprehensive range of collection types.
• Nested Objects: Automatically handles serialization of complex object graphs with nested classes and structs.
• Flexible Collection Handling:
  • Custom Count Prefixes: Specify the integer type used for collection counts (e.g., byte, ushort, int).
  • Fixed-Size Collections: Serialize collections with a constant number of elements without a count prefix.
  • Dynamic Count References: Link a collection's count to another property in the class.
  • Unlimited Collections: Serialize collections that consume the rest of the data stream.
• Advanced Polymorphic Serialization:
  • Automatic Type Inference: The generator can automatically handle type discriminators without needing a TypeId property in your model.
  • Explicit Type Discriminators: Link polymorphism to a property in your model.
  • Custom Discriminator Types: Use byte, ushort, long, or enums for type discriminators to save space.
  • Polymorphic Collections: Serialize collections of different types, either with a single type discriminator for the whole collection or individual discriminators for each element.
• Easy to Use: Simply add attributes to your data structures to enable serialization.

Limitations

The source generator keeps a strict wire contract:

• Nested generated reference members and collection items are required on the wire.
• string and dynamic-count collections keep canonical null behavior: null serializes as empty or zero-count and deserializes back as empty.
• Pure IEnumerable<T> members are replayable-only across separate GetPacketSize(obj) and Serialize(obj, ...) calls. Each generated method enumerates at most once internally, but one-shot enumerables are not cached or mutated across calls.

For the full contract rationale, see docs/design-choices.md.

Quick Start

1. Install the NuGet Package

First, add the FourSer.Gen package to your project:

dotnet add package FourSer.Gen

2. Define Your Data Structures

Create a partial class or struct and add the [GenerateSerializer] attribute. All properties and fields will be automatically included in the serialization.

// In your project, e.g., in a file named "Packets.cs"
using FourSer.Contracts;

[GenerateSerializer]
public partial class Player
{
    public uint Id { get; set; }
    public string Username { get; set; }
    public float Health { get; set; }
}

[GenerateSerializer]
public partial class GameState
{
    public long GameId { get; set; }

    [SerializeCollection] // This attribute is needed for collections
    public List<Player> Players { get; set; }
}

3. Use the Generated Methods

The source generator creates static GetPacketSize, Serialize, and Deserialize methods on your types.

// Create an instance of your data structure
var state = new GameState
{
    GameId = 98765,
    Players = new List<Player>
    {
        new() { Id = 1, Username = "Hero", Health = 100.0f },
        new() { Id = 2, Username = "Villain", Health = 85.5f }
    }
};

// 1. Get the required buffer size
int size = GameState.GetPacketSize(state);

// 2. Serialize the object into a buffer
var buffer = new byte[size];
var span = new Span<byte>(buffer);
GameState.Serialize(state, span);

// The buffer now contains the binary representation of your object
// You can now send it over the network, save it to a file, etc.

// 3. Deserialize the object from the buffer
var readSpan = new ReadOnlySpan<byte>(buffer);
var deserializedState = GameState.Deserialize(readSpan);

// Now you have a deep copy of the original object
Console.WriteLine($"Game ID: {deserializedState.GameId}");
foreach (var player in deserializedState.Players)
{
    Console.WriteLine($"- Player: {player.Username}, Health: {player.Health}");
}

Generated Interface

Each class marked with [GenerateSerializer] implements ISerializable<T>. This interface provides the core methods for serialization and deserialization.

public interface ISerializable<T> where T : ISerializable<T>
{
    // Calculates the total size in bytes required to serialize the object.
    static abstract int GetPacketSize(T obj);

    // Serializes the object into the provided span, advancing the span.
    static abstract void Serialize(T obj, ref Span<byte> data);

    // Serializes the object into the provided span.
    static abstract void Serialize(T obj, Span<byte> data);

    // Serializes the object into the provided stream.
    static abstract void Serialize(T obj, Stream stream);

    // Deserializes an object from the provided span.
    // The span is advanced by the number of bytes read.
    static abstract T Deserialize(ref ReadOnlySpan<byte> data);

    // Deserializes an object from the provided span without advancing it.
    static abstract T Deserialize(ReadOnlySpan<byte> data);

    // Deserializes an object from the provided stream.
    static abstract T Deserialize(Stream stream);
}

Collection Serialization

The generator provides powerful options for serializing collections using the [SerializeCollection] attribute.

Controlling the Count Prefix

By default, the generator prefixes a collection with an int (4 bytes) to store the number of elements. You can customize this behavior.

1. Custom Count Type

Use the CountType property to specify a different integer type for the count prefix. This is useful for optimizing space.

[GenerateSerializer]
public partial class MyPacket
{
    // Use ushort (2 bytes) for the count prefix instead of the default int (4 bytes).
    [SerializeCollection(CountType = typeof(ushort))]
    public List<int> Numbers { get; set; } = new();
}

2. Fixed-Size Collections

If the collection always has a fixed number of elements, use CountSize. This completely removes the count prefix from the binary data, saving space and improving performance.

[GenerateSerializer]
public partial class MyPacket
{
    // Always serialize exactly 16 bytes. No count is written to the stream.
    // If the collection has fewer than 16 items, an exception will be thrown.
    [SerializeCollection(CountSize = 16)]
    public byte[] Data { get; set; } = new byte[16];
}

3. Dynamic Count Reference

If the collection's count is stored in another property, use CountSizeReference to link to it. This is common in protocols where a field specifies the length of a subsequent list.

[GenerateSerializer]
public partial class MyPacket
{
    public byte NameLength { get; set; }

    [SerializeCollection(CountSizeReference = nameof(NameLength))]
    public List<char> Name { get; set; }
}

During serialization, the generator writes the actual count derived from Name. During deserialization, the count read from the wire is assigned back to NameLength.

4. Unlimited Collections

For collections that should be serialized until the end of the data stream, use the Unlimited property. This is useful for top-level objects or when the length is implicitly known.

[GenerateSerializer]
public partial class MyPacket
{
    public int SomeHeader { get; set; }

    [SerializeCollection(Unlimited = true)]
    public List<byte> Payload { get; set; }
}

IMemoryOwner<T> (Pooled Buffers)

FourSer.Gen supports System.Buffers.IMemoryOwner<T> as a collection type when used with [SerializeCollection]. This enables zero-copy deserialization with pooled memory, reducing GC pressure for high-throughput scenarios.

How It Works

• Serialization: The buffer contents are serialized via owner.Memory.Span.
• Deserialization: The generator rents a buffer from MemoryPool<T>.Shared and slices it to the requested length.
• Length Handling: The count is taken from owner.Memory.Length, and all standard count options (CountType, CountSize, CountSizeReference) are supported.

Basic Example

using System.Buffers;
using FourSer.Contracts;

[GenerateSerializer]
public partial class BinaryPayload
{
    public long Size { get; set; }

    [SerializeCollection(CountSizeReference = nameof(Size))]
    public IMemoryOwner<byte> Data { get; set; }
}

Automatic IDisposable Generation

When a type contains IMemoryOwner<T> properties (directly or via nested types), FourSer.Gen automatically generates IDisposable implementation to ensure proper cleanup of pooled memory:

[GenerateSerializer]
public partial class PooledPacket
{
    [SerializeCollection(CountType = typeof(ushort))]
    public IMemoryOwner<byte> Payload { get; set; }
}
// PooledPacket automatically implements IDisposable
// and disposes Payload when Dispose() is called.

// Wrapper types also get IDisposable:
[GenerateSerializer]
public partial class PacketWrapper
{
    public PooledPacket Inner { get; set; }
}
// PacketWrapper implements IDisposable and disposes Inner.

// Collections of disposable types are also handled:
[GenerateSerializer]
public partial class PacketCollection
{
    [SerializeCollection]
    public List<PooledPacket> Packets { get; set; }
}
// PacketCollection implements IDisposable and disposes each packet in Packets.

Important: Always dispose types containing IMemoryOwner<T> to return buffers to the pool.

Limitations

• Unlimited = true is not supported for IMemoryOwner<T> (the pool requires a known size).
• The requested length must fit into int (due to MemoryPool<T>.Rent(int) signature).
• If you provide your own IDisposable implementation, the generator will not override it.

Polymorphic Collections

The generator supports serializing collections of polymorphic types. This is useful when a list can contain objects of different derived types.

1. Homogeneous Polymorphic Collections (SingleTypeId)

If all elements in the collection are of the same derived type, you can use PolymorphicMode.SingleTypeId. A single type discriminator is written once for the entire collection.

[GenerateSerializer]
public partial class Scene
{
    public byte EntityType { get; set; } // Restored from the wire on deserialize

    [SerializeCollection(PolymorphicMode = PolymorphicMode.SingleTypeId, TypeIdProperty = nameof(EntityType))]
    [PolymorphicOption((byte)1, typeof(Player))]
    [PolymorphicOption((byte)2, typeof(Monster))]
    public List<Entity> Entities { get; set; }
}

During serialization, the discriminator is derived from the actual contents of Entities, not from the current EntityType value. During deserialization, the discriminator read from the wire is assigned back to EntityType.

2. Heterogeneous Polymorphic Collections (IndividualTypeIds)

If the elements in the collection can be of different derived types, use PolymorphicMode.IndividualTypeIds. Each element is prefixed with its own type discriminator.

[GenerateSerializer]
public partial class Inventory
{
    // Each item in the list will have its own type ID (byte) written before it.
    [SerializeCollection(PolymorphicMode = PolymorphicMode.IndividualTypeIds, TypeIdType = typeof(byte))]
    [PolymorphicOption((byte)10, typeof(Sword))]
    [PolymorphicOption((byte)20, typeof(Shield))]
    [PolymorphicOption((byte)30, typeof(Potion))]
    public List<Item> Items { get; set; }
}

Nested Objects

The generator automatically handles nested objects, as long as the nested types are also marked with [GenerateSerializer].

[GenerateSerializer]
public partial class ContainerPacket
{
    public int Id;
    public NestedData Data;
}

[GenerateSerializer]
public partial class NestedData
{
    public string Name;
    public float Value;
}

Polymorphic Serialization

The generator supports serializing fields and properties that can hold one of several different types, which is known as polymorphic serialization. This is configured using the [SerializePolymorphic] and [PolymorphicOption] attributes.

There are two main approaches to handle the type discriminator (the value that identifies which concrete type is being used).

Approach 1: Implicit Type Discriminator

In this approach, the type discriminator is written to and read from the binary stream, but it is not stored as a property in your model. This keeps your data models clean.

[GenerateSerializer]
public partial class AutoPolymorphicEntity
{
    public int Id { get; set; }
    
    // The type discriminator will be inferred automatically.
    [SerializePolymorphic]
    [PolymorphicOption(1, typeof(EntityType1))]
    [PolymorphicOption(2, typeof(EntityType2))]
    public BaseEntity Entity { get; set; }
}

• Serialization: The generator checks the actual type of Entity and writes the corresponding ID (1 or 2) to the stream.
• Deserialization: The generator reads the ID from the stream and creates an instance of the correct type (EntityType1 or EntityType2).

Approach 2: Explicit Type Discriminator

In this approach, the type discriminator is linked to a property in your model. The generated serializer writes the discriminator that matches the actual runtime value and restores the linked property from the wire during deserialization.

[GenerateSerializer]
public partial class PolymorphicEntity
{
    public int Id { get; set; }
    public int TypeId { get; set; } // The type discriminator property

    [SerializePolymorphic(nameof(TypeId))] // Link to the TypeId property
    [PolymorphicOption(1, typeof(EntityType1))]
    [PolymorphicOption(2, typeof(EntityType2))]
    public BaseEntity Entity { get; set; }
}

var entity = new PolymorphicEntity
{
    Id = 100,
    TypeId = 999, // Stale value; the wire discriminator is derived from Entity
    Entity = new EntityType1 { Name = "Test" }
};

// Serialization writes discriminator 1 for EntityType1.
var bytesWritten = PolymorphicEntity.Serialize(entity, buffer);

Customizing the Type Discriminator Type

To save space, you can change the underlying type of the type discriminator from the default int to a smaller type like byte or ushort, or even an enum. This is done using the TypeIdType property on the [SerializePolymorphic] attribute.

// Using byte (1 byte)
[SerializePolymorphic(TypeIdType = typeof(byte))]
[PolymorphicOption((byte)1, typeof(EntityType1))]
[PolymorphicOption((byte)2, typeof(EntityType2))]
public BaseEntity Entity { get; set; }

// Using a custom enum (backed by ushort)
public enum EntityType : ushort
{
    Type1 = 100,
    Type2 = 200
}

[SerializePolymorphic(TypeIdType = typeof(EntityType))]
[PolymorphicOption(EntityType.Type1, typeof(EntityType1))]
[PolymorphicOption(EntityType.Type2, typeof(EntityType2))]
public BaseEntity Entity { get; set; }

Using custom discriminator types offers several benefits:

• Space Efficiency: A byte uses 1 byte, a ushort uses 2, and an int uses 4. Choose the smallest type that fits your needs.
• Type Safety: Enums provide strong typing and make your code more readable and maintainable.
• Automatic Casting: The generator handles all necessary type conversions automatically.

Custom Serializers

For special cases where the default serialization logic is not sufficient, you can provide your own custom serializer for any given type. This is useful for handling legacy binary formats, complex data structures, or types that require special encoding.

1. Create a Custom Serializer

A custom serializer is a class that implements the ISerializer<T> interface, where T is the type you want to serialize.

public interface ISerializer<T>
{
    int GetPacketSize(T obj);
    int Serialize(T obj, Span<byte> data);
    void Serialize(T obj, Stream stream);
    T Deserialize(ref ReadOnlySpan<byte> data);
    T Deserialize(Stream stream);
}

All members are required. Generated stream serializers call Serialize(T, Stream) and Deserialize(Stream) directly; the generator does not bridge stream paths through the span-based members.

Here is an example of a custom serializer for handling MFC-style Unicode strings, which have a specific length prefix format:

public class MfcStringSerializer : ISerializer<string>
{
    public int GetPacketSize(string obj) { /* ... */ }
    public int Serialize(string obj, Span<byte> data) { /* ... */ }
    public void Serialize(string obj, Stream stream) { /* ... */ }
    public string Deserialize(ref ReadOnlySpan<byte> data) { /* ... */ }
    public string Deserialize(Stream stream) { /* ... */ }
}

2. Apply the Custom Serializer

You can apply a custom serializer in two ways:

On a Specific Property

Use the [Serializer(typeof(MySerializer))] attribute on a property to override its serialization logic.

[GenerateSerializer]
public partial class LegacyPacket
{
    public int PlayerId { get; set; }

    [Serializer(typeof(MfcStringSerializer))]
    public string PlayerName { get; set; }
}

In this example, PlayerName will be serialized using MfcStringSerializer, while PlayerId will use the default integer serialization.

As a Default for a Type

Use the [DefaultSerializer(typeof(TargetType), typeof(MySerializer))] attribute on a class to set a default serializer for all properties of a specific type within that class.

[GenerateSerializer]
[DefaultSerializer(typeof(string), typeof(MfcStringSerializer))]
public partial class AllMfcStringsPacket
{
    // This will use MfcStringSerializer by default
    public string PlayerName { get; set; }

    // This will also use MfcStringSerializer
    public string GuildName { get; set; }

    // You can still override the default if needed
    [Serializer(typeof(StandardStringSerializer))] // Assuming a standard one exists
    public string ChatMessage { get; set; }
}

This approach is useful when an entire class or data structure consistently uses a non-standard format for a certain type.

It also works on the assembly level by placing the attribute in any source file within the assembly:

[assembly: DefaultSerializer(typeof(string), typeof(MfcStringSerializer))]

Supported Types

Primitive Types

• byte, sbyte
• short, ushort
• int, uint
• long, ulong
• float, double
• bool
• string (UTF-8 encoded with length prefix)

Collections

The generator supports a wide range of collection types, where T can be any supported primitive, custom struct/class, or polymorphic type.

• List<T>
• T[] (Arrays)
• ICollection<T>
• IEnumerable<T> (replayable across separate GetPacketSize and Serialize calls only)
• IList<T>
• IReadOnlyCollection<T>
• IReadOnlyList<T>
• System.Buffers.IMemoryOwner<T> (with [SerializeCollection])
• System.Collections.ObjectModel.Collection<T>
• System.Collections.ObjectModel.ObservableCollection<T>
• System.Collections.Concurrent.ConcurrentBag<T>
• HashSet<T>
• Queue<T>
• Stack<T>
• LinkedList<T>
• SortedSet<T>
• ImmutableList<T>
• ImmutableArray<T>
• ImmutableHashSet<T>
• ImmutableQueue<T>
• ImmutableStack<T>
• ImmutableSortedSet<T>

Custom Types

• Any partial class or struct marked with [GenerateSerializer].
• Polymorphic types configured with [SerializePolymorphic] and [PolymorphicOption] attributes.
• enum types are serialized based on their underlying integer type.

Project Structure

src/
├── FourSer.Contracts/          # Attributes and interfaces
│   ├── ISerializable.cs           # Main serialization interface
│   ├── GenerateSerializerAttribute.cs
│   └── SerializeCollectionAttribute.cs
├── FourSer.Gen/          # Source generator implementation
│   ├── SerializerGenerator.cs     # Main generator logic
│   └── ClassToGenerate.cs         # Data model for generation
└── FourSer.Consumer/           # Example usage and tests
    ├── UseCases/                  # Example packet definitions
    ├── Extensions/                # Span read/write extensions
    └── Program.cs                 # Test runner

Performance Characteristics

• Zero allocations during serialization/deserialization
• Compile-time code generation eliminates reflection overhead
• Direct memory access using Span<T> for maximum throughput
• Pattern matching for polymorphic type detection (faster than reflection)
• Little-endian byte order for cross-platform compatibility
• UTF-8 string encoding with length prefixes

Example: Game Network Protocol

[GenerateSerializer]
public partial class LoginAckPacket
{
    public byte bResult;
    public uint dwUserID;
    public uint dwKickID;
    public uint dwKEY;
    public uint Address;
    public ushort Port;
    public byte bCreateCardCnt;
    public byte bInPcRoom;
    public uint dwPremiumPcRoom;
    public long dCurrentTime;
    public long dKey;
}

// Usage
var loginAck = new LoginAckPacket
{
    bResult = 1,
    dwUserID = 12345,
    // ... set other fields
};

var size = LoginAckPacket.GetPacketSize(loginAck);
var buffer = new byte[size];
LoginAckPacket.Serialize(loginAck, buffer);

// Send buffer over network...

// On receive:
var readSpan = new ReadOnlySpan<byte>(receivedBuffer);
var received = LoginAckPacket.Deserialize(readSpan);

Misc

String Behavior

• Strings are serialized as UTF-8 with a length prefix ()

Requirements

• .NET 9.0 or later
• C# 12.0 or later (for static abstract interface members)

Generator Optimization Options

The generator now ships with an internal planning and optimization pipeline between type discovery and C# emission. The default package setting is AggressivePortable.

You can override the optimizer from MSBuild:

<PropertyGroup>
  <FourSerOptimizationLevel>AggressivePortable</FourSerOptimizationLevel>
  <FourSerMinBatchBytes>8</FourSerMinBatchBytes>
  <FourSerStackallocThreshold>256</FourSerStackallocThreshold>
  <FourSerMaxBatchBytes>8192</FourSerMaxBatchBytes>
  <FourSerEmitOptimizationComments>false</FourSerEmitOptimizationComments>
</PropertyGroup>

Available values for FourSerOptimizationLevel:

• Off: disables batching, string fusion, and collection fast paths while keeping planning and validation active.
• Conservative: enables guard/count/type-id caching, constant size folding, and primitive batching.
• AggressivePortable: adds portable collection fast paths and fused stream string writes.
• AggressiveNativeLayout: adds runtime-guarded native-layout bulk paths with portable fallback.

Notes:

• AggressivePortable is the default imported by FourSer.Gen.props.
• AggressiveNativeLayout is opt-in and always keeps a portable fallback path.
• Optimizer diagnostics FSGOPT001 to FSGOPT004 are informational and are emitted only when a fast path is evaluated and declined.
• The branch currently assumes .NET 9+ consumer capabilities.

For the internal architecture and pass order, see docs/optimizer-design.md.

Building

dotnet build

Testing

The solution includes a comprehensive suite of tests to ensure correctness and stability.

• FourSer.Tests: Contains snapshot tests for the source generator using Verify.Xunit. These tests take input source code, run the generator, and compare the output against approved snapshots. This ensures that any change to the generated code is intentional.

  When a snapshot test fails, Verify will create a .received.txt file next to the .verified.txt file. To approve the changes, you can use a diff tool to compare the two files and then copy the content of the received file to the verified file. Many IDEs and diff tools provide a way to do this with a single click.

  Alternatively, you can use the following bash commands:

  To accept all changes:

  find . -name "*.received.txt" -exec sh -c 'mv "$1" "${1%.received.txt}.verified.txt"' _ {} \;
  

  To accept a specific change:

  mv path/to/your.received.txt path/to/your.verified.txt
  

• FourSer.Analyzers.Test: Contains unit tests for the Roslyn analyzers. These tests ensure that the analyzers correctly identify issues in the source code and that the code fixes work as expected.

• FourSer.Tests.Behavioural: Contains behavioural tests that use the generated serializers to perform round-trip serialization and deserialization of various data structures. These tests verify the runtime behavior of the generated code.

• Serializer.Package.Tests: An integration test project that consumes the FourSer.Gen NuGet package. This test ensures that the package works correctly in a real-world scenario, from installation to usage.

To run all tests, use the following command from the root of the repository:

dotnet test

To accept all Verify snapshot updates on PowerShell:

Get-ChildItem . -Recurse -Filter *.received.txt |
    ForEach-Object {
        Copy-Item -LiteralPath $_.FullName -Destination ($_.FullName -replace '\.received\.txt$', '.verified.txt') -Force
    }

Benchmarking

src/FourSer.Gen.Benchmark.Simple benchmarks both generator and runtime behavior from in-memory compilations. It reports:

• full generator run time
• tracked incremental step timings
• generated source count and size
• runtime GetPacketSize
• runtime span serialize and deserialize
• runtime stream serialize and deserialize
• per-operation allocations from GC.GetAllocatedBytesForCurrentThread

Example:

dotnet run --project src/FourSer.Gen.Benchmark.Simple/FourSer.Gen.Benchmark.Simple.csproj -- --levels=Off,Conservative,AggressivePortable,AggressiveNativeLayout --cases=SimplePacket --generation-iterations=5 --runtime-iterations=1000

Contributing

This project uses source generators to provide compile-time serialization code generation. When adding new features:

1. Update the generator logic in SerializerGenerator.cs
2. Add corresponding attributes in FourSer.Contracts
3. Create test cases in FourSer.Consumer/UseCases
4. Run the test suite to verify functionality

License

This project is licensed under the MIT License. See the LICENSE file for details.