GaldrDbEngine 1.0.0-rc2

GaldrDb

Native AOT compatible single-file document database for .NET 8.0, 9.0, and 10.0.

GaldrDb is a strongly typed, single-file document database for .NET that works everywhere your app does — servers, desktop on Windows, macOS, and Linux (x64 and ARM64), and mobile. It works great with standard .NET, but its source generators validate queries at compile time with zero reflection, giving you full Native AOT compatibility where ORMs like EF Core still struggle. Just one NuGet package, no external dependencies.

Features

• Type-safe queries validated at compile time
• ACID transactions with snapshot isolation and optimistic concurrency
  • Automatic retry with jitter on page-level conflicts
• Fluent query API with filtering, sorting, pagination, and projections
  • Equality, range, set membership, and string operations (StartsWith, Contains, EndsWith)
• Indexes with automatic query plan optimization
  • Single-field and compound (multi-field) indexes
  • Unique constraints
  • Nested document path indexes
• Write-ahead logging (WAL) for crash recovery and durability
• Encryption at rest with AES-256-GCM and per-page nonces (optional)
• Dynamic API for runtime schema flexibility with JSON documents
• Native AOT compatible — no reflection, no runtime code generation

Quick Start

Installation

dotnet add package GaldrDbEngine --prerelease

GaldrDb is currently in release candidate. Once stable, use dotnet add package GaldrDbEngine.

Example

// Define a model
[GaldrDbCollection]
public class Book
{
    public int Id { get; set; }
    [GaldrDbIndex]
    public string Title { get; set; }
    public string Author { get; set; }
}

// Create a database
GaldrDbOptions options = new() { PageSize = 8192, UseWal = true };
GaldrDb db = GaldrDb.Create("books.db", options);

// Insert a document
int id = db.Insert(new Book { Title = "The Pragmatic Programmer", Author = "Andy Hunt" });

// Query documents
Book book = db.Query<Book>()
    .Where(BookMeta.Title, FieldOp.Equals, "The Pragmatic Programmer")
    .FirstOrDefault();

Getting Started

Creating and Opening Databases

// Create a new database
GaldrDb db = GaldrDb.Create("database.db", new GaldrDbOptions());

// Open an existing database
GaldrDb db = GaldrDb.Open("database.db");

// Open or create as needed
GaldrDb db = GaldrDb.OpenOrCreate("database.db");

Defining Models

[GaldrDbCollection]
public class Person
{
    public int Id { get; set; }
    [GaldrDbIndex]
    public string Name { get; set; }
    public int Age { get; set; }
}

[GaldrDbCollection("customers")]
public class Customer
{
    public int Id { get; set; }
    [GaldrDbIndex(Unique = true)]
    public string Email { get; set; }
}

Basic CRUD Operations

// Insert
int id = db.Insert(new Person { Name = "Alice", Age = 30 });

// Get by ID
Person person = db.GetById<Person>(id);

// Replace
person.Age = 31;
db.Replace(person);

// Delete
db.DeleteById<Person>(id);

// Partial update (update specific fields without loading the full document)
db.UpdateById<Person>(id)
    .Set(PersonMeta.Age, 32)
    .Execute();

Querying

// Filter
List<Person> adults = db.Query<Person>()
    .Where(PersonMeta.Age, FieldOp.GreaterThanOrEqual, 18)
    .ToList();

// Range query
List<Person> people = db.Query<Person>()
    .WhereBetween(PersonMeta.Age, 25, 40)
    .ToList();

// Sort and limit
List<Person> results = db.Query<Person>()
    .OrderByDescending(PersonMeta.Age)
    .Limit(10)
    .ToList();

// In/NotIn queries
List<Person> selected = db.Query<Person>()
    .WhereIn(PersonMeta.Name, "Alice", "Bob", "Charlie")
    .ToList();

// Pagination
List<Person> page = db.Query<Person>()
    .OrderBy(PersonMeta.Name)
    .Skip(20)
    .Limit(10)
    .ToList();

// Check existence
bool hasAdults = db.Query<Person>()
    .Where(PersonMeta.Age, FieldOp.GreaterThanOrEqual, 18)
    .Any();

// Query explanation (shows execution plan)
QueryExplanation explanation = db.Query<Person>()
    .Where(PersonMeta.Age, FieldOp.GreaterThan, 21)
    .Explain();

ASP.NET Core Integration

Install the GaldrDbAspNetCore package for dependency injection support:

dotnet add package GaldrDbAspNetCore --prerelease

Basic setup (single database):

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddGaldrDb(options =>
{
    options.FilePath = "app.db";
    options.OpenMode = GaldrDbOpenMode.OpenOrCreate;
});

WebApplication app = builder.Build();

// Inject IGaldrDb directly
app.MapGet("/users/{id}", (IGaldrDb db, int id) =>
{
    return db.GetById<User>(id);
});

Multiple databases (named instances):

// Register named databases
builder.Services.AddGaldrDb(options =>
{
    options.FilePath = "users.db";
});

builder.Services.AddGaldrDb("orders", options =>
{
    options.FilePath = "orders.db";
});

// Access named instances via IGaldrDbFactory
app.MapGet("/orders/{id}", (IGaldrDbFactory factory, int id) =>
{
    IGaldrDb db = factory.Get("orders");
    return db.GetById<Order>(id);
});

Advanced Features

Transactions

using (ITransaction tx = db.BeginTransaction())
{
    Person person = tx.GetById<Person>(id);
    person.Age++;
    tx.Replace(person);
    tx.Commit();
}

Indexes

Add [GaldrDbIndex] to properties to create indexes. Use Unique = true to enforce uniqueness:

[GaldrDbCollection]
public class Product
{
    public int Id { get; set; }
    [GaldrDbIndex]
    public string Category { get; set; }
    [GaldrDbIndex(Unique = true)]
    public string Sku { get; set; }
}

List<Product> products = db.Query<Product>()
    .Where(ProductMeta.Category, FieldOp.Equals, "Electronics")
    .ToList();

Compound Indexes

Compound indexes optimize queries that filter on multiple fields. Define them at the class level:

[GaldrDbCollection]
[GaldrDbCompoundIndex("Status", "CreatedDate")]
[GaldrDbCompoundIndex("Category", "Priority")]
public class Order
{
    public int Id { get; set; }
    public string Status { get; set; }
    public DateTime CreatedDate { get; set; }
    public string Category { get; set; }
    public int Priority { get; set; }
}

// Uses the Status_CreatedDate compound index
List<Order> orders = db.Query<Order>()
    .Where(OrderMeta.Status, FieldOp.Equals, "Pending")
    .Where(OrderMeta.CreatedDate, FieldOp.GreaterThan, DateTime.Today.AddDays(-7))
    .ToList();

// Prefix queries also use the compound index
List<Order> pendingOrders = db.Query<Order>()
    .Where(OrderMeta.Status, FieldOp.Equals, "Pending")
    .ToList();

Compound indexes follow the leftmost-prefix rule: an index on (A, B, C) can serve queries on A, A AND B, or A AND B AND C. The query planner automatically selects the best index based on filter selectivity.

Nested Document Indexes

Indexes work on nested object properties too. Add [GaldrDbIndex] to nested class properties, or use dot notation in compound indexes:

public class Address
{
    [GaldrDbIndex]
    public string City { get; set; }
    public string Country { get; set; }
}

[GaldrDbCollection]
[GaldrDbCompoundIndex("Status", "Address.City")]
public class Person
{
    public int Id { get; set; }
    public string Status { get; set; }
    public Address Address { get; set; }
}

// Uses index on Address.City
List<Person> results = db.Query<Person>()
    .Where(PersonMeta.Address.City, FieldOp.Equals, "Seattle")
    .ToList();

// Uses compound index
List<Person> active = db.Query<Person>()
    .Where(PersonMeta.Status, FieldOp.Equals, "Active")
    .Where(PersonMeta.Address.City, FieldOp.Equals, "Seattle")
    .ToList();

Null handling follows standard database semantics: null nested values are excluded from equality queries, and multiple null values are allowed for unique indexes (NULL != NULL).

Collection Field Queries

Query documents based on elements within arrays or lists:

[GaldrDbCollection]
public class Order
{
    public int Id { get; set; }
    public List<OrderItem> Items { get; set; }
}

public class OrderItem
{
    public string ProductName { get; set; }
    public decimal Price { get; set; }
}

// Find orders containing items over $100
List<Order> results = db.Query<Order>()
    .WhereAny(OrderMeta.Items.Price, FieldOp.GreaterThan, 100m)
    .ToList();

Projections

Projections let you query a subset of fields from a document without deserializing the entire object. Define a projection class with only the fields you need:

[GaldrDbCollection]
public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
    public string Email { get; set; }
    public Address Address { get; set; }
}

[GaldrDbProjection(typeof(Person))]
public partial class PersonSummary
{
    public int Id { get; set; }
    public string Name { get; set; }
}

List<PersonSummary> summaries = db.Query<PersonSummary>().ToList();

Vacuum and Compact

// Remove old versions and compact pages
GarbageCollectionResult result = db.Vacuum();

// Create a compacted copy
DatabaseCompactResult compactResult = db.CompactTo("compacted.db");

Encryption at Rest

// Create an encrypted database
GaldrDbOptions options = new()
{
    Encryption = new EncryptionOptions
    {
        Password = "your-secret-password",
        KdfIterations = 500000,  // PBKDF2 iterations (higher = more secure, slower open)
    }
};
GaldrDb db = GaldrDb.Create("encrypted.db", options);

// Open an encrypted database (same password required)
GaldrDb db = GaldrDb.Open("encrypted.db", options);

Encryption uses AES-256-GCM with per-page nonces. The WAL file is also encrypted when encryption is enabled.

Checkpoint Control

// Manually checkpoint WAL to main database file
db.Checkpoint();

// Supports async
await db.CheckpointAsync();

Dynamic API

For scenarios where types aren't known at compile time, use the dynamic API with JSON strings:

// Insert with JSON
int id = db.InsertDynamic("people", """{"Name": "Alice", "Age": 30}""");

// Get by ID (returns JsonDocument)
JsonDocument doc = db.GetByIdDynamic("people", id);

// Replace
db.ReplaceDynamic("people", id, """{"Id": 1, "Name": "Alice", "Age": 31}""");

// Partial update
db.UpdateByIdDynamic("people", id)
    .Set("Age", 32)
    .Execute();

// Query
List<JsonDocument> results = db.QueryDynamic("people")
    .Where("Age", FieldOp.GreaterThan, 25)
    .OrderBy("Name")
    .ToList();

// Delete
db.DeleteByIdDynamic("people", id);

Architecture & Design Decisions

Storage Engine

• Page-based storage with configurable page size (default 8KB)
• Large document support — documents automatically span multiple pages when needed
• B+ tree indexing for primary key lookups, range scans, and secondary indexes
• Page cache with configurable size for frequently accessed pages
• Encryption at rest — optional AES-256-GCM encryption with per-page nonces

Concurrency Model

• MVCC with snapshot isolation — readers never block writers, each transaction sees a consistent point-in-time view
• Optimistic concurrency control with conflict detection and automatic retry at commit time

Durability

• Write-Ahead Logging (WAL) — all writes are logged before applying to the main database, with checksums for data integrity
• Crash recovery — committed transactions are replayed automatically on startup
• Automatic checkpointing — WAL changes are periodically merged to the main database

Configuration Options

GaldrDbOptions options = new()
{
    PageSize = 8192,                      // Page size (must be power of 2, >= 1024)
    UseWal = true,                        // Enable write-ahead logging
    AutoCheckpoint = true,                // Automatically checkpoint WAL
    WalCheckpointThreshold = 1000,        // Frames before auto-checkpoint
    UseMmap = false,                      // Use memory-mapped files (with auto-fallback)
    WarmupOnOpen = true,                  // Warmup pools on database open
    JsonWriterBufferSize = 4096,          // Initial buffer size for JSON serialization
    JsonWriterPoolWarmupCount = 4,        // JSON writer pool warmup size
    AutoGarbageCollection = true,         // Automatically collect old versions
    GarbageCollectionThreshold = 250,     // Commits before auto-gc
    ExpansionPageCount = 256,             // Pages to add on expansion (2MB with 8KB pages)
    PageCacheSize = 2000,                 // Max cached pages (16MB with 8KB pages)
    Encryption = new EncryptionOptions    // Optional encryption (null = disabled)
    {
        Password = "your-secret-password",
        KdfIterations = 500000,           // PBKDF2 iterations for key derivation
    },
};

Benchmarks

These are the current benchmark results from SingleOperationAotBenchmarks and SingleOperationBenchmarks. The results are of the current in development build, with future optimizations planned. Additional benchmarks with more advanced queries will be added at a later date.

Setup

BenchmarkDotNet v0.15.8, Linux Fedora Linux 43 (Workstation Edition) AMD Ryzen AI 7 PRO 350 w/ Radeon 860M 0.62GHz, 1 CPU, 16 logical and 8 physical cores .NET SDK 10.0.101

InvocationCount=16384 IterationCount=20 WarmupCount=3

SingleOperationAotBenchmarks

Native AOT comparison between GaldrDb and SQLite ADO.NET. Both use WAL mode with equivalent durability guarantees (synchronous=FULL). GaldrDb shows strong read and delete performance; write performance is an area of active optimization.

SingleOperationBenchmarks

JIT comparison including EF Core. GaldrDb outperforms EF Core across all operations while providing similar developer ergonomics. Note that EF Core Insert has significant per-operation overhead from change tracking and context management.

Testing

• 1073 unit and integration tests covering CRUD, transactions, queries, ACID properties, and recovery scenarios
• 63 deterministic simulation tests for concurrent operations, conflict resolution, and edge cases
• 1136 total tests across the test suite
• Stress tests simulating high contention scenarios with concurrent transactions in different workloads
• Performance benchmarks for single operations, bulk inserts, and query performance

Project Structure

• GaldrDbEngine/ - Core database engine library
• GaldrDbAspNetCore/ - ASP.NET Core dependency injection support
• GaldrDbBrowser/ - Cross-platform database browser for inspecting and editing data
• GaldrDbConsole/ - Console application for benchmarking, stress testing, and diagnostics
• GaldrDbSourceGenerators/ - Roslyn source generators for metadata generation
• Tests/GaldrDb.UnitTests/ - Integration and unit tests
• Tests/GaldrDb.SimulationTests/ - Deterministic simulation tests for concurrent scenarios