Stellar.FastDB 2.1.0

FastDB - High-Performance Embedded Storage for C# (.NET)

Stellar.FastDB is an exceptionally fast document store for C# that is 100x faster than similar products. Designed for optimal performance and high concurrency, it excels in embedded workflows that demand efficiency.

Key Features

• Serverless, embedded document storage.
• Simple, thread-safe API that closely resembles a .NET dictionary, with async/await support.
• Written entirely in C#, targeting .NET 8, 9 and 10.
• Delivered as a single, compact DLL (about 80 KB).
• Supports multiple concurrent readers and writers. Writers that arrive together share one disk write.
• Automatic background compaction reclaims space without stopping writes.
• Fast loading: records are decoded on all cores.
• Schema-less NoSQL storage that adapts to changes.
• Optimized storage footprint with configurable storage formats (JSON, MessagePack).
• LZ4 compression and AES encryption.
• Supports querying with LINQ.
• Ensures end-to-end type safety for data integrity.
• Accommodates composite keys for complex data structuring.
• Open source and free for both personal and commercial use.
• Install from NuGet: Install-Package Stellar.FastDB

TL;DR

It's essentially a ConcurrentDictionary with built-in persistence.

Benchmarks

A detailed list of benchmarks, along with a reproducible project, is available.

FastDB 2.0

Measured on an AMD Ryzen 9 7950X with Windows 11 and .NET 10, using BenchmarkDotNet (1 warmup, 3 iterations).

Compared with other embedded databases

Insert

Delete

Upsert

Bulk Insert

Query

Common Questions

Why was Stellar.FastDB created?

As a game developer, I need a high-concurrency storage solution for player-managed game servers. Requiring players to install local databases isn't practical. However, most available storage options are either too slow or struggle with concurrent reads and writes. These limitations make them unsuitable for game servers, which require high-volume reads and writes without performance issues.

Should I use Stellar.FastDB?

Use Stellar.FastDB if you need:

• Embedded, durable storage suitable for applications like game servers and desktop apps.
• Thread safety with robust support for high concurrency.
• High throughput for both reading and writing data.
• A minimal data storage footprint.

Do not use this database if you need to:

• Share connections between processes.
• Store all data in a single file. Each collection is stored in its own file.

How to use Stellar.FastDB

• Stellar.FastDB's APIs are designed to be intuitive and closely resemble .NET collections. If you are familiar with using a Dictionary, you'll find the transition to FastDB seamless!
• By default, a write returns once it has reached the operating system, so it survives a crash of your process. See Durability for the other modes.
• To get started with Stellar.FastDB, install it from NuGet. Search for 'Stellar.FastDB' or use the package manager command 'Install-Package Stellar.FastDB'.

Durability

The Durability option decides when a write returns and which failures a write survives.

Synced is the slowest mode for a single writer, since every write waits for the disk; concurrent writers share each flush.

Buffered is the fastest mode. A background thread writes the queue to the file; Flush() or FlushAsync() waits for it. MaxBufferedBytes (256 MB by default) caps the queue, and writes wait when it is full.

When a write fails (for example, a full disk), the change stays in memory but isn't written to the file. The call throws an IOException, or returns false if StorageFailureBehavior is ReturnFalse. In Buffered mode, the next Flush() throws instead.

After a crash, earlier writes are intact: every block carries a checksum, and a partly written block is discarded on open.

FastDBOptions options = new FastDBOptions()
{
   Durability = DurabilityMode.Synced,
};
FastDB fastDB = new FastDB(options);

Code Samples

Getting Started

Below is a basic example demonstrating how to interact with Stellar.FastDB. This includes creating a database instance, storing customer data, updating it, retrieving it, and properly closing the database connection:

// create a class
public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public DateTime DOB { get; set; }
    public string Phone { get; set; }
    public bool IsActive { get; set; }
}

// create database
FastDB fastDB = new FastDB();

// create a collection (key, value)
var customers = fastDB.GetCollection<int, Customer>();

// create your new customer instance
var customer = new Customer
{
   Id = 1,
   Name = "John Wick",
   Phone = "555-555-5555",
   DOB = new DateTime(2000, 1, 1),
   IsActive = true,
};

// add customer
customers.Add(customer.Id, customer);

// update customer
customer.Name = "John Wick's Dog";
customers.Update(customer.Id, customer);

// read customer
if (customers.TryGet(1, out Customer found))
    Console.WriteLine(found.Name);

// use LINQ to query
var matches = customers.Where(a => a.Name.StartsWith("John") && a.Phone != null);

// close database
fastDB.Close();

Async

// each write can be awaited
await customers.AddAsync(customer.Id, customer);

// a bulk add writes all of its records together
await customers.AddBulkAsync(newCustomers.ToDictionary(a => a.Id, a => a));

await fastDB.CloseAsync();

Encryption

FastDBOptions options = new FastDBOptions()
{
   IsEncryptionEnabled = true,
   EncryptionPassword = "open-sesame",
};
FastDB fastDB = new FastDB(options);

Compression

Compression applies to the MessagePack serializers, which are the default.

FastDBOptions options = new FastDBOptions()
{
   IsCompressionEnabled = true,
};
FastDB fastDB = new FastDB(options);

Compaction

Every update and removal leaves the old record behind in the file. Background compaction reclaims that space while you write, and finishes the job once writes pause for CompactionIdleDelay (250 ms by default). CompactionEffort sets how hard it works while writes are coming in: Off, Low, Balanced, High or Maximum. The default, Balanced, keeps a file at about twice the size of its live data under steady writing. Off turns background compaction off, and Maximum compacts as fast as possible.

FastDBOptions options = new FastDBOptions()
{
   CompactionEffort = CompactionEffort.Balanced,
};
FastDB fastDB = new FastDB(options);

// or compact everything now
fastDB.Compact();

Large Records

Encryption and compression cost more as records grow. Buffered writes return without waiting for the disk, which recovers much of that cost.

FastDBOptions options = new FastDBOptions()
{
   Durability = DurabilityMode.Buffered,
   IsEncryptionEnabled = true,
   EncryptionPassword = "open-sesame",
   IsCompressionEnabled = true,
};
FastDB fastDB = new FastDB(options);

FastDB 2.0, 10,000 records of about 2 KB each:

Optional Serialization Contracts

To achieve the smallest possible storage footprint, Stellar.FastDB supports serialization contracts using MessagePack. By adding MessagePack attributes to your data models (as shown in the example below), you can instruct the serializer to package the data more efficiently. Note that this feature is disabled by default and is not used in most benchmarks.

// create a class
public class Customer
{
    [Key(0)]
    public int Id { get; set; }
    [Key(1)]
    public string Name { get; set; }
    [Key(2)]
    public DateTime DOB { get; set; }
    [Key(3)]
    public string Phone { get; set; }
    [Key(4)]
    public bool IsActive { get; set; }
}

FastDBOptions options = new FastDBOptions()
{
   Serializer = SerializerType.MessagePack_Contract,
};
FastDB fastDB = new FastDB(options);

// create a collection (key, value)
var customers = fastDB.GetCollection<int, Customer>();

// add customer
customers.Add(customer.Id, customer);

// close database
fastDB.Close();

FastDB 2.0, 10,000 records:

Options

Pass a FastDBOptions to new FastDB(options) to apply it to every collection, or to GetCollection to apply it to one collection.

Where data is stored

Serialization

Writing

Compaction

Loading

Encryption and compression

Error handling

Each of these is Exception (the default) or ReturnFalse. The duplicate-key options can also be Upsert, which replaces the existing record.

Contact

You can reach me on the Stellar Conquest Discord (stonstad) or on X (stonstad).