RocksDb.Net 11.8.1.2

RocksDb.Net

A modern C# wrapper for RocksDb, the high-performance embedded key-value store developed by Meta. Built on .NET's LibraryImport source generator with zero-copy spans and deterministic disposal.

API reference · Guides · Samples · Changelog

Features

• Full RocksDb C API coverage — every exported function in the official rocksdb/c.h header, auto-generated into P/Invoke bindings
• Modern .NET — targets .NET 8, 9 and 10, uses LibraryImport, ReadOnlySpan<byte>, and ref struct iterators
• NativeAOT and trimming — IsAotCompatible, no reflection, and callbacks reached through [UnmanagedCallersOnly] function pointers rather than marshalled delegates. CI publishes samples with PublishAot=true and runs them, one of them with a managed merge operator, so the claim is checked rather than asserted
• Idiomatic C# API — IDisposable handles, properties, string overloads, LINQ-compatible iterators
• Column families — create, drop, and operate on multiple column families, with metadata inspection
• Merge operators — built-in UInt64Add and custom merge operator support
• Compaction filters — filter or transform key-value pairs during compaction
• Transactions — WriteBatch and WriteBatchWithIndex for atomic multi-key operations
• Backups & checkpoints — BackupEngine and Checkpoint for point-in-time snapshots
• SST file ingestion — bulk-load data with SstFileWriter
• Bloom/Ribbon filters — configurable filter policies for point lookups
• Large values — integrated BlobDB stores values above a threshold outside the SST files, with their own cache and garbage collection
• Event listeners — observe flush, compaction, ingestion and background error events, with table properties and compaction statistics
• Write-ahead log — list log files, and stream changes with GetUpdatesSince for replication
• WAL filter — inspect, rewrite or skip records during recovery
• Cross-platform — ships native binaries via the RocksDb.Net.Runtimes package

Versioning

The package version is <RocksDbVersion>.<Revision>, so 11.8.1.1 wraps RocksDb 11.8.1.

Breaking changes land only when the RocksDb version changes. A revision bump alone, such as 11.8.1.1 to 11.8.1.2, never breaks compatibility.

The dependency on the native RocksDb.Net.Runtimes package is bounded to revisions of the same RocksDb version, currently [11.8.1.2, 11.8.2). The P/Invoke declarations are generated from exactly that version's c.h, so a runtimes package built from a different RocksDb version could disagree with them about the native ABI, and nothing would catch it at build or load time.

Upgrading from 11.1.2.1 to 11.8.1.1 has breaking changes. See the changelog for the full list and migrations. In short: the 12 deprecated fluent setters on DbOptions are gone in favour of the properties that replaced them, FlushWal requires its sync argument on both database types, options.EventListener = x becomes options.AddEventListener(x), the size options that were nuint are ulong, ReadTier, Checksum and VerifyOutputFlags are enums, and a handful of members that could not work are removed.

Documentation

• API reference — every public type and member, generated from the source.
• Ownership and lifetime — which side frees each native handle. RocksDb is inconsistent about this and the wrapper follows it rather than hiding it, so this is worth reading before writing much code.
• Callbacks and exceptions — what happens when your comparator or merge operator throws, which thread each callback runs on, and why most options only take effect at open time.
• Samples — runnable examples, one per feature area.
• Changelog — what changed, and how to migrate across a breaking release.

Requirements

• .NET 8.0, 9.0 or 10.0
• RocksDb native binaries (provided by the RocksDb.Net.Runtimes NuGet package)

Quick Start

Every snippet below is compiled and run as part of this repository's test suite, so they are known to work rather than merely to look right.

Install

dotnet add package RocksDb.Net

Basic Usage

using RocksDbNet;

// No `using` on the options: Open takes ownership of them.
var options = new DbOptions { CreateIfMissing = true };
using var db = RocksDb.Open(options, "mydb");

// Write
db.Put("hello", "world");

// Read
string? value = db.GetString("hello");
Console.WriteLine(value); // "world"

// Delete
db.Delete("hello");

Important lifetime note:

• RocksDb.Open* takes ownership of the DbOptions instance you pass in.
• After opening, do not reuse that same DbOptions instance for other operations (for example Destroy, Repair, or ListColumnFamilies).
• If you need options again, create a new DbOptions, or Clone() before passing ownership. A clone shares the original's attached comparator, logger and the rest rather than deep-copying them, and registers itself as another holder, so either can be disposed first.

For static utilities that do not open a DB handle (Destroy, Repair, ListColumnFamilies), ownership is not transferred; dispose those options yourself.

Iteration

using var iterator = db.NewIterator();
iterator.SeekToFirst();

foreach (var entry in iterator)
{
    // Spans into the iterator's own buffers, valid until it moves.
    Console.WriteLine($"{Encoding.UTF8.GetString(entry.Key)} = {Encoding.UTF8.GetString(entry.Value)}");
}

Column Families

var options = new DbOptions
{
    CreateIfMissing = true,
    CreateMissingColumnFamilies = true
};

var descriptors = new List<ColumnFamilyDescriptor>
{
    new("default"),
    new("logs"),
    new("metrics")
};

using var db = RocksDb.Open(options, "mydb", descriptors);

var logsCf = db.GetColumnFamily("logs");
db.Put("entry1", "data", logsCf);

WriteBatch (Atomic Operations)

using var batch = new WriteBatch();
batch.Put("key1", "val1")
     .Put("key2", "val2")
     .Delete("old_key");

db.Write(batch);

Snapshots

using var snapshot = db.NewSnapshot();
using var readOpts = new ReadOptions();
readOpts.SetSnapshot(snapshot);

// Reads see the database state at snapshot time
string? val = db.GetString("key", options: readOpts);

Merge Operators

// Built-in UInt64 addition
var options = new DbOptions { CreateIfMissing = true };
options.SetUInt64AddMergeOperator();
using var db = RocksDb.Open(options, "counters");

db.Merge("visits"u8, BitConverter.GetBytes(1UL));
db.Merge("visits"u8, BitConverter.GetBytes(5UL));

ulong total = BitConverter.ToUInt64(db.Get("visits"u8));
// total == 6

Nested handle lifetime note:

• MergeOperator, CompactionFilterFactory, EventListener, SliceTransform and FilterPolicy are transferred to native ownership when assigned. RocksDb wraps each in a new shared pointer of its own, so one instance per options object: assigning the same one twice would give it two independent owners that each delete it, and the second assignment throws rather than letting that corrupt the heap later.
• Comparator, CompactionFilter, Env, WalFilter, Logger and RateLimiter are released with the DbOptions. These may be shared: attaching one registers a hold and the release happens when the last holder lets go, so disposing one options object never pulls an object out from under another, or from under an open database.
• Disposing one of these yourself while it is still attached is therefore deferred, not obeyed, which makes the usual using shape safe even though the block ends before the database does.
• In all cases, these objects must outlive the open RocksDb instance that uses them.

See Ownership and lifetime for the full rules.

Metadata and statistics

// Statistics live on the options, so keep a reference to read them back.
// These are the options the database owns; do not dispose them yourself.
var options = new DbOptions { CreateIfMissing = true };
options.EnableStatistics();

using var db = RocksDb.Open(options, "stats_db");

db.Put("a", "1");
db.Flush();

var metadata = db.GetColumnFamilyMetadata();
Console.WriteLine(metadata?.Name); // "default"

var histogram = options.GetHistogramData(Histogram.DbWrite);
Console.WriteLine(histogram?.Count);

Live files and approximate sizes

using var db = RocksDb.Open(new DbOptions { CreateIfMissing = true }, "inspection_db");

db.Put("a", "1");
db.Put("z", "2");
db.Flush();

// Read in full and copied out, so there is nothing to dispose.
IReadOnlyList<LiveFileMetadata> liveFiles = db.GetLiveFiles();
Console.WriteLine(liveFiles.Count);

ulong[] sizes = db.ApproximateSizes(new[] { ("a", "z") });
Console.WriteLine(sizes[0]);

ulong[] cfSizes = db.ApproximateSizes(db.GetDefaultColumnFamily(), new[] { ("a", "z") });
Console.WriteLine(cfSizes[0]);

Advanced maintenance helpers

using var db = RocksDb.Open(new DbOptions { CreateIfMissing = true }, "maintenance_db");

using var compactOpts = new WaitForCompactOptions { Flush = true, TimeoutMicros = 5_000_000 };
db.SuggestCompactRange(Encoding.UTF8.GetBytes("a"), Encoding.UTF8.GetBytes("z"));
db.DeleteFilesInRange("a", "z");
db.WaitForCompact(compactOpts);

// Last, and not before WaitForCompact: cancelling puts the database into
// shutdown, and waiting after that fails with "Shutdown in progress".
db.CancelAllBackgroundWork(wait: false);

Backup & Restore

// The options say how to reach the database being backed up; the path is
// where the backups go.
using var backupOptions = new DbOptions();
using var engine = BackupEngine.Open(backupOptions, "backups");

engine.CreateNewBackup(db);

// Later: restore, into a database directory and a WAL directory.
engine.RestoreDbFromLatestBackup("restored_db", "restored_db");

SST File Ingestion

using var envOpts = new EnvOptions();
using var dbOpts = new DbOptions();
using var writer = SstFileWriter.Create(envOpts, dbOpts);

writer.Open("data.sst");

// Keys and values are bytes here, and must be in sorted order.
writer.Put("key1"u8, "val1"u8);
writer.Put("key2"u8, "val2"u8);
writer.Finish();

using var ingestOptions = new IngestExternalFileOptions();
db.IngestExternalFile(new[] { "data.sst" }, ingestOptions);

Bloom Filters

using var tableOptions = new BlockBasedTableOptions();
tableOptions.SetFilterPolicy(FilterPolicy.CreateBloomFull(10));

var options = new DbOptions { CreateIfMissing = true };
options.BlockBasedTableFactory = tableOptions;

using var db = RocksDb.Open(options, "filtered_db");

Large Values (BlobDB)

Values above MinBlobSize are written to separate blob files instead of into the SST files, so compaction moves keys around without rewriting the values attached to them. Worth turning on when values are large; not worth it when they are small, because every read of a blob costs an extra file access.

var options = new DbOptions
{
    CreateIfMissing = true,
    EnableBlobFiles = true,
    MinBlobSize = 1024,          // values at or above this go to a blob file

    // Reclaim space from blob files whose values have been overwritten.
    EnableBlobGarbageCollection = true,
};

// Blobs live outside the SST files, so the block cache never holds them.
// Without a blob cache, every blob read goes to the file system.
using var blobCache = Cache.CreateLru(256 * 1024 * 1024);
options.BlobCache = blobCache;
options.PrepopulateBlobCache = PrepopulateBlobCache.FlushOnly;

using var db = RocksDb.Open(options, "blob_db");

db.Put("large", new string('v', 4096));   // stored as a blob
db.Put("small", "v");                     // stays in the SST

Samples

The Samples/ directory contains runnable examples:

Run any sample with:

dotnet run --project Samples/BasicSample

Architecture

RocksDb.Net/
├── Native/
│   ├── NativeMethods.g.cs        # Generated P/Invoke bindings (1,745 functions)
│   └── NativeMethods.Helpers.cs  # Native library resolver and helpers
├── RocksDb.cs                    # Main database class
├── DbOptions.cs                  # Database configuration options
├── WriteBatch.cs                 # Atomic write operations
├── Iterator.cs                   # Key-value iteration
├── ColumnFamilyHandle.cs         # Column family management
├── BackupEngine.cs               # Backup and restore
├── Checkpoint.cs                 # Database checkpoints
├── SstFileWriter.cs              # SST file creation for bulk loading
├── MergeOperator.cs              # Custom merge operators
├── CompactionFilter.cs           # Compaction-time key filtering
├── EventListener.cs              # Database event notifications
└── ...                           # Options, filters, cache, etc.

Building from Source

git clone https://github.com/zcsizmadia/RocksDb.Net.git
cd RocksDb.Net
dotnet build
dotnet test

Two files are auto-generated from RocksDb's own headers at the pinned version: the P/Invoke bindings in NativeMethods.g.cs, from c.h, and the Ticker and Histogram enums in StatisticsEnums.g.cs, from statistics.h. The statistics counters are not declared in c.h, and their values are positional, so they are read from where they are defined rather than written out by hand. To regenerate both:

dotnet run --project NativeMethodsGenerator

Run it from the repository root. The version comes from RocksDbVersion in Directory.Build.props, which is the property that decides which native library the package binds to, so bumping that first and then regenerating is the whole upgrade sequence. --version and --project override the two if you need to.

CI regenerates both and fails if either differs from what is committed, so a hand edit, a generator change that was never re-run, or a version bump that left the output behind all fail there rather than shipping.

Acknowledgements

RocksDB is developed and maintained by Meta Platforms, Inc. (formerly Facebook, Inc.) and contributors, at github.com/facebook/rocksdb. This project is a wrapper around their work and would not exist without it.

RocksDb.Net is not affiliated with, endorsed by, or sponsored by Meta Platforms, Inc.

License

The wrapper is MIT. See LICENSE.

RocksDB is dual-licensed under the GPLv2 and Apache 2.0 License, and its terms apply to the native library and to the generated bindings derived from its C header. See THIRD-PARTY-NOTICES.md for attribution and detail, and RocksDB's own licence files for the authoritative terms.