Soenneker.Librarian.FileSystem 4.0.69

Soenneker.Librarian

Document storage for .NET 10 with interchangeable memory, JSON file, Redis, PostgreSQL, Cloudflare D1, Cloudflare R2, and Cloudflare Workers KV providers.

• Async document operations — store, retrieve, update, and delete JSON by ID.
• Atomic batches — update related documents together, with conditions to prevent conflicting writes.
• Typed repositories — work with document classes through ILibrarianRepository<TDocument>.
• Indexed queries — filter, sort, count, and page with LINQ or explicit async index methods.

Website · Quick start · Providers · Queries · Typed repositories · Performance

Installation

Install the provider you need. Shared dependencies are included automatically.

dotnet add package Soenneker.Librarian.Memory
# Or: dotnet add package Soenneker.Librarian.FileSystem
# Or: dotnet add package Soenneker.Librarian.Maui.Secure
# Or: dotnet add package Soenneker.Librarian.IndexedDb
# Or: dotnet add package Soenneker.Librarian.LocalStorage
# Or: dotnet add package Soenneker.Librarian.SessionStorage
# Or: dotnet add package Soenneker.Librarian.Redis
# Or: dotnet add package Soenneker.Librarian.Postgres
# Or: dotnet add package Soenneker.Librarian.D1
# Or: dotnet add package Soenneker.Librarian.R2
# Or: dotnet add package Soenneker.Librarian.Kv

Quick start

This standalone example registers the memory provider, stores a document, and reads it back:

using Microsoft.Extensions.DependencyInjection;
using Soenneker.Librarian.Abstractions;
using Soenneker.Librarian.Memory.Registrars;

var services = new ServiceCollection();
services.AddLogging();
services.AddMemoryLibrarianDatabaseAsSingleton();

await using var provider = services.BuildServiceProvider();
var database = provider.GetRequiredService<ILibrarianDatabase>();
var users = await database.GetContainer("users");

await users.AddItem("user-1", """{"name":"Alex","age":30,"active":true}""");
string? json = await users.GetItem("user-1");

In a hosted application, register the provider on builder.Services and inject ILibrarianDatabase into your service.

Native AOT and generated JSON contracts

Typed operations now require source-generated JSON metadata. Register document types once at startup, before building queries or using repositories and typed index reads. Raw JSON CRUD needs no registration.

using System.Text.Json.Serialization;
using Soenneker.Librarian.Abstractions.Serialization;

LibrarianJson.Register(AppJsonContext.Default.User);

[JsonSourceGenerationOptions(PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase,
    PropertyNameCaseInsensitive = true, UseStringEnumConverter = true)]
[JsonSerializable(typeof(User))]
internal partial class AppJsonContext : JsonSerializerContext;

Use the same generated contract when serializing documents. Register enums (and nullable enums) separately when used as index values or remote projection columns. Primitive scalars already have generated contracts. Registrations are process-wide and immutable; registering a different contract for an already registered type throws. Custom converters must themselves support trimming and Native AOT. See migration and query restrictions.

All production projects enable AOT/trimming analyzers. CI publishes and runs a native executable with reflection-based JSON serialization disabled, including Redis and PostgreSQL integration checks.

Generated query accessors and delegates

Reference Soenneker.Librarian.Generators in the application that contains the models and query calls, with PrivateAssets="all". Keep the existing System.Text.Json context and LibrarianJson.Register calls.

using Soenneker.Librarian.Abstractions;

public sealed class Customer
{
    public int Age { get; set; }
    public string Name { get; set; } = "";
}

public static class CustomerQueries
{
    public static IQueryable<string> Adults(ILibrarianContainer container, bool filter)
    {
        var source = container.BuildQueryable<Customer>();
        if (filter) source = source.Where(customer => customer.Age >= 18);
        return source.OrderBy(customer => customer.Name).Select(customer => customer.Name);
    }
}

No Librarian attributes are required for this example. The generator recognizes Librarian container, repository, and database query APIs by their symbols and discovers document types from those calls and resolvable LibrarianJson.Register<T> calls. It follows LINQ chains, local aliases, assignments, branches, and awaited repository queries. It registers model getters, camel-case/JsonPropertyName mappings, and scalar index-key factories, and intercepts supported single-parameter Queryable lambdas to register compiled delegates while preserving the provider's expression tree. Supported explicit constructor projections also get generated materializers for PostgreSQL and Redis. Query translation still happens at runtime; this is not full query precompilation.

Locals that can receive queries from another provider, query fields, and helpers accepting an ordinary IQueryable<T> are conservatively left alone. [GenerateLibrarianQueries] remains an optional opt-in for such helper classes; [LibrarianModel] remains available for additional models that cannot be discovered from query roots. Attributes use the Soenneker.Librarian.Abstractions.Queries namespace.

Captured locals/outer parameters, anonymous projections, and other unsupported lambdas produce informational LIBGEN002 diagnostics for automatically discovered queries and retain existing runtime execution, including deferred capture evaluation. Explicitly annotated query classes retain warning-level diagnostics. This release does not guarantee reflection-free LINQ. LIBGEN001 identifies inaccessible or generic explicitly annotated models. Use the .NET 10 SDK; the package enables its interceptor namespace automatically. For a project reference, set OutputItemType="Analyzer", ReferenceOutputAssembly="false", and add Soenneker.Librarian.Generated to InterceptorsNamespaces.

When the consuming project references the Mongo provider, generated Soenneker.Librarian.Generated.LibrarianModels.RegisterMongo(registry) registers discovered or annotated documents and supported nested documents, scalars, nullable values, arrays, lists, and string-keyed dictionaries. Call it once per registry, then pass that registry to the provider's explicit serializer API. LIBGEN003 reports types requiring a manual factory. These mappings assume camel-case JSON unless overridden by JsonPropertyName.

Choose a provider

Register one provider for ILibrarianDatabase. Memory, FileSystem, Redis, and PostgreSQL registrars also offer an AsScoped() variant; a scoped memory database has its own data.

Cloudflare D1, R2, and KV

using Soenneker.Librarian.D1.Registrars;
// Or: using Soenneker.Librarian.R2.Registrars;
// Or: using Soenneker.Librarian.Kv.Registrars;

builder.Services.AddD1LibrarianDatabaseAsSingleton();
// Or: builder.Services.AddR2LibrarianDatabaseAsSingleton();
// Or: builder.Services.AddKvLibrarianDatabaseAsSingleton();

D1 uses Soenneker.Cloudflare.D1; R2 uses Soenneker.Cloudflare.R2; KV uses Soenneker.Cloudflare.Workers.Kv. Configure the selected provider:

For KV, configure Librarian:Kv:AccountId, Librarian:Kv:ApiKey, and Librarian:Kv:NamespaceId; Librarian:Kv:Key defaults to librarian.json. Provision the namespace first and grant the token Workers KV Storage read/write access. Keyed registration is available through AddKvLibrarianDatabaseAsSingleton(serviceKey, factory).

KV snapshots are limited to 25 MiB per value. Saves are subject to Cloudflare rate limits; failed saves retain pending changes for retry. KV is eventually consistent, including missing-key reads: a newly opened instance can see an older or missing snapshot after a successful save. Coordinate owner handoffs and allow propagation before reopening; do not use KV when immediate read-after-write consistency across restarts is required. Batch conditions are checked against local state, without distributed transaction guarantees.

Provision the D1 database or R2 bucket first and provide a token with read/write access. D1 creates its librarian_snapshots table automatically, using parameterized queries. The D1 snapshot and logical name together are limited to 1,900,000 UTF-8 bytes to leave room below D1's row-size limit.

These providers load the complete snapshot into memory and use the same local queries and indexes as Memory and FileSystem. Use one database owner per D1 logical name, R2 object key, or KV namespace/key. They do not coordinate multiple application instances or refresh external changes. Indexes are rebuilt after unloading or restarting.

Ordinary mutations remain pending until await database.Save(), container unloading, or asynchronous disposal. There is no periodic background save. Atomic batches persist the complete snapshot before publishing changes, including other pending mutations. Failed saves retain changes for retry; failed disposal can be retried. A transport failure after dispatch can leave the remote commit outcome unknown, so reconcile persisted state before retrying non-idempotent work. The providers do not dispose injected Cloudflare clients.

PostgreSQL

Use Soenneker.Librarian.Postgres for shared persistent documents with SQL-backed filtering, sorting, paging, and aggregates. Writes and cross-container conditional batches commit immediately; indexes persist across restarts. It requires PostgreSQL 16 or later.

using Soenneker.Librarian.Postgres.Registrars;

builder.Services.AddPostgresLibrarianDatabaseAsSingleton();

Configure Librarian:Postgres:ConnectionString and Librarian:Postgres:Key. See PostgreSQL setup, queries, and transaction semantics.

FileSystem

using Soenneker.Librarian.FileSystem.Registrars;

builder.Services.AddFileSystemLibrarianDatabaseAsSingleton();

Add to appsettings.json:

{
  "Librarian": {
    "FileSystem": { "FilePath": "data/librarian.json" }
  }
}

• Use one database owner per file.
• Await database.Save() when changes must be flushed explicitly.
• Dispose the database's owner asynchronously to complete shutdown persistence.

MAUI Secure

using Soenneker.Librarian.Maui.Secure.Registrars;

// Scope must identify the application, user and organization unambiguously.
builder.Services.AddMauiSecureLibrarianDatabaseAsSingleton("myapp/user-id/organization-id");

The provider keeps documents in memory and atomically replaces an AES-256-GCM encrypted file under the app-data directory. SecureStorage holds only the key. Encryption authenticates the scope and file format. Missing or invalid keys, corrupt files, and unavailable SecureStorage throw; there is no plaintext fallback or automatic reset. Keep authentication tokens directly in SecureStorage.

Use one owner per scope. Ordinary writes persist on Save, unload, or disposal; batches persist before publication. Call Save after important changes and before suspension. Mobile process termination does not guarantee disposal. For account switching, stop operations and dispose the old owner before creating the new scope's owner. Fixed scopes also support keyed singleton registration with (serviceKey, scope, directoryPath).

After stopping operations, IMauiSecureLibrarianDatabase.DiscardAsync() releases an owner without saving, including when secure storage is unavailable. MauiSecureLibrarianDatabase.DeleteStorage(scope, secureStorage, directoryPath) deletes that scope's file and protected key after its owners have been disposed. Use the same directory supplied at construction. Other scopes remain intact.

Browser storage

using Soenneker.Librarian.IndexedDb.Registrars;
using Soenneker.Librarian.LocalStorage.Registrars;
using Soenneker.Librarian.SessionStorage.Registrars;

builder.Services.AddIndexedDbLibrarianDatabaseAsScoped("myapp/user-id/organization-id");
// Alternatively, for small snapshots:
// builder.Services.AddLocalStorageLibrarianDatabaseAsScoped("myapp/user-id/organization-id");
// Or, for tab-session storage:
// builder.Services.AddSessionStorageLibrarianDatabaseAsScoped("myapp/user-id/organization-id");

These providers use the interactive client's IJSRuntime. The Browser package supplies the JavaScript module as a static web asset; no external CDN or manual script tag is required. Invoke database operations after interactive rendering, not during prerender. Scoped instances prevent sharing a Blazor Server client's runtime with other clients. All three browser registrars support (serviceKey, key) for independently keyed registrations.

Each storage key holds a complete snapshot; document queries and indexes run in memory. IndexedDB commits snapshot replacement inside a read/write transaction. LocalStorage uses the Web Locks API and requires a secure context and cooperating writers. LocalStorage is intended for small snapshots. Neither provider encrypts browser data or stores secrets securely.

SessionStorage uses the same Web Locks coordination as LocalStorage, but its data belongs to the origin and tab session. It survives reloads; independent tabs have separate storage. A tab opened with an opener may initially receive a copy, and browser session restoration follows browser policy. It is unencrypted and intended for small snapshots.

Call Save after ordinary mutations; batches persist before publication. Quota, access-denied, unavailable-feature and transaction failures propagate without reporting success. Do not rely on browser shutdown or circuit disposal for a final flush.

Cached reads are not live-synchronized between owners. Writes compare the last loaded snapshot with persisted storage to detect external changes. A conflict throws LibrarianConcurrencyException. Stop operations, call IBrowserLibrarianDatabase.DiscardAsync(), create a fresh owner, and reapply the intended change to freshly loaded data. Interop cancellation or disconnection after dispatch can leave the commit outcome unknown; reopen and reconcile before retrying. Discard leaves persisted storage intact.

Redis

using Soenneker.Librarian.Redis.Registrars;

builder.Services.AddRedisLibrarianDatabaseAsSingleton();

Add to application configuration:

{
  "Azure": { "Redis": { "ConnectionString": "localhost:6379" } },
  "Librarian": { "Redis": { "Key": "my-app:librarian", "KeyPrefix": "librarian", "Database": -1 } }
}

• Key is the required namespace; Database is optional (-1 uses the connection default).
• Reads fetch current Redis data; writes update documents and indexes together.
• Requires atomic Lua scripts and conditional transactions; no RedisJSON, Redis Search, or SORT support is required.
• Redis persistence and eviction settings determine durability. Use a non-evicting database for document storage.
• Garnet servers must enable --lua --lua-transaction-mode. Persistence and recovery must be configured separately.

See the Redis guide for supported queries, index costs, concurrency, and deployment details.

Applications that already resolve a shared Redis database can use new RedisLibrarianDatabase(key, storeFactory) with a Func<CancellationToken, ValueTask<IDatabase>>. The caller retains connection ownership. GetServerTime() reads UTC time from the primary owning the namespace's hash slot using native Redis commands, without Lua.

Query documents

The following examples use the users container from the quick start and this model:

public sealed record User(string Name, int Age, bool Active);

LINQ

using System.Linq;

var adults = users.BuildQueryable<User>()
    .Where(user => user.Active && user.Age >= 18 && user.Age <= 65)
    .OrderBy(user => user.Age)
    .Take(25)
    .ToList();

Indexes are created automatically for supported filters and ordering, then maintained on writes. The first indexed query pays the index creation cost.

Use the async terminal extensions or the explicit index methods below:

using Soenneker.Librarian.Abstractions.Queries;

var adults = await users.BuildQueryable<User>()
    .Where(user => user.Age >= 18).OrderBy(user => user.Age)
    .Take(25).ToListAsync(cancellationToken);

See query capabilities, async execution, and provider differences.

Async index queries

await users.EnsureIndex("age");

var page = await users.FindRangeByIndex<User>(
    "age", minimum: 18, maximum: 65, skip: 0, take: 25);

var thirtyYearOlds = await users.FindByIndex<User>("age", 30, take: 10);
int count = await users.CountByIndex("age", 30);
bool exists = await users.ExistsByIndex("age", 30);

foreach (User user in page.Items)
    System.Console.WriteLine(user.Name);

• Use case-sensitive JSON field paths, such as age or address.city.
• Call EnsureIndex before explicit queries; missing indexes throw instead of scanning.
• Range bounds are inclusive; null means unbounded. skip must be nonnegative and take positive.
• Index values support strings, decimal-compatible numbers, booleans, and null. Missing fields are excluded.
• Results expose Items, Index, IndexEntriesExamined, and DocumentsDeserialized.

Document operations

All methods above are awaitable and accept a cancellation token. Document IDs are case-insensitive; container names are case-sensitive. The database owns container lifetime; use UnloadContainer() to release a container after stopping its concurrent operations.

Atomic batches

Update related documents across containers in one all-or-nothing operation with ILibrarianDatabase.Execute. Add conditions to prevent overwriting concurrent changes; if a condition fails, it returns false and applies no writes.

Available with all six providers, including coordination across application instances with Redis and PostgreSQL. See the atomic batch guide for an example, provider guarantees, and retry handling.

Typed repositories

Use LibrarianRepository<TDocument> for serialization and typed CRUD. Models must inherit from Document and have a nonempty ID when added.

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Soenneker.Documents.Document;
using Soenneker.Librarian.Abstractions;
using Soenneker.Librarian.Core;

public sealed class Customer : Document
{
    public string Email { get; set; } = "";
}

public sealed class CustomerRepository(
    IConfiguration configuration,
    ILogger<LibrarianRepository<Customer>> logger,
    ILibrarianDatabase database)
    : LibrarianRepository<Customer>(configuration, logger, database, "customers")
{
}

Register with builder.Services.AddScoped<ILibrarianRepository<Customer>, CustomerRepository>(), then inject ILibrarianRepository<Customer>.

await repository.AddItem(new Customer { Id = "customer-1", Email = "alex@example.com" });
Customer? customer = await repository.GetItem("customer-1");

Typed repositories also expose explicit index methods. Batch additions and updates execute sequentially; they are not atomic transactions.

Performance

100,000 documents, warm indexes, .NET 10.0.12 Release. Librarian LINQ, LiteDB 5.0.21 native expression queries, and sqlite-net-pcl 1.11.285 all use in-memory storage. Query construction and execution are included. Times are medians over seven rounds.

Managed bytes allocated per operation:

SQLite native allocations are excluded. sqlite-net uses its synchronous expression API except for existence checks, which use parameterized SQL SELECT EXISTS. These measurements cover warm reads; index construction, writes, persistence, and concurrency are outside the comparison.