CrystalData 0.47.0

CrystalData

CrystalData is a persistence engine for .NET. It combines snapshot files, optional journals, backups, and independently loaded storage nodes with Tinyhand serialization and ValueLink collections.

Contents

• Requirements
• Installation
• Quick start
• NativeAOT
• Configuration
• Paths and backups
• Saving and shutdown
• Journaling
• Auxiliary storage
• S3 storage
• Recovery
• Samples

Requirements

• .NET 10 SDK or later

Installation

dotnet add package CrystalData

Package Manager Console:

Install-Package CrystalData

Quick start

Define a Tinyhand-serializable data type:

using System.ComponentModel;
using Tinyhand;

[TinyhandObject]
public partial class FirstData
{
    [Key(0)]
    public int Id { get; set; }

    [Key(1)]
    [DefaultValue("Hoge")]
    public string Name { get; set; } = string.Empty;
}

Register the crystal, prepare the storage services, update the data, and shut down cleanly:

using CrystalData;
using Microsoft.Extensions.DependencyInjection;

var product = new CrystalUnit.Builder()
    .ConfigureCrystal(context =>
    {
        context.AddCrystal<FirstData>(
            new CrystalConfiguration(
                new LocalFileConfiguration("Local/FirstData.tinyhand"))
            {
                SaveFormat = SaveFormat.Utf8,
                NumberOfFileHistories = 1,
            });
    })
    .Build();

var services = product.Context.ServiceProvider;
var control = services.GetRequiredService<CrystalControl>();
var result = await control.PrepareAndLoad(useQuery: false);
if (result.IsFailure())
{
    throw new InvalidOperationException($"CrystalData initialization failed: {result}");
}

var data = services.GetRequiredData<FirstData>();
data.Id++;
data.Name = "Updated";

await control.StoreAndRip();

StoreAndRip is terminal: the CrystalControl instance cannot be used after it completes. Call it during application shutdown.

NativeAOT

CrystalData supports NativeAOT. Data types must use Tinyhand source generation and be registered through the generic AddCrystal<TData>, CreateCrystal<TData>, or GetOrCreateCrystal<TData> APIs so their closed generic forms are visible at build time.

Publish an application for a specific runtime identifier:

dotnet publish -c Release -r win-x64 --self-contained -p:PublishAot=true

The CrystalData project enables the .NET AOT and trimming compatibility analyzers, and CI publishes and runs QuickStart as a native executable.

Configuration

Each registered type has a CrystalConfiguration.

Global defaults and limits are configured with CrystalOptions:

context.SetCrystalOptions(new CrystalOptions
{
    GlobalDirectory = new LocalDirectoryConfiguration("Data"),
    DefaultBackup = new LocalDirectoryConfiguration("Backup"),
    DefaultSaveFormat = SaveFormat.Binary,
    MemoryUsageLimit = 512L * 1024 * 1024,
});

Register more than one crystal when an application has independently persisted data sets:

context.AddCrystal<FirstData>(
    new CrystalConfiguration(new GlobalFileConfiguration("First.tinyhand")));
context.AddCrystal<SecondData>(
    new CrystalConfiguration(new GlobalFileConfiguration("Second.tinyhand")));

Crystals can also be created at runtime with CreateCrystal<TData> or retrieved or created with GetOrCreateCrystal<TData>.

Paths and backups

• LocalFileConfiguration and LocalDirectoryConfiguration use absolute paths as-is. Relative paths are resolved against CrystalOptions.DataDirectory.
• GlobalFileConfiguration and GlobalDirectoryConfiguration are resolved relative to CrystalOptions.GlobalDirectory.
• EmptyFileConfiguration and EmptyDirectoryConfiguration disable the corresponding file or directory.
• S3FileConfiguration and S3DirectoryConfiguration identify objects in an S3 bucket.

Set BackupFileConfiguration for one crystal, or set CrystalOptions.DefaultBackup to derive backup locations for crystals, journals, and auxiliary storage that do not define one explicitly.

Snapshot histories provide recovery candidates when the current file is missing or invalid. Journaling requires at least one history file for every journaled crystal.

Saving and shutdown

Use the lifecycle method that matches the operation:

For a single crystal, call ICrystal.StoreData(). For independently loaded nodes, call StoragePoint<T>.AddToSaveQueue() to schedule persistence or StoragePoint<T>.StoreData() to request it directly.

Journaling

Journaling records changes to Tinyhand structural objects between snapshots. Configure a journal and use [TinyhandObject(Structural = true)] on journaled data:

[TinyhandObject(Structural = true)]
public partial class JournalData
{
    [Key(0)]
    public partial int Count { get; set; }
}

var product = new CrystalUnit.Builder()
    .ConfigureCrystal(context =>
    {
        context.SetJournal(
            new SimpleJournalConfiguration(
                new LocalDirectoryConfiguration("Data/Journal")));

        context.AddCrystal<JournalData>(
            new CrystalConfiguration(
                new LocalFileConfiguration("Data/JournalData.tinyhand"))
            {
                NumberOfFileHistories = 3,
            });
    })
    .Build();

Structural members must be compatible with Tinyhand's structural serialization rules. ValueLink collections can also be used as journaled roots.

Auxiliary storage

StoragePoint<T> keeps a child object in an independently loaded file. Configure auxiliary storage on the owning crystal:

context.AddCrystal<RootData>(
    new CrystalConfiguration(new LocalFileConfiguration("Data/Root.tinyhand"))
    {
        StorageConfiguration = new SimpleStorageConfiguration(
            new LocalDirectoryConfiguration("Data/Storage"))
        {
            NumberOfHistoryFiles = 3,
        },
    });

The type containing a storage point must be a Tinyhand structural object. StoragePoint<T> reserves Tinyhand key 0; derived storage-point types must start their own keys at 1.

Read with TryGet. Use TryLock whenever data will be changed, and dispose the returned DataScope<T> to release the lock:

using var scope = await root.Child.TryLock(AcquisitionMode.GetOrCreate);
if (scope.IsValid)
{
    scope.Data.Count++;
}

Avoid replacing a storage-point instance with Set unless instance replacement is specifically required.

S3 storage

Supply bucket credentials through IStorageKey, then use an S3 file or directory configuration:

var storageKey = services.GetRequiredService<IStorageKey>();
storageKey.AddKey(
    "my-bucket",
    new AccessKeyPair("ACCESS_KEY_ID", "SECRET_ACCESS_KEY"));

var configuration = new CrystalConfiguration(
    new S3FileConfiguration("my-bucket", "app/FirstData.tinyhand"));

Do not embed production credentials in source code. Provide them through the application's secret-management mechanism.

Recovery

PrepareAndLoad(useQuery: true) consults the registered ICrystalDataQuery when recovery requires a decision. Pass false for non-interactive startup behavior. The second argument, loadCrystals, can defer loading registered crystals while still preparing persistence services.

CrystalData checks the primary snapshot, available histories, and configured backups. For previously stored data with RequiredForLoading = true, a load failure is passed to ICrystalDataQuery; initialization fails when the query chooses to abort.

Samples

• QuickStart contains the smallest complete application.
• Advanced covers backups, dynamic configuration, journals, paths, dependency injection, save timing, and storage points.

License

CrystalData is licensed under the MIT License.