SoftwareDriven.Persistence.S3 3.2.0

SoftwareDriven.Persistence.S3

S3-compatible object storage provider for SoftwareDriven.Persistence. Works with AWS S3 and S3-compatible servers such as MinIO. Targets net10.0.

Installation

dotnet add package SoftwareDriven.Persistence.S3

Setup

// Register services
services.AddS3();
services.AddSingleton<S3RepositoryProvider<MyModel>>();

// Connect against AWS S3
var manager = serviceProvider.GetRequiredService<S3Manager>();
await manager.Connect("accessKey", "secretKey", "eu-central-1", "my-bucket");

S3-compatible endpoint (e.g. MinIO)

await manager.ConnectToEndpoint("http://localhost:9000", "minioadmin", "minioadmin", "my-bucket");

A MinIO instance for local development or for running the tests of this repository:

docker run -d --name sd-persistence-minio -p 9000:9000   -e MINIO_ROOT_USER=minioadmin -e MINIO_ROOT_PASSWORD=minioadmin   quay.io/minio/minio:latest server /data

The endpoint, the credentials and the bucket the tests use are configured in the S3 section of SoftwareDriven.Persistence.Test/appsettings.json.

Pre-configured client

await manager.Connect(myAmazonS3Client, "my-bucket");

Multi-instance (keyed DI)

services.AddS3("InstanceA");
services.AddS3("InstanceB");

var managerA = serviceProvider.GetRequiredKeyedService<S3Manager>("InstanceA");
var managerB = serviceProvider.GetRequiredKeyedService<S3Manager>("InstanceB");

Usage

var provider = serviceProvider.GetRequiredService<S3RepositoryProvider<MyModel>>();

// CRUD
await provider.CreateOrUpdate(new MyModel { Title = "Hello" });
var item = await provider.FindById("some-id");
var results = await provider.FindByExpression(x => x.Title == "Hello");
await provider.Delete("some-id");

// Files
await provider.StoreFile("logo.png", bytes);
var data = await provider.LoadFile("logo.png");

// Pagination
var page = await provider.GetAll(new PaginationOptions<MyModel>
{
    PageSize = 10,
    PageNumber = 0
});

Storage layout

Each model type is stored as a single JSON object in the bucket under {BasePath}/{TableName}.json (default BasePath is data). Files stored via IFileProvider are kept as individual objects under {BasePath}/files/{FileFolder}/{fileId}.

S3 as file backend of other providers

S3FileProvider implements IFileProvider on its own and can serve as the file backend of any other persistence provider, so that the models stay in their database while the files go to a bucket:

services.AddMongoDb();
services.AddSingleton<MongoDbRepositoryProvider<MyModel>>();

services.AddS3("Files");                        // the S3Manager holding the bucket
services.AddS3FileStorage(managerKey: "Files"); // files of all providers go to that bucket

Restrict the file backend to the providers of one service key by passing the instanceKey:

services.AddS3FileStorage("InstanceA", managerKey: "Files");

Alternatively the provider of a single instance can be exchanged at runtime:

provider.FileProvider = new S3FileProvider(s3Manager, logger) { BasePath = "data" };

Queries are executed in memory after loading the collection object. This keeps the full IPersistenceProvider<T> contract (expression queries, projections, element match) available, but is best suited for small to medium collections. Read-modify-write on a collection is not atomic across concurrent writers.

Features

• SupportsTotalCount: true
• SupportsPagingState: false (offset-based pagination)
• DefaultIdSource: GuidSource
• SupportsMaintenance: false (a deleted object is gone; there is nothing to compact)
• Full IPersistenceProvider<T> support including nested collection projections and element match queries

License

MIT