Centeva.ObjectStorage 7.1.0-beta.1

Centeva Object Storage Library

Centeva.ObjectStorage is a .NET 8 and .NET Standard 2.0 library that provides a generic interface to local or cloud-hosted object ("blob") storage providers.

Supported providers are:

• Amazon AWS S3 (and anything compatible with its API such as MinIO)
• Google Cloud Storage
• Azure Blob Storage
• Azure Files
• Local disk

Built With

• .NET 8

Getting Started

Import the Centeva.ObjectStorage package to your projects where needed, along with the desired provider sub-packages:

• Centeva.ObjectStorage.GCP
• Centeva.ObjectStorage.AWS
• Centeva.ObjectStorage.Azure.Blob
• Centeva.ObjectStorage.Azure.File

Create an instance of StorageFactory and register the providers you require, then build an instance of IObjectStorage using a connection string:

var factory = new StorageFactory()
    .UseAwsS3Storage()
    .UseAzureBlobStorage()
    .UseAzureFileStorage()
    .UseGoogleCloudStorage();

var storage = factory.CreateConnection("provider://key1=value1;key2=value2");

In modern .NET applications, you will likely do this as part of service registration in Program.cs, obtaining the connection string from configuration. See Dependency Injection for extension methods that handle this for you.

In some cases you may prefer to instantiate storage providers directly rather than using connection strings. Each provider's constructor allows the needed parameters to be provided.

Examples

// Local Disk
var diskFromConnectionString = factory.CreateConnection("disk://path=C:\\temp\\files");
var diskFromConstructor = new DiskObjectStorage("C:\\temp\\files");

// AWS S3
var s3FromConnectionString = factory.CreateConnection("aws.s3://bucket=myfiles;accessKey=mykey;secretKey=secret");
var s3FromConstructor = new AwsS3ObjectStorage("myfiles", "regionName", "endpointUrl", "accessKey", "secret");

// Azure Blob Storage
var blobFromConnectionString = factory.CreateConnection("azure.blob://container=myfiles;accountName=myaccount;accountKey=myAccountKey");
var blobFromConstructor = new AzureBlobObjectStorage("accountName", "accountKey", "containerName");

// Azure Files
var fileFromConnectionString = factory.CreateConnection("azure.file://share=myfiles;accountName=myaccount;accountKey=myAccountKey");
var fileFromConstructor = new AzureFileStorage("accountName", "accountKey", "shareName");

// Google Cloud Storage
var googleFromConnectionString = factory.CreateConnection("google.storage://bucket=myfiles;credentialsFilePath=/path/to/creds.json");
var googleFromConnectionString2 = factory.CreateConnection("google.storage://bucket=myfiles;credentials=base64EncodedCredentialsJson");
var googleFromConstructor = GoogleObjectStorage.CreateFromCredentialsFile("bucketName", "/path/to/creds.json");
var googleFromConstructor2 = GoogleObjectStorage.CreateFromCredentialsJson("bucketName", "credentialsJsonString");

// MinIO (using AWS S3 provider)
var minioFromConnectionString = factory.CreateConnection("aws.s3://endpoint=http://localhost:9000;region=us-east-1;bucket=myfiles;accessKey=myAccount;secretKey=myPassword");
var minioFromConstructor = new AwsS3ObjectStorage("myfiles", "us-east-1", "http://localhost:9000", "myAccount", "myPassword");

Dependency Injection

The AddObjectStorage extension method registers an IObjectStorage singleton with the built-in .NET dependency injection container:

builder.Services.AddObjectStorage(config => config
    .UseAwsS3Storage()
    .UseAzureBlobStorage()
    .UseAzureFileStorage()
    .UseGoogleCloudStorage()
    .UseConnectionString(builder.Configuration.GetConnectionString("ObjectStorage")!));

Providers can also be configured directly, without a connection string:

// AWS S3
builder.Services.AddObjectStorage(config =>
    config.UseAwsS3Storage("myfiles", "us-east-1", null, "accessKey", "secret"));

// Azure Blob Storage
builder.Services.AddObjectStorage(config =>
    config.UseAzureBlobStorage("accountName", "accountKey", "containerName"));

builder.Services.AddObjectStorage(config =>
    config.UseAzureBlobStorage("accountName", "containerName", new DefaultAzureCredential()));

// Azure Files
builder.Services.AddObjectStorage(config =>
    config.UseAzureFileStorage("accountName", "accountKey", "shareName"));

builder.Services.AddObjectStorage(config =>
    config.UseAzureFileStorage("accountName", "shareName", new DefaultAzureCredential()));

// Google Cloud Storage
builder.Services.AddObjectStorage(config =>
    config.UseGoogleCloudStorageFromCredentialsFile("bucketName", "/path/to/creds.json"));

builder.Services.AddObjectStorage(config =>
    config.UseGoogleCloudStorageFromCredentialsJson("bucketName", credentialsJsonString));

You can also supply any IObjectStorage instance you have constructed yourself:

builder.Services.AddObjectStorage(config =>
    config.UseStorage(new DiskObjectStorage(@"C:\temp\files")));

Then inject IObjectStorage where you need it:

public class MyService(IObjectStorage storage)
{
    public Task<bool> ExistsAsync(StoragePath path) => storage.ExistsAsync(path);
}

If the configured provider supports optional capabilities, the corresponding interfaces are registered against the same instance, so you can inject them directly:

public class MyService(ISupportsSignedUrls signedUrls)
{
    public Task<Uri> GetUrlAsync(StoragePath path) => signedUrls.GetDownloadUrlAsync(path);
}

ISupportsSignedUrls and ISupportsMetadata are only registered when the provider actually implements them.

If your application needs more than one storage configuration, register them as keyed services:

builder.Services.AddKeyedObjectStorage("documents", config => config
    .UseAwsS3Storage()
    .UseConnectionString(builder.Configuration.GetConnectionString("Documents")!));

builder.Services.AddKeyedObjectStorage("thumbnails", config => config
    .UseAzureBlobStorage()
    .UseConnectionString(builder.Configuration.GetConnectionString("Thumbnails")!));

Then resolve them by key:

public class MyService([FromKeyedServices("documents")] IObjectStorage storage)
{
    public Task<bool> ExistsAsync(StoragePath path) => storage.ExistsAsync(path);
}

Capability interfaces are registered with the same key.

If the connection string isn't known until runtime -- for example, in a multi-tenant application where each tenant has its own storage configuration -- register an IObjectStorageFactory instead. It resolves connections from a connection string on demand rather than building a single instance at startup:

builder.Services.AddObjectStorageFactory(config => config
    .UseAwsS3Storage()
    .UseAzureBlobStorage()
    .UseAzureFileStorage()
    .UseGoogleCloudStorage());

public class MyService(IObjectStorageFactory storageFactory)
{
    public Task<bool> ExistsAsync(string tenantConnectionString, StoragePath path) =>
        storageFactory.CreateConnection(tenantConnectionString).ExistsAsync(path);
}

IObjectStorageFactory doesn't register capability interfaces, since each call to CreateConnection can return a different provider; check for them with an is pattern (e.g. storage is ISupportsSignedUrls) after resolving a connection.

MinIO and macOS Compatibility

When using MinIO or other S3-compatible storage services with custom endpoints on macOS, the library automatically uses AuthenticationRegion instead of RegionEndpoint to avoid DNS resolution conflicts with the AWS SDK. This ensures proper functionality across all platforms (Windows, Linux, and macOS) without requiring any special configuration.

TODO: Write API documentation

Contributing

Please use a Pull Request to suggest changes to this library. As this is a shared library, strict semantic versioning rules should be followed to avoid unexpected breaking changes.

Running Tests

Use the dotnet test command, or your Visual Studio Test Explorer. The cloud provider integration tests require configuration and credentials for each of the cloud providers - you can set those via User Secrets or environment variables. Bucket, container, and share names are already set in testsettings.json; only the credentials need to be supplied.

Adding a New Storage Provider

A provider needs an IConnectionFactory that builds an IObjectStorage from a parsed connection string (returning null if it doesn't recognize the provider name), plus a StorageFactoryExtensions class with a UseXStorage() registration method:

public static class StorageFactoryExtensions
{
    public static TRegistry UseXStorage<TRegistry>(this TRegistry registry)
        where TRegistry : IObjectStorageProviderRegistry
    {
        registry.Register(new XConnectionFactory());

        return registry;
    }
}

Keep it generic over TRegistry : IObjectStorageProviderRegistry exactly as shown -- a non-generic version compiles but silently breaks chaining into AddObjectStorage's UseConnectionString/UseStorage (see IObjectStorageProviderRegistry's doc comment for why). Add a test to ProviderRegistrationTests.cs exercising the same chain -- no live credentials needed, since a regression here is a compile error, not a runtime one.

If the provider also supports configuring storage directly, without a connection string, add an overload targeting ObjectStorageBuilder specifically (see the existing providers' StorageFactoryExtensions for examples) -- it doesn't need to be generic, since ObjectStorageBuilder is already the most-derived type in the hierarchy.

Deployment

This library is versioned by GitVersion. Create a Git tag for an official release (e.g., "v1.0.0"). Version numbers can be incremented via commit message using the GitVersion approaches.