DevelopmentHelpers.Storage.Core 11.0.0

DevelopmentHelpers.Storage.Core

A small library to simplify working with Azure Blob Storage, Azure FileShare, and local filesystem directories. It provides an easy-to-use API for uploading, downloading and syncing entire directory structures and individual files. The library supports both local and Azure storage implementations and integrates with dependency injection.

Key features

• Unified IStorage interface with implementations for Azure Blob Storage and local filesystem.
• Azure FileShare support via IAzureFileStorage implementation.
• Upload/download entire directories and containers.
• Upload large files in chunks (resumable/large-file friendly).
• Container synchronization helpers (container-to-container and directory-to-container).
• Create ZIP archives from containers.
• Easy configuration using appsettings.json and dependency injection.
• Multiple Azure auth modes via AzureAuthMode (DevelopmentStorage, ConnectionString, ManagedIdentity, UserToken).

Prerequisites

• .NET 10
• Azure Storage account (when using Azure implementations)

Configuration (appsettings.json)

Configuration is bound to DevelopmentHelpers.Storage.Core.Models.AzureConfiguration from the DevelopmentHelpers:AzureConfiguration section. The AuthMode property controls how the library authenticates.

Common structure

"DevelopmentHelpers": {
  "AzureConfiguration": {
    "ConnectionString": "<connection string or UseDevelopmentStorage=true>",
    "AuthMode": "<DevelopmentStorage|ConnectionString|ManagedIdentity|UserToken>",
    "AccountName": "<storage-account-name>",
    "TenantId": "<aad-tenant-id>",
    "ClientId": "<aad-client-id>",
    "ClientSecret": "<aad-client-secret>"
  },
  "AzureFileStorageConfiguration": {
    "ConnectionString": "<file-storage-connection-string>",
    "ShareName": "<share-name>"
  }
}

Only a subset of properties is required for each AuthMode.

AuthMode: DevelopmentStorage (Azurite)

Use the Azurite/emulator connection string:

"DevelopmentHelpers": {
  "AzureConfiguration": {
    "ConnectionString": "UseDevelopmentStorage=true",
    "AuthMode": "DevelopmentStorage"
  }
}

AuthMode: ConnectionString (production account key)

Use a full Azure Storage connection string:

"DevelopmentHelpers": {
  "AzureConfiguration": {
    "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net",
    "AuthMode": "ConnectionString"
  }
}

AuthMode: ManagedIdentity

Use a managed identity (system-assigned or user-assigned). ConnectionString is not required; AccountName is.

"DevelopmentHelpers": {
  "AzureConfiguration": {
    "AccountName": "<your-storage-account-name>",
    "AuthMode": "ManagedIdentity"
  }
}

AuthMode: UserToken (client credentials)

Use an Azure AD application (client credentials) to access storage via RBAC:

"DevelopmentHelpers": {
  "AzureConfiguration": {
    "AccountName": "<your-storage-account-name>",
    "AuthMode": "UserToken",
    "TenantId": "<your-tenant-id>",
    "ClientId": "<your-client-id>",
    "ClientSecret": "<your-client-secret>"
  }
}

Note: AuthMode can also be omitted; in that case AzureConfiguration will infer DevelopmentStorage when ConnectionString contains UseDevelopmentStorage=true, otherwise it will default to ConnectionString.

Registering services (dependency injection)

The core DI extension methods are defined in DependencyInjection.cs.

1. Register using IConfiguration

This reads the DevelopmentHelpers:AzureConfiguration section and registers IStorage with AzureStorage:

// in Program.cs / Startup.cs
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddAzureBlobStorage(builder.Configuration); // registers IStorage
builder.Services.AddAzureFileStorage(builder.Configuration); // registers IAzureFileStorage

var app = builder.Build();

You can also use the legacy combined registration if you want both IStorage and IAzureFileStorage from a single call:

builder.Services.AddAzureStorage(builder.Configuration);

2. Register using a connection string (blob only)

When you have a connection string at runtime (env var, secret store, etc.):

var builder = WebApplication.CreateBuilder(args);
var connectionString = builder.Configuration["StorageAccountConnectionString"]; // or any source

builder.Services.AddAzureBlobStorage(new AzureConfiguration
{
    ConnectionString = connectionString,
    AuthMode = AzureAuthMode.ConnectionString
});

var app = builder.Build();

3. Register using a strongly-typed AzureConfiguration

This is useful in tests or when you build config in code:

var azureConfig = new AzureConfiguration
{
    ConnectionString = "UseDevelopmentStorage=true",
    AuthMode = AzureAuthMode.DevelopmentStorage
};

builder.Services.AddAzureBlobStorage(azureConfig);

For managed identity:

var azureConfig = new AzureConfiguration
{
    AccountName = "mystorageaccount",
    AuthMode = AzureAuthMode.ManagedIdentity
};

builder.Services.AddAzureBlobStorage(azureConfig);

For user token:

var azureConfig = new AzureConfiguration
{
    AccountName = "mystorageaccount",
    AuthMode = AzureAuthMode.UserToken,
    TenantId = "<tenant-id>",
    ClientId = "<client-id>",
    ClientSecret = "<client-secret>"
};

builder.Services.AddAzureBlobStorage(azureConfig);

4. Register Azure File Storage

Using configuration:

builder.Services.AddAzureFileStorage(builder.Configuration);

Using explicit values:

var fileStorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net";
var shareName = "my-share";

builder.Services.AddAzureFileStorage(fileStorageConnectionString, shareName);

Basic usage examples

Create storage instances (examples)

// Using DI (recommended)
public class MyService
{
    private readonly IStorage _storage;
    private readonly IAzureFileStorage _fileStorage;

    public MyService(IStorage storage, IAzureFileStorage fileStorage)
    {
        _storage = storage;
        _fileStorage = fileStorage;
    }
}

// Or instantiate manually for quick testing
var azureConfig = new AzureConfiguration
{
    ConnectionString = "UseDevelopmentStorage=true",
    AuthMode = AzureAuthMode.DevelopmentStorage
};
var logger = loggerFactory.CreateLogger<AzureStorage>();
IStorage storage = new AzureStorage(azureConfig, logger);

Uploading and downloading

// Upload a single file
var fileInfo = new FileInfo("/path/to/file.txt");
var blobUrl = await _storage.UploadFileAsync(fileInfo, MimeTypes.txt, "my-container");

// Upload a Directory
await _storage.UploadDirectoryAsync(new DirectoryInfo("/path/to/folder"), "my-container");

// Download a file (by blob URL)
var downloaded = await _storage.DownloadToFileAsync(blobUrl, "/local/download/path");

// Download entire container to local folder
await _storage.DownloadContainer("my-container", "/local/download/path");

// Create a zip from a container
var zipPath = await _storage.CreateZipFromContainerAsync("my-container", "/tmp", "archive.zip");

Chunked uploads (large files)

// Upload large file in chunks
var uri = await _storage.UploadInChunks(fullFilePath, "container-name", "file-name.txt", MimeTypes.txt);

Checking blobs and containers

// Check existence
bool exists = await _storage.BlobExistsAsync(blobUrl);

// Create a container if not exists
var response = await _storage.CreateContainerAsync("container-name", publicAccess: false);

// List containers
var containers = await _storage.GetContainersAsync();

Sync helpers

The library includes helpers for synchronizing between containers or from a directory to a container.

// Container to container sync
var syncHelper = new ContainerToContainerSyncHelper(sourceStorage, targetStorage);
var (success, message) = await syncHelper.SyncSourceToTargetContainersAsync("source", "target", deleteExtraFiles: false, localTempPath: "/tmp");

// Directory to container sync helper
var directorySyncHelper = new DirectoryToContainerSyncHelper(storage);
var (blobUrls, _) = await directorySyncHelper.GetBlobUrlListExistInTargetButNotInSourceAsync(sourceDirectoryPath, "target-container");

Tips and best practices

• Use UploadInChunks for large files to avoid memory issues.
• Keep container names lowercase when interacting with Azure Blob Storage.
• When uploading directories with nested folders, use consistent path separators (/) for blob names.
• Clean up temporary local directories after operations to avoid consuming disk space.
• Prefer managed identities or AAD-based UserToken mode over account keys for production deployments.

Testing

The DevelopmentHelpers.Storage.Core.Tests project includes integration tests for all authentication modes (DevelopmentStorage, ConnectionString, ManagedIdentity, UserToken) using per-mode appsettings.*.json files in the test project. These tests exercise common flows such as uploading/downloading files, chunked uploads, container sync and zip creation.

Contributing

Contributions and issues are welcome. Please open an issue describing the problem or the feature request and submit pull requests for fixes.

License

Include your license information here (e.g., MIT) or a note about internal use.

If you need a targeted example or the exact code snippet for a specific scenario, describe the scenario and I will add a short sample.