DoIt.S3Client 3.0.0

do IT S3 Client

A simple client for S3 (Simple Storage Service) compatiable services, with support for uploading objects of unknown size.

Why on earth would I want to upload objects of unknown size?!

Well, you might, for example, want to compress or encrypt (or both!) a file/stream on the fly while uploading it to you S3 bucket.

Good point! How would I do that?

using DoIt.S3Client;

// Create a client, the main interface to you S3 bucket.
using var client = new S3Client(new Uri("https://your-bucket-endpoint"), "your-region", "access-key", "secret-key");

// Open a stream for uploading data of (beforehand) unknown size, such as a ZIP archive created on the fly.
using (var zip = new ZipArchive(await client.OpenObjectForWritingAsync("test.zip", "application/zip"), ZipArchiveMode.Create))
{
    // Dynamically add data to you ZIP archive until you've written all you need to write.
    var entry = zip.CreateEntry("test.txt");
    using var writer = new StreamWriter(entry.Open());
    await writer.WriteLineAsync("Hello, this is a test object.");
}

How does it work?

The client initiates a multipart upload when an S3 stream is created. The "current" part is held in memory and sent to your S3 bucket only when either the current part reaches the maximum part size or the stream is closed/disposed.

Even if you upload very large files to your S3 bucket, only a single part will be held in memory at any time.

What other operations does the client support?

The primary rationale for this client is to be able to upload objects of unknown sizes in a simple way. The support for other operations is rather limited. The only supported operations are

• Upload an object (OpenObjectForWritingAsync())
• Download an object (OpenObjectForReadingAsync())
• Get an object's metadata (GetObjectMetadataAsync())
• Delete an object (DeleteObjectAsync())

Specifically, bucket operations (for example, creating a bucket) is not supported by this client.

Observability?

This library uses .NET's ActivitySource to create traces available to anyone who is listening. This hopefully means you can use your application's existing observability/tracing integration (using OpenTelemetry or some other, proprietary, tooling) and simply complement that with traces from this library as well. All you need to do is start listening for traces from the ActivitySource with the name defined in the constant DoIt.S3Client.S3Client.ActivitySourceName. Depending on how you bootstrap your observability this might look something like:

services
    .AddOpenTelemetry()
        .WithTracing(tracing => tracing
            // Add traces from this library!
            .AddSource(DoIt.S3Client.S3Client.ActivitySourceName);
            // Add a bunch of other sources/configuration.
            // .
            // .
            // .
            // Add an exporter.
            .AddOtlpExporter());

Note that this library currently only produces traces. No logs and no metrics.