Hmz.Core.Storage
0.1.0
See the version list below for details.
dotnet add package Hmz.Core.Storage --version 0.1.0
NuGet\Install-Package Hmz.Core.Storage -Version 0.1.0
<PackageReference Include="Hmz.Core.Storage" Version="0.1.0" />
<PackageVersion Include="Hmz.Core.Storage" Version="0.1.0" />
<PackageReference Include="Hmz.Core.Storage" />
paket add Hmz.Core.Storage --version 0.1.0
#r "nuget: Hmz.Core.Storage, 0.1.0"
#:package Hmz.Core.Storage@0.1.0
#addin nuget:?package=Hmz.Core.Storage&version=0.1.0
#tool nuget:?package=Hmz.Core.Storage&version=0.1.0
Hmz.Core.Storage
Hmz.Core.Storage stores application files in an S3-compatible object store
(MinIO or AWS S3): uploads of known or unknown length, downloads, deletes,
presigned read URLs, public URLs, and a HybridCache-backed cache that reuses
signed read URLs. It is opt-in and does not depend on SharedKernel, so an
application without file storage does not take the AWS SDK. It grew out of the
Monosbase Media module.
Layers
| Layer | Owner | Purpose |
|---|---|---|
Port (IMediaObjectStorage, …) |
Application UseCase | What the use cases need, with application concepts (asset ids, providers). |
| Adapter | Application Infrastructure | Implements the port over IObjectStorage and IObjectReadUrlCache. |
IObjectStorage / S3ObjectStorage |
Core | Put, get, delete, presigned read URL, public URL in one bucket. |
IObjectReadUrlCache / HybridObjectReadUrlCache |
Core | Reuses a signed URL under a caller-chosen cache key. |
AddS3ObjectStorage(...) |
Core | Options binding and validation, the S3 client configured for MinIO. |
Object keys, key prefixes, upload policies (allowed types, size limits) and cache keys stay with the application.
MinIO setup
- Create one bucket per environment (for example
production,staging,local) and one access key per environment, limited to its bucket. - If objects are served publicly, grant anonymous read on a prefix (for
example
public/) and pointPublicBaseUrlat the MinIO host or a CDN in front of it. - Private objects are read through presigned URLs (
GetReadUrlAsync).
The client uses path-style addressing ({endpoint}/{bucket}/{key}), the
us-east-1 signing region, and sends or validates checksums only when an
operation requires them, which MinIO needs.
Configuration
{
"AppStorage": {
"Endpoint": "https://minio.example.com",
"AccessKey": "<from secrets>",
"SecretKey": "<from secrets>",
"Bucket": "staging",
"PublicBaseUrl": "https://files.example.com",
"Secure": true,
"Region": "us-east-1",
"ForcePathStyle": true
}
}
| Setting | Default | Rule |
|---|---|---|
Endpoint |
— | Absolute URI of the S3 API. |
AccessKey, SecretKey |
— | Required. Keep them in secrets, never in appsettings. |
Bucket |
— | Required S3 bucket name: 3 to 63 lowercase letters, digits, dots or hyphens. |
PublicBaseUrl |
— | Absolute URI. Public URLs are {PublicBaseUrl}/{Bucket}/{key}; changing it re-points every built URL. |
Secure |
true |
false uses plain HTTP, for the client and for signed read URLs. |
Region |
us-east-1 |
Signing region. |
ForcePathStyle |
true |
Path-style addressing. |
VerifyObjectOnSign |
false |
true sends a HEAD request before signing a read URL, so a missing object returns null. |
MultipartConcurrency |
1 |
Parts of one unknown-length upload sent at once (1 to 16); each part in flight buffers up to 5 MB. |
// Infrastructure DI. Test hosts that replace storage can skip start validation.
services.AddS3ObjectStorage("AppStorage", validateOnStart: !configuration.GetValue<bool>("Testing:IsTestHost"));
services.AddHybridCache();
services.AddObjectReadUrlCache();
AddS3ObjectStorage registers singletons IAmazonS3, ITransferUtility and
IObjectStorage. The S3 client and TransferUtility are thread-safe and
built to be shared: one instance serves every concurrent request and reuses
its connections. Validation always runs when the options are first read;
validateOnStart only decides whether invalid options also fail host start.
S3StorageOptions.Validate(options, sectionName) returns the failure
messages, prefixed with the section name. They name the setting and never
contain a configured value, so they are safe to log.
ObjectKeys.IsValidPrefix(prefix) checks an application's object-key prefix:
lowercase segments of letters, digits and hyphens separated by /, without a
leading or trailing / (for example app/staging).
Uploads
await storage.PutAsync(
new ObjectUpload(key, content, detected.ContentType, contentLength, "public, max-age=31536000, immutable"),
ct);
- A known
ContentLengthis sent in onePutObjectrequest. A seekable stream is rewound first. ObjectUpload.UnknownLength(-1) sends the stream as a multipart upload (TransferUtility, 5 MB parts,MultipartConcurrencyparts at a time), because MinIO and S3PutObjectneed the length up front. Use it for request bodies streamed straight to storage.CacheControlis stored with the object and returned on direct (public) reads.- The caller's stream is never closed.
GetAsync returns null for a missing object; the caller disposes the
returned stream. DeleteAsync of a missing object succeeds.
Errors
A missing object is the only expected outcome and is null. Every other
failure (access denied, an unreachable store, a timeout) is logged once by
S3ObjectStorage with the operation, object key, bucket, upstream status, S3
error code and request id, then rethrown unchanged for the application's global
error handling. A cancelled call is neither logged nor wrapped.
Read URLs and the cache
GetReadUrlAsync(key, lifetime) signs a GET URL whose response carries
Cache-Control: private, max-age=<lifetime seconds>, immutable, so a browser
keeps the content while the URL is reused. This assumes the content of a key
never changes: store a replaced file under a new key.
Signing does not contact the store. The SDK computes an AWS Signature V4 (an
HMAC-SHA256 chain over the method, path, expiry and headers, keyed with the
secret key) and puts it in the query string; MinIO or S3 recomputes it when the
URL is opened and only then checks expiry and whether the object exists. So by
default the URL of a missing object answers 404 when used. Applications that
already know the object exists (from their own records) need nothing more; set
VerifyObjectOnSign to have a HEAD request return null for a missing object
instead.
Read URLs play video and audio in the browser: S3 and MinIO answer range
requests, so <video src="{url}"> streams and seeks. Thumbnails and poster
frames need transcoding, which belongs to the application's media pipeline.
IObjectReadUrlCache reuses one signed URL per cache key:
var url = await readUrls.GetOrSignAsync($"media:read-url:v1:{assetId}", storage, objectKey, lifetime, ct);
await readUrls.InvalidateAsync([$"media:read-url:v1:{assetId}"], ct);
- An entry lives for half of the URL lifetime, so a returned URL always has at least half of its lifetime left.
- A missing object (
null) is never cached. - The cache key is the caller's: key by a stable id (an asset id) rather than the object key when that is what callers invalidate by, and version the key when the cached shape changes.
- The overload with a
signcallback signs through the application's own port; the cached value is anObjectReadUrleither way. - Invalidate when an object stops being readable (deleted, released, cancelled). The cache does not check existence; callers verify access before asking for a URL.
- Keys are enough when the application knows which URLs to drop. To drop many
at once (every URL of an owner or folder), pass
tagswhen caching and callInvalidateTagsAsync:
var url = await readUrls.GetOrSignAsync(key, storage, objectKey, lifetime, [$"media:owner:{ownerId}"], ct);
await readUrls.InvalidateTagsAsync([$"media:owner:{ownerId}"], ct);
Build, test, and pack
dotnet build Hmz.Core.Storage/Hmz.Core.Storage.csproj --configuration Release
dotnet run `
--project tests/Hmz.Core.Storage.UnitTests/Hmz.Core.Storage.UnitTests.csproj `
--configuration Release `
-- --minimum-expected-tests 1
dotnet pack Hmz.Core.Storage/Hmz.Core.Storage.csproj `
--configuration Release `
--output <package-output>
Dependencies
AWSSDK.S3, Microsoft.Extensions.Caching.Hybrid,
Microsoft.Extensions.Logging and
Microsoft.Extensions.Options.ConfigurationExtensions. Versions are centrally
managed in Directory.Packages.props.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- AWSSDK.S3 (>= 4.0.104.1)
- Microsoft.Extensions.Caching.Hybrid (>= 10.9.0)
- Microsoft.Extensions.Logging (>= 10.0.11)
- Microsoft.Extensions.Options.ConfigurationExtensions (>= 10.0.11)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.