Tyto.Caching 0.1.0-alpha.11

Tyto.Caching

A cache with two layers — memory (L1) and a distributed store (L2) — where the entry, not the reader, carries its own deadlines, one factory runs per key, and an invalidation that cannot be delivered is reported rather than logged.

Packages

Getting started

builder.AddTyto(tyto => {
    tyto.AddDistributedCaching(caching => {
        caching.Configure(options => options.KeyPrefix = "prism");

        caching.AddInMemoryProvider("memory");
        caching.AddRedisProvider("redis", "localhost:6379");

        caching.AddProfile<string, ProductDto>("products", profile => {
            profile.UseKey("prod");                                         // keys read prism:prod:v1:{key}
            profile.DefaultAbsoluteExpiration = TimeSpan.FromMinutes(10);
            profile.DefaultStaleWhileRevalidateAfter = TimeSpan.FromMinutes(8);
            profile.ConfigureFactoryGuard(guard => guard
                .WithSoftTimeout(TimeSpan.FromMilliseconds(200))
                .WithStaleRetention(TimeSpan.FromMinutes(30)));
        });
    });
});

public sealed class ProductService(ICache<string, ProductDto> cache, IProductRepository repository) {
    public ValueTask<ProductDto?> GetAsync(string id, CancellationToken cancellationToken) {
        return cache.GetOrSetAsync(
            id,
            (context, token) => {
                context.AddTag($"product:{id}");
                return repository.GetAsync(id, token);
            },
            cancellationToken: cancellationToken);
    }
}

How a key is stored

{prefix}:{profile alias}:v{version}:{key}
prism:app-access:v2:7263:9912:web

The value type is not in the key. It used to be, as a namespace-qualified CLR name, which was most of a key's length and meant moving a class to another namespace emptied the cache. Instead a profile holds one key type and one value type — asking for it with a second pair throws — and KeyVersion is what you raise when the stored shape changes, so old entries are never read rather than failing to deserialize one at a time.

A key that is genuinely long can be hashed with HashKeysLongerThan; the readable prefix stays. Use it as a guard, not as a habit: a hashed key cannot be read back in redis-cli.

What an entry carries

Each entry stores when it was written, when it turns stale, when it expires, and its tags. Every reader therefore judges an entry by the options it was written with. Before this, expiry was computed from the reader's own options, so an entry written with a five-minute lifetime was served for the profile's hour, and a batch copied into memory was given one shared lifetime that resurrected the oldest entry in it.

An entry moves through three windows:

• Fresh — served as is.
• Stale (StaleWhileRevalidateAfter) — served, while exactly one background refresh replaces it.
• Expired — not served, except by the factory guard below, and only for StaleRetentionDuration.

One factory per key

Concurrent misses for one key run the factory once; the others wait on it. The factory runs on its own cancellation token, cancelled only when everyone waiting on it has gone, so one caller giving up — a cancelled request, or a soft timeout — does not cancel the work the others still need.

WithStampedeProtection() extends that across processes with a distributed lock, and re-reads the distributed layer after taking the lock. A caller that cannot take the lock in time runs the factory anyway: failing a request because a lock service is slow is worse than one duplicated read.

The factory guard

• Soft timeout — the caller is handed the expired value; the factory keeps running and its result is stored.
• Fail-safe — the factory throws and an expired value exists: it is served.
• Hard timeout — WithHardTimeout(...) cancels a factory that overruns it, and the caller gets a TimeoutException. Unlike the soft timeout this ends the work; without it a factory holds its connection for as long as whatever it waits on takes.

The first two need StaleRetentionDuration, which is what keeps an expired entry in storage to fall back on.

Invalidation

await cache.InvalidateAsync(id);                                   // this profile, one key
await cacheFactory.GetInvalidator("products").InvalidateAsync(id); // no value type needed
await tagInvalidator.InvalidateTagAsync($"app:{appId}");           // every profile, every entry tagged

An invalidation that cannot be delivered throws CacheInvalidationException. Memory is cleared first, then the distributed layer, then other replicas over the backplane; if any of those cannot be reached, the caller is told. It used to be logged and swallowed, which left the old value served for the rest of its lifetime with nothing to retry it — so invalidation driven by an event now lets the failure reach the broker, and the redelivery invalidates again.

Event-driven invalidation on an endpoint:

endpoint.InvalidateCache("products")
        .On<ProductUpdatedEvent>(e => e.ProductId);

endpoint.InvalidateCacheTags()
        .On<AppDeletedEvent>(e => $"app:{e.AppId}");

Rules are collected, not replaced: two rules for one event — a second profile, or a second endpoint — both run. Registered singly, one of them silently stopped invalidating anything.

Proactive refresh and pre-warming

In high-read systems, invalidation alone causes the next caller to take a cache miss and run the factory. Proactive refresh populates or updates entries when domain events arrive, eliminating latency spikes.

Event-driven refresh on an endpoint:

// 1. Direct payload upsert (single or batch)
endpoint.RefreshCache<ProductDto>("products")
        .On<ProductUpdatedEvent>(e => (e.ProductId, e.UpdatedProduct));

endpoint.RefreshCache<ProductDto>("products")
        .On<ProductBatchUpdatedEvent>(e => e.Items.Select(x => (x.Id, x.Dto)));

// 2. Direct payload upsert with custom entry options (e.g. custom TTL)
endpoint.RefreshCache<ProductDto>("products")
        .On<ProductUpdatedEvent>(e => (e.ProductId, e.UpdatedProduct, new CacheEntryOptions {
            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(30)
        }));

// 3. Factory-driven pre-warming (asynchronous reload via scoped services)
endpoint.RefreshCache<ProductDto>("products")
        .On<ProductPriceChangedEvent>(
            e => e.ProductId,
            async (services, key, ct) => {
                var repo = services.GetRequiredService<IProductRepository>();
                return await repo.GetByIdAsync(key, ct);
            });

• Factory returns null: If the factory returns null (e.g. the entity was deleted or is no longer accessible), the existing entry is automatically evicted (InvalidateAsync).
• Batched writes: When returning an IEnumerable of entries, writes are grouped by entry options and committed via SetManyAsync in a single round trip per layer.
• Failures bubble up: Just like invalidation, refresh errors are not swallowed, allowing your message broker to retry the event on transient errors.

Tags

An entry can carry tags; invalidating a tag stops every entry carrying it from being served, in every profile sharing the distributed layer. Nothing is deleted: the moment of invalidation is recorded and entries are judged against it as they are read — scanning for tagged keys would be a KEYS over the whole cache, and the entries would still sit in other processes' memory afterwards.

TagCheckInterval (default 5s) is how long a process trusts what it last read about a tag. With a backplane, other processes hear a tag invalidation at once.

The backplane

WithBackplane() tells other replicas to drop a key from memory when it is written or invalidated here. Only invalidation used to be announced, so a SetAsync left every other replica serving the old value from its own memory until it expired.

Resilience

Every distributed call goes through one guarded path with the profile's circuit breaker. A failing or open layer is a miss for reads and a skipped write — never an error for the caller — while an invalidation that cannot reach it throws, because silence there means stale data.

Observability

Spans come from the Tyto.Caching activity source and counters from the Tyto.Caching meter, tagged with the profile. The key is deliberately not a tag: traces are exported to a wider audience than the cache, and a cache key routinely names a tenant, a recipient or a session.