HostLoom.Valkey 0.17.0

HostLoom.Valkey

Standalone Valkey caching and coordination locks over ValkeyDotNet 1.1.0. Targets .NET 10. Applications continue to consume ICache and IDistributedLock from the backend-neutral kernels. The Redis adapter remains separately available.

using System.Text.Json;
using HostLoom.Caching.DependencyInjection;
using HostLoom.Locking.DependencyInjection;
using HostLoom.Valkey;
using ValkeyDotNet;

builder.Services.AddHostLoomCaching(options => options.Namespace = "catalog-worker")
    .UseValkey(options =>
    {
        options.Connection = new ValkeyClientOptions
        {
            Host = "localhost",
            Port = 6379,
            // Configure UseTls and ACL credentials for your deployment.
        };
    })
    .UseSystemTextJson(new JsonSerializerOptions
    {
        TypeInfoResolver = CatalogJsonContext.Default,
    })
    .AddHealthChecks();
builder.Services.AddHostLoomLocking(options => options.Namespace = "catalog-worker")
    .UseValkey()
    .AddHealthChecks();

CatalogJsonContext is an application-owned source-generated JsonSerializerContext containing its cached types. A runnable example is in examples/HostLoom.Examples.ValkeyAot in the repository.

Both registrations share one lazily connected ValkeyConnection, owned by the container. Configuration callbacks run in registration order; configure the endpoint once. Public constructors also work without a container: create ValkeyConnection, then ValkeyCacheStore, ValkeyLockProvider, and optionally ValkeyCacheInvalidationChannel. Stores and providers borrow the connection; dispose channels before disposing the connection. Connection settings are snapshotted at construction. Command recovery reuses the configured TLS, ACL and database settings. A standalone ServiceCollection needs a DI container implementation to call BuildServiceProvider.

Cache behavior

• Binary payloads and fully prefixed keys pass through unchanged. The kernel owns serialization, key domains, L1, single-flight, failure metrics and fail-open behavior.
• Lua reads the value and remaining TTL together. Bulk reads pipeline these scripts in one call.
• Untagged writes use SET PX; conditional writes use SET NX PX. Tagged writes atomically add memberships and extend tag expiry only when the value is written. Lua runtime errors do not roll back partial writes; reserved key domains must contain the expected value/index types.
• Tag memberships are monotonic: rewriting under other tags can leave earlier memberships. Tag removal is best-effort across concurrent writers, as in the Redis adapter.
• Tag removal unlinks only members under the namespace's :cache:data: prefix. A member outside it is dropped from the index with SREM, left untouched as a key, and counted on hostloom.valkey.tag.members_rejected, so an index something else has written to heals itself.
• Pipeline server errors are checked individually and surfaced as CacheStoreException.

Tagged writes through the store's public constructor require a value key in namespace:cache:data:name and tag keys in namespace:cache:tag:name with the same namespace. The adapter validates every tag before issuing any backend command; invalid combinations throw ArgumentException, including conditional writes. RemoveByTagAsync also requires a canonical tag key. Untagged operations still accept opaque keys. Calls through ICache already use these domains. Direct store callers using custom tagged keys must migrate them to these forms; changing only the index would leave old values outside the invalidation filter.

Invalidation

Auto uses explicit Pub/Sub only. Tracking and Broadcast are rejected, since server-assisted tracking and keyspace notification handling are not implemented. Changes made outside HostLoom are observed through expiry. The probe describes this explicit-channel fallback.

Only RemoveAsync and RemoveByTagAsync publish. SetAsync, SetIfAbsentAsync, get-or-create fills, and warmup write Valkey and the writer's own in-process tier and publish nothing, and no server-side transport reports the write either: after an overwrite, another instance that already holds the key keeps serving its old in-process copy until that copy expires. When other instances must not serve an old value, write the source of truth and then call RemoveAsync, or keep CacheEntryOptions.LocalExpiration short. A cache recognises the echo of its own publish, so the refill that follows its own removal stays cached.

The dedicated subscriber starts on the first subscription. StartAsync(token) can be used to wait for its first server acknowledgement before beginning work. The channel is {namespace}:cache:invalidate:db:{database} because Valkey Pub/Sub ignores logical database selection. Messages use a bounded versioned JSON format, separate from the Redis adapter's wire format; mixed Redis/Valkey adapters do not share invalidation messages.

Recovery uses exponential backoff from 100 ms to 30 seconds, reset only after a subscription has remained established for at least 30 seconds. Acknowledged but flapping sockets retain their increasing delay. Pub/Sub cannot replay messages lost during a disconnection, so every acknowledged subscription, including the first, hands a local flush to its observers when FlushLocalOnReconnect is enabled. This covers entries filled before the initial acknowledgement as well as reconnect gaps. Hosting does not await subscription readiness; use StartAsync(token) when work must wait for acknowledgement. The channel publishes a probe every InvalidationProbeInterval to a private subscription on the same socket. ACLs must permit the invalidation channel and its :probe:* suffix. Its independent reader and queue keep probes separate from slow handlers and invalidation queue overflow. A probe missing for CommandTimeout replaces the subscriber, so a silent socket death is noticed within InvalidationProbeInterval + CommandTimeout (35 seconds by default); TimeSpan.Zero disables probes. ProbesReceived and SubscriberResets on the channel count them. Queue overflow drops incoming messages, and with the flush disabled a disconnection loses them too; in either case L1 expiry bounds staleness; choose a suitable CacheEntryOptions.LocalExpiration. No stronger consistency is promised. A published flush is a fourth array element on the wire; an instance on an earlier package version counts it as malformed. Handler exceptions do not stop other handlers. Handlers must not block.

Messages are bounded to 1 MiB and 10 000 items per array, and every key and tag in them is validated as a cache key (at most Caching:MaxKeyLength, no whitespace or control characters); a message with one bad item is dropped whole and counted as malformed. A well-formed flush or removal from any client that may publish on the channel still clears other instances' in-process entries, so restrict PUBLISH with an ACL channel pattern such as &{namespace}:cache:invalidate:db:* to the service's own user.

The HostLoom.Valkey meter records invalidation failures, queue drops, malformed messages, and handler failures; recovery also records the kernel's resubscription counter. Warnings are limited to one per channel per 30 seconds. ValkeyDotNet connection-owner telemetry is opt-in through EnableTelemetry. Neither adapter warnings nor probe text include credentials or payloads.

Lock behavior

Acquisition is SET key owner NX PX lease. Lua release and extension compare the owner before DEL or PEXPIRE. Positive durations round up to milliseconds. Contention and owner mismatch return false; backend failures become LockProviderException; caller cancellation propagates.

Command, pipeline, and script transport failures are never automatically replayed. A lost reply may mean the operation executed, so an unconfirmed acquisition is reported as a failure. When that failure is a command timeout, the lock issues one owner-checked release for the abandoned owner, so a lease the server did grant is freed instead of waiting out its expiry; a connection failure reported as unavailable releases nothing, and its lease expires. The next independent operation can reconnect. Script-cache recovery retries only a received NOSCRIPT response.

These are coordination leases on one primary, without fencing or consensus guarantees. A primary failover can lose a lease. Database constraints, transactions, or idempotency own persisted-state correctness. Cluster routing, Sentinel, replica reads, tracking and sharded invalidation are outside this adapter's current scope.

Options and verification

FailFast=false permits startup while unavailable; later independent operations reconnect and readiness reports backend health. It does not turn lock failures into successful acquisition.

docker compose up -d --wait valkey
HOSTLOOM_REQUIRE_VALKEY=1 dotnet test --project tests/HostLoom.IntegrationTests/HostLoom.IntegrationTests.csproj -c Release -- --filter-class '*Valkey*'
dotnet publish examples/HostLoom.Examples.ValkeyAot -c Release -r linux-x64 -o artifacts/valkey-aot
artifacts/valkey-aot/HostLoom.Examples.ValkeyAot

Tests and the sample default to localhost:16379; override HOSTLOOM_VALKEY_PORT for another local port. Use your platform's RID for Native AOT (for example osx-arm64). The sample verifies a source-generated serialized L2 round trip, leases and explicit invalidation against a real server. HOSTLOOM_REQUIRE_VALKEY=1 or HOSTLOOM_REQUIRE_BROKERS=1 makes an absent listener fail tests.

Validation on 2026-09-06: 44 shared conformance cases and eight adapter cases passed on Valkey 9.1.2; the eight adapter cases also passed on 8.1.10 and 7.2.14. Binary cache and lease cases cover RESP2 and RESP3; shared conformance and invalidation use RESP3. Twelve deterministic adapter tests cover configuration, DI, error mapping, invalidation encoding, lifecycle, startup policy and non-replay after ambiguous lock writes. Source and package-only Native AOT consumers passed on macOS arm64. Those initial live runs used disposable Linux arm64 containers without TLS or persistence. TLS/ACL and restart coverage is provided separately below.

TLS/ACL and restart checks

just test-valkey-deployment builds Release and runs seven opt-in deployment cases against its own TLS-only container. Docker and OpenSSL are required on Linux or macOS; first cache the image with docker pull valkey/valkey:9.1. Use uv run --locked python scripts/test-valkey-deployment.py --no-build --image valkey/valkey:8.1 to test another cached image (7.2 is also accepted).

The temporary certificate authority is trusted only by the test clients. Certificate name and chain validation remain enabled. A restricted ACL user exercises cache and lease scripts, expiry, bulk reads and Pub/Sub; invalid trust, hostnames, credentials, keys, commands and channels are rejected. Restart cases reuse the same connection and subscriber, verify TLS/ACL/database settings on recovery, reload scripts, observe L1 expiry and lost leases, and preserve a replacement owner's lock. The runner removes only its owned container and temporary credentials/certificates.

The normal suite skips these cases without a fixture. The release workflow runs them separately as a required step. This covers server-authenticated TLS with ACL passwords and stop/start of one nonpersistent primary; mutual TLS, replica failover and cluster semantics remain unverified.

Deployment validation on 2026-09-06 passed all seven cases on each of valkey/valkey:9.1, valkey/valkey:8.1, and valkey/valkey:7.2 (21 executions, no failures or skips), using .NET 10 on macOS arm64 and Linux arm64 server containers. The complete solution gate passed 890 cases; the seven opt-in deployment cases were skipped there and verified by the separate runner above.