Webority.Webhooks 0.6.1

Webority.Webhooks

Outbound signed webhooks for Webority products: a product's customers register HTTPS endpoints, the product publishes events in the same transaction as the state change they describe, and a dispatcher delivers them signed, with a fixed retry schedule, dead-lettering and replay. The same signing recipe verifies customers who sign their API calls to the product.

Receiving webhooks from a provider (a payment gateway, a mail service) is not in this package; that stays with the domain package that understands the provider.

Using it in a product

1. Tables and model

The package owns no DbContext. Map the four entities in the product's own context, so an event commits with the change it describes:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.ApplyWeborityWebhooks();
}

Copy the scripts from Webority.Webhooks/Schema/ (also shipped in the nupkg under content/schema/) into the product's Scripts/, in this order: WebhookEndpoint.sql, WebhookEvent.sql, WebhookDelivery.sql, WebhookDeliveryAttempt.sql. They are idempotent. SigningSecret is a plain column by decision; the database is what is secured.

Upgrading a database made from an earlier copy. Copy these guarded batches into a journaled migration of the product's own:

• From WebhookEndpoint.sql: the ALTER batches for PreviousSigningSecret and PreviousSigningSecretExpiresDateTimeUtc (secret rotation), and the one for ConsecutiveDeadLetters (INT NOT NULL, default 0; auto-disable).
• WebhookDeliveryAttempt.sql in full: a new table, so its CREATE TABLE batch and its index batch.
• From WebhookDelivery.sql: the batch that rebuilds IX_WebhookDelivery_Due with INCLUDE (ClaimedUntilDateTimeUtc) (DROP_EXISTING). The rebuild blocks writes to the table while it runs, so schedule it for a quiet moment on a large table.

Run the new WebhookEndpoint column before deploying code that maps it, or every endpoint query fails.

2. Registration

services.AddWeborityWebhooks<ProductDbContext>(new WebhookEventCatalog(new[]
{
    "order.created",
    "order.shipped"
}));

{ "Webhooks": { "SignatureHeaderName": "X-Acme-Signature" } }

SignatureHeaderName is required and carries the product's brand; startup fails without it. DispatchBatchSize (default 100) and DispatchConcurrency (default 8: endpoints sent to at once, each still one delivery at a time and in order), SecretRotationOverlapHours (default 24, see "The signature") and AutoDisableAfterDeadLetters (default 0, see "Auto-disable") are optional. Event types are lowercase dot-separated wire names, pinned as strings; webhook. is reserved for the package's own test ping.

3. Publishing

order.Ship(now);
await publisher.PublishAsync(customerKey, isLive: true, "order.shipped", order.PublicCode, new { order.TrackingNumber }, ct);
await db.SaveChangesAsync(ct); // the order, the event and its deliveries commit together

PublishAsync never saves. It fans out to the owner's active endpoints whose IsLive matches and whose subscription wants the event. Keep payloads thin, at most 256 KB of UTF-8 JSON: the receiver fetches detail over the product's API.

4. Delivering

Host the dispatcher on a timer, one fresh DI scope per run:

[Function("WebhookDelivery")]
public async Task RunAsync([TimerTrigger("0 */1 * * * *")] TimerInfo timer, CancellationToken cancellationToken)
{
    using var scope = _scopes.CreateScope();
    await scope.ServiceProvider.GetRequiredService<IWebhookDispatcher>().RunAsync(cancellationToken);
}

A Functions (isolated worker) head needs a direct PackageReference to Webority.Webhooks; transitive references drop out of the worker's closure.

• Retry schedule: 1 minute, 5 minutes, 25 minutes, 2 hours, 10 hours, then dead-lettered (six attempts).
• Lease: a run claims rows for 5 minutes with one conditional update, so several instances can run at once; a crashed run's rows return when the lease lapses. A run stops attempting before its lease could lapse mid-send.
• Batches: a run claims Webhooks:DispatchBatchSize rows at a time (default 100) and claims again while a batch comes back full and time is left, so a burst drains in one run rather than one batch per timer tick.
• An endpoint that gives no answer (timeout, refused connection, unknown host) is not tried again in the same run; its other deliveries go back to the pool untouched and the next run tries again.
• Delivered means a 2xx. Redirects are not followed. The timeout is 15 seconds.
• Destination safety: HTTPS only, no credentials in the URL, and the host must resolve only to public addresses (private, loopback, link-local, unique-local, carrier-grade NAT, multicast and reserved ranges are refused, including IPv4 carried in IPv6). Checked at create, update, activate and once per endpoint in each dispatcher run; the delivery client re-checks the address of every connection it opens. A private destination dead-letters; an unresolvable host retries.

5. Managing endpoints

IWebhookEndpointService holds the rules (URL check, known event types, ten active endpoints per owner, no duplicate active URL, a random 32-byte secret shown once) and returns FluentResults with a WebhookError carrying a stable Code and a message safe to show. Read endpoints and the delivery log with plain queries on the product's context.

A delivery is named outside the platform by its event's EventId and its endpoint's PublicId; replay with ReplayAsync(ownerKey, eventId, endpointPublicId).

6. Attempt log, metrics, auto-disable and retention

Attempt log. Every attempt writes one append-only WebhookDeliveryAttempt row in the same save as the delivery's new state: the attempt number (restarts at 1 after a replay), the HTTP status, the send duration in milliseconds, the first 512 characters of the answer body (ResponseSnippet, read for a short time only, so a stalled body costs the log its snippet and never the delivery) and the error. Read it with a plain query on the product's context.

Metrics. Subscribe by name: .AddMeter(WeborityWebhooksDiagnostics.MeterName) in an OpenTelemetry pipeline (the package only uses System.Diagnostics.Metrics and references no OpenTelemetry package). Instruments, each tagged event_type and nothing else, never the owner, endpoint or URL:

Auto-disable. WebhookEndpoint.ConsecutiveDeadLetters rises on every dead-lettered delivery and resets when the endpoint accepts one and when the endpoint is activated. Set Webhooks:AutoDisableAfterDeadLetters (0 to 1000; 0, the default, is off): when the run reaches it, the dispatcher calls the endpoint's Disable and saves it. Register an IWebhookEndpointNotifier to tell the owner; it is optional, resolved from the run's scope and skipped when absent:

public sealed class EmailEndpointNotifier : IWebhookEndpointNotifier
{
    public Task EndpointDisabledAsync(WebhookEndpoint endpoint, string reason, CancellationToken cancellationToken = default) =>
        /* look up endpoint.OwnerKey and email them; reason is safe to show */ Task.CompletedTask;
}

It is called after the run's deliveries are recorded. An exception from it is logged and does not fail the run. Counting is not atomic across several dispatcher instances: two runs on one endpoint may each count from the same starting value, so the threshold can be crossed one delivery late.

Retention. The package runs no timer. Host a job of your own and call IWebhookRetention from a fresh scope:

await scope.ServiceProvider.GetRequiredService<IWebhookRetention>()
    .PurgeAsync(DateTimeOffset.UtcNow.AddDays(-90), cancellationToken);

It deletes, in batches of 500, deliveries that are delivered or dead-lettered and were created (and, when delivered, delivered) before the cut-off, their attempts, and then events older than the cut-off that no delivery references. Pending deliveries are never touched. It returns a WebhookPurgeSummary with the three counts.

The signature

Header (named by Webhooks:SignatureHeaderName):

t=<unix-seconds>,n=<nonce>,v1=<lowercase hex HMAC-SHA256>

A header can carry more than one v1= entry, and a receiver accepts the request when any of them verifies. After RotateSecretAsync the old secret keeps signing beside the new one for Webhooks:SecretRotationOverlapHours (default 24; 0 is a hard cutover), new secret first, so an integrator switches secrets without a failed delivery. A receiver that reads exactly one v1= entry must be updated to try each (WebhookSignature.TryParseHeaderSignatures and VerifyAny do this). A header carries at most 8 entries (WebhookSignature.MaxSignatureEntries); one with more does not parse.

Signed string, newline-delimited:

<timestamp>\n<nonce>\n<METHOD>\n<target>\n<lowercase hex SHA-256 of the body>

The target is the raw request target exactly as sent on the wire: the percent-encoded path, then ? and the raw query when there is one (Uri.PathAndQuery on the sending side). It is never decoded, re-ordered or normalised, so no query parameter can be added, dropped or altered on a signed call. With no query, the target is the path. A receiver verifies against the raw request target it received (in ASP.NET Core IHttpRequestFeature.RawTarget), not a decoded Request.Path plus Request.QueryString. A receiver checks the timestamp is within 5 minutes, verifies the HMAC in fixed time, and remembers the nonce. The body is JSON:

{ "eventId": "...", "eventType": "order.shipped", "occurredDateTimeUtc": "...", "subjectPublicId": "...", "data": { } }

Delivery is at-least-once; receivers dedupe on eventId.

Verifying customers' signed API calls

services.AddWeborityWebhookRequestVerification();
services.AddScoped<IWebhookNonceStore, ProductNonceStore>(); // required; backed by a unique index

// The raw request target, path and query, exactly as received (see "The signature").
var target = HttpContext.Features.Get<IHttpRequestFeature>().RawTarget;

var result = await verifier.VerifyAsync(
    new WebhookSignedRequest(Request.Headers[options.SignatureHeaderName], Request.Method, target, body),
    keySecret, nonceScope: keyId, ct);

There is deliberately no in-memory nonce store in the main package: on more than one instance it lets a replay through. Startup fails if none is registered.

Build, test, release

Before pushing, run locally (CI does not check code):

dotnet build Webority.Webhooks.slnx -c Release -m:4
dotnet test --solution Webority.Webhooks.slnx -c Release --no-build

The version of record is Directory.Build.props VersionPrefix. A merge to main packs the solution and pushes to nuget.org; version-check.yml fails a release PR that does not raise the version.

Release notes

The version of record is VersionPrefix in Directory.Build.props. Add each change to Next release as it lands; cutting a release moves its rows into a new version entry below, newest first, and empties the section.

Next release

0.5.0

• Diagnostics console section. AddWeborityWebhooks<TContext> registers an "Outbound webhooks" section for Webority.Diagnostics (dispatch settings and active versus switched-off endpoints) and two log metrics: deliveries dead-lettered and signed requests refused. The package now depends on Webority.Diagnostics.Abstractions.
• Refused signed requests are logged. WebhookRequestVerifier logs one Warning per refusal (event 7120, the reason only) and takes an ILogger<WebhookRequestVerifier>; a host that constructs it by hand passes a logger.

0.4.0

• Signed-call verification is bounded. VerifyAny hashes the request once however many v1= entries it is given, and a header with more than 8 entries (WebhookSignature.MaxSignatureEntries) is refused as malformed. Before, an unsigned request could make the verifier hash its body once per entry.
• A dead endpoint no longer holds up the run. After an attempt gets no HTTP answer, the dispatcher leaves that endpoint's other deliveries for the next run instead of waiting out the timeout on each.
• A run drains more than one batch. The dispatcher claims again while batches come back full and time is left before its lease.
• One destination check per endpoint per run, with a 5-second limit on the host lookup. The per-connection address check is unchanged.
• Released claims on failure. A run whose save fails releases the deliveries it claimed but did not finish, instead of leaving them locked until the lease lapses.
• The due index includes the lease column. A database made from an earlier copy needs the new converge batch from WebhookDelivery.sql in a journaled migration of the product's own (see "Upgrading a database made from an earlier copy"); it rebuilds IX_WebhookDelivery_Due with INCLUDE (ClaimedUntilDateTimeUtc).

0.3.0

• Attempt history. A new append-only table, WebhookDeliveryAttempt, records every attempt (number, HTTP status, duration, a 512-character response snippet, the error). Run its script, Schema/WebhookDeliveryAttempt.sql, and the ConsecutiveDeadLetters batch from Schema/WebhookEndpoint.sql before deploying the code, or the dispatcher and every endpoint query fail.
• Metrics. WeborityWebhooksDiagnostics publishes webhooks.delivered, webhooks.failed, webhooks.dead_lettered and webhooks.attempt.duration; subscribe with .AddMeter(WeborityWebhooksDiagnostics.MeterName).
• Auto-disable. Webhooks:AutoDisableAfterDeadLetters (default 0, off) disables an endpoint after that many consecutive dead letters; register an IWebhookEndpointNotifier to tell the owner.
• Retention purge. IWebhookRetention.PurgeAsync(cutoff) deletes settled deliveries, their attempts and unreferenced events. Host the job yourself.

0.2.0

• Secret rotation overlap. After RotateSecretAsync the old secret keeps signing beside the new one for Webhooks:SecretRotationOverlapHours (default 24; 0 is a hard cutover). The signature header can carry two v1= entries, so a receiver that reads exactly one must be updated to try each.
• Parallel dispatch. Webhooks:DispatchConcurrency (default 8) sends to that many endpoints at once, each still one delivery at a time and in order.
• Two WebhookEndpoint columns: PreviousSigningSecret and PreviousSigningSecretExpiresDateTimeUtc. Run the ALTER batches from Schema/WebhookEndpoint.sql before deploying code that maps them.