Webority.Retention 0.2.14

Webority.Retention

Deletes finished work rows from a product's SQL tables once they pass their retention window: webhook intake, outboxes, idempotency records, side effects, job state. The tables and their rules come from the product's storage register, so the rules live in one place and a push check can verify them.

Using it in a product

1. List the work tables in the register

.webority/data-placement.json names every table with a MAX column or that grows by appending. Entries of class workflow:<days> are trimmed:

{
  "schemaVersion": 1,
  "tables": [
    { "table": "dbo.InboundWebhookDelivery", "class": "workflow:30", "finishedAt": "ProcessedDateTimeUtc", "status": { "column": "Status", "finished": ["Processed", "Ignored"] } },
    { "table": "dbo.NotificationOutboxMessage", "class": "workflow:30", "key": "MessageId", "finishedAt": "SentDateTimeUtc" },
    { "table": "dbo.SalesLead", "class": "record" }
  ]
}

A wrong entry, or one that names a missing table or column, fails the whole run before anything is deleted.

Each table needs an unfiltered index whose first column is finishedAt (with status as its second column when the entry has one), because each run reads the oldest due rows first. A table without one fails the run before anything is deleted, and the message names the index to add.

2. Copy the register into the Functions output

<None Include="..\.webority\data-placement.json" Link="data-placement.json" CopyToOutputDirectory="PreserveNewest" />

3. Register and configure

services.AddWeborityRetention();

"Retention": {
  "RegisterPath": "data-placement.json",
  "ConnectionStringName": "WeborityDbConnection",
  "Brake": { "Mode": "DryRun" }
}

Start in DryRun, read the run log, then set Retention:Brake:Mode to Live in the app settings. No deploy is needed.

4. Call it from a timer

public sealed class RetentionFunction(RetentionRunner runner)
{
    [Function("Retention")]
    public Task RunAsync([TimerTrigger("0 15 * * * *")] TimerInfo timer, CancellationToken cancellationToken)
    {
        return runner.RunAsync(cancellationToken);
    }
}

RunAsync returns a RetentionRunReport with one result per table: Trimmed, NothingDue, DryRun, Refused or OutOfTime. When the brake refuses a table, the other tables are still trimmed, and the run then throws RetentionRunFailedException so it shows as failed.

How a run works

1. Reads every workflow:<days> entry and checks its table, columns, types and finished-time index against the live schema.
2. For each table, reads the row count from the partition metadata and the keys of up to MaxPerRun due rows, oldest first.
3. Checks the plan with the Webority.Jobs destructive-run guard (job Retention).
4. In Live, deletes in batches. Each delete repeats the due check, so a row that changed after it was planned stays.

Build, test, release

dotnet restore Webority.Retention.slnx
dotnet build Webority.Retention.slnx -c Release --no-restore -v q
dotnet test --solution Webority.Retention.slnx -c Release --no-build

A merge to main publishes to nuget.org; the version of record is VersionPrefix in Directory.Build.props.