Endatix.Modules.Jobs 0.7.6

Endatix.Modules.Jobs

Durable background job queue for long-running tenant operations.

Installation:

dotnet add package Endatix.Modules.Jobs

Recommended Usage:

For running and hosting the Endatix Platform, Endatix.Api.Host is the recommended main package as it simplifies the installation and setup process.

dotnet add package Endatix.Api.Host

The module is registered automatically by EndatixBuilder.UseDefaults() — no host code change is needed.

What it is

The BackgroundJobs table is the queue. Enqueueing is a single insert. Retry state — attempt count, next attempt time, and the terminal statuses — lives on the job row, so no second system can disagree with it about what a job is doing.

Work that outlives a request belongs here: exporting a large submission set, delivering a webhook to one endpoint, backfilling or purging tenant data.

Module layout

The abstractions live in Endatix.Core rather than here so that assemblies which cannot reference this module — Endatix.Infrastructure, most notably — can still enqueue.

Schema

Database schema: jobs

The schema carries its own __EFMigrationsHistory, so job migrations advance independently of app-schema migrations.

Status

Pending → Processing → Completed | Failed | DeadLettered | Retrying | Canceled, and Retrying → Processing. A job cancelled before it is claimed never runs.

Failed and DeadLettered are separate because one status cannot express both "do not retry this" and "retried and gave up", and operators need to tell them apart.

Pending and Retrying both mean eligible to run at NextAttemptAt, so a single query dispatches either.

Enqueueing

var jobId = await backgroundJobQueue.EnqueueAsync(
    new BackgroundJobRequest("SubmissionExport", payloadJson, tenantId, userId),
    cancellationToken);

Use EnqueueManyAsync for fan-out — one job per webhook endpoint, say. The batch commits in a single transaction, so a partial fan-out cannot deliver to some destinations and silently drop the rest.

Writing a handler

Implement IBackgroundJobHandler and register it in DI. Routing is by JobType, and handlers may live in any assembly.

Four obligations, each invisible until it hurts in production:

1. Return a failure Result for deterministic errors; throw only for transient ones. This is the only retry signal there is. Throwing on a permanent error re-runs expensive work until the attempt budget is gone.
2. Scope every query to the job's TenantId explicitly. Outside a request the ambient tenant filter is permissive, not restrictive — a handler that queries as if it were in a request reads every tenant's data.
3. Honour the CancellationToken, or the job cannot be cancelled or time-limited.
4. Do not hold one DbContext for the length of the job. Open a scope per chunk via IServiceScopeFactory; a change tracker held for minutes accumulates every row streamed through it.

Registration

Registered via EndatixBuilder.UseDefaults() → UseModule(JobsModule.Instance). The module implements IEndatixModule and IHasDbMigrations.

Gated by Endatix:FeatureFlags:JobsModule, off by default. The module owns a DbContext and its own migrations, so registering it where nothing enqueues would create a schema no code writes to.

Whether a given process executes jobs is a separate, configuration-level question — which is what allows API and worker roles to be deployed separately from the same image.

Migrations

PostgreSQL is currently the only supported provider. With the flag off — the default — nothing is registered and other providers are unaffected. With the flag on and a different provider configured, the host fails at startup naming the constraint, rather than deferring the failure to whichever call site enqueues first.

A second provider therefore has to exist by the time a feature depends on the queue.

Persistence is nonetheless provider-split: JobsPostgreSqlDbContext derives from JobsDbContextBase and owns its migrations and model snapshot under Persistence/Migrations/PostgreSql.

This is not stylistic. EF Core keeps one model snapshot per context type, so generating two providers' migrations against a single shared context makes the second generation overwrite the first's snapshot — after which the next migration for the first provider diffs against the wrong model and emits nonsense. Adding a provider therefore means adding a derived context, its own design-time factory, its own Config/<Provider>/ entity configuration and its own migrations folder — never reusing an existing one.

Run the commands from the repository root.

PostgreSQL

dotnet ef migrations add <Name> \
  --startup-project src/Endatix.WebHost \
  --project src/Endatix.Modules.Jobs \
  --context JobsPostgreSqlDbContext \
  --output-dir Persistence/Migrations/PostgreSql

Migrations apply automatically at startup when Endatix:Data:EnableAutoMigrations is enabled.