GM.Messaging.Persistence
4.13.0
See the version list below for details.
dotnet add package GM.Messaging.Persistence --version 4.13.0
NuGet\Install-Package GM.Messaging.Persistence -Version 4.13.0
<PackageReference Include="GM.Messaging.Persistence" Version="4.13.0" />
<PackageVersion Include="GM.Messaging.Persistence" Version="4.13.0" />
<PackageReference Include="GM.Messaging.Persistence" />
paket add GM.Messaging.Persistence --version 4.13.0
#r "nuget: GM.Messaging.Persistence, 4.13.0"
#:package GM.Messaging.Persistence@4.13.0
#addin nuget:?package=GM.Messaging.Persistence&version=4.13.0
#tool nuget:?package=GM.Messaging.Persistence&version=4.13.0
<p align="center"> <img src="https://raw.githubusercontent.com/gmetskhvarishvili/GM.Messaging/master/icon.png" alt="GM.Messaging" width="140" height="140" /> </p>
GM.Messaging
A lightweight messaging setup for .NET built on Wolverine and RabbitMQ.
One call wires up the message bus from configuration — connection, resilience (retry + circuit
breaker), and producer/consumer topology — with a transactional outbox/inbox for reliable,
integration events that are published at least once and handled once per consumer. Part of the GM.* family of packages. Targets net10.0.
Packages
This repository ships four packages that version and release in lockstep:
| Package | What it provides |
|---|---|
GM.Messaging |
AddGMMessaging (Wolverine + RabbitMQ wiring, resilience, topology), AddGMOutboxRelay, and the broker tools. |
GM.Messaging.Domain |
IntegrationEvent, [IntegrationEventName], IntegrationEventRegistry, and the InboxMessage / OutboxMessage entities. |
GM.Messaging.Persistence |
EF Core configurations, the transactional inbox (AddGMInbox), the inbox worker (AddGMInboxWorker), payload protection, and telemetry names. |
GM.Messaging.AspNetCore |
MapGMMessagingAdmin(): operations endpoints for the outbox, inbox and broker queues, behind authorization. |
The outbox and inbox are tested against PostgreSQL and SQL Server (leasing, concurrent claims and ordering), and SQLite for the unit tests.
Install
dotnet add package GM.Messaging
Register
using GM.Messaging;
builder.Services.AddGMMessaging(builder.Configuration);
Configure
Everything comes from the Messaging configuration section (shape matches MessagingOptions);
every value has a sensible default:
{
"Messaging": {
"ServiceName": "orders-service",
"RabbitMq": {
"HostName": "localhost",
"Port": 5672,
"VirtualHost": "/",
"UserName": "guest",
"Password": "guest",
"PublisherConfirms": true,
"UnroutedExchange": "gm.unrouted",
"UpgradeExchanges": false,
"QueueType": "Quorum",
"MaxQueueLength": 100000,
"DeadLetterRetentionHours": 336,
"UnroutedRetentionHours": 336
},
"LargeMessages": { "MaxMessageBytes": 16777216, "OffloadAboveBytes": 262144, "FileSystemPath": "/mnt/claim-checks", "RetentionDays": 14 },
"Retry": { "MaxAttempts": 5, "InitialDelayMilliseconds": 250 },
"CircuitBreaker": { "Enabled": true, "FailurePercentageThreshold": 25 },
"OutboxRelay": {
"BatchSize": 50,
"PollingIntervalSeconds": 5,
"MaxRetries": 5,
"RetryBaseDelaySeconds": 5,
"RetryMaxDelaySeconds": 300,
"LeaseSeconds": 60,
"RetentionDays": 7,
"CleanupIntervalMinutes": 60,
"WakeUpExchange": "gm.outbox.wake-up"
},
"Inbox": {
"RetentionDays": 30,
"CleanupIntervalMinutes": 60,
"BatchSize": 50,
"PollingIntervalSeconds": 5,
"MaxConcurrency": 10,
"MaxAttempts": 5,
"RetryBaseDelaySeconds": 5,
"RetryMaxDelaySeconds": 300,
"LeaseSeconds": 60,
"ConsumerNames": [],
"ConsumerName": "billing"
}
}
}
Connecting to a managed or clustered broker. Instead of HostName … Password, set RabbitMq:Uri
(amqps://user:password@host/vhost turns TLS on). RabbitMq:UseTls (with an optional TlsServerName)
enables TLS for host-based settings and moves the default port to 5671, and RabbitMq:ClusterNodes
(["mq-2", "mq-3:5673"]) lists further nodes to connect or fail over to. For a broker whose certificate
comes from a private CA, RabbitMq:TlsCaCertificatePath points at that CA's certificate (the host name must
still match; the management API over https trusts it too). For mutual TLS, RabbitMq:TlsClientCertificatePath
gives a client certificate (PFX with TlsClientCertificatePassword, or PEM with TlsClientKeyPath), and
RabbitMq:UseExternalAuthentication signs in with it instead of a password (RabbitMQ's EXTERNAL mechanism).
Wolverine's transport, the outbox wake-ups, the broker tools and the health check all use these settings.
Quorum queues. RabbitMq:QueueType: "Quorum" declares every durable queue GM.Messaging creates as a
quorum queue: the consumer queues, the dead-letter queue and the unrouted queue. Each is replicated across
the cluster's nodes, so losing a node loses no messages. RabbitMQ recommends quorum queues for anything that
must survive a failure. "Classic" keeps a queue on one node. Unset, the broker's default applies, which is
classic unless the virtual host says otherwise. Set the same value in every service, because the dead-letter
and unrouted queues are shared. A consumer binding can override it for its own queue
(ConsumerTopology:Bindings[]:QueueType). Ordered (Sequential) queues work the same on either type, and the
outbox wake-up queues are temporary and stay classic.
RabbitMQ won't change an existing queue's type. If a queue already exists as the other type, the service stops
at startup and names it instead of failing with a bare PRECONDITION_FAILED. To move it: stop the services
that use it, let it drain (or move its messages with a shovel), delete it, and start them again. They
recreate it with the new type.
Securing the broker. Use TLS for every connection, give each service its own RabbitMQ user with permissions limited to the exchanges and queues it uses (a producer can't read consumers' queues), keep the management UI off public networks, and encrypt the broker's disks. The GM.Messaging.Samples docker-compose shows TLS with a private CA and per-service users.
Optional ProducerTopology / ConsumerTopology sections declare exchange/queue routes, and the
configureWolverine callback on AddGMMessaging lets you customize Wolverine directly. A route's
MessageType can be the event's [IntegrationEventName] (it survives class renames), its full type
name, or "Namespace.Type, Assembly":
"ProducerTopology": { "Routes": [ { "MessageType": "orders.order-placed", "Exchange": "orders" } ] }
Publish an integration event
[IntegrationEventName("orders.order-placed")] // optional, see "Event names" below
public record OrderPlaced(Guid OrderId) : IntegrationEvent;
// Save the event in the same transaction as the change that raised it.
dbContext.Add(order);
dbContext.Add(OutboxMessage.From(new OrderPlaced(order.Id)));
await dbContext.SaveChangesAsync();
The outbox relay then publishes it through Wolverine.
Wire the outbox and inbox
Both work directly on your DbContext. Map the tables with OutboxMessageConfiguration<T> and
InboxMessageConfiguration<T>, then:
// Producer: the outbox in one call: administration, erasure events, and the relay (relay: true) or, in a
// process that only saves events (an API), the wake-up that makes the relays publish at once (relay: false)
builder.Services.AddGMOutbox<OutboxMessage, AppDbContext>(builder.Configuration, relay: true);
// Consumer: a scoped IInboxProcessor, plus deletion of handled rows after RetentionDays
builder.Services.AddGMInbox<InboxMessage, AppDbContext>();
TOutboxMessage / TInboxMessage can be your own subclasses of the base entities; a private
parameterless constructor is enough (use Populate(evt) in a derived outbox type's factory).
Interceptors
GM.Messaging needs a few save interceptors on the DbContext that holds outbox or inbox rows:
- the payload interceptor, for the host's registry and payload protection;
- the size guard;
- ordering, a per-key lock with the database clock;
- the outbox signal, which wakes the relays.
AddGMOutbox, AddGMOutboxAdministration, AddGMOutboxRelay and AddGMInbox attach them themselves, through EF
Core's ConfigureDbContext, pooled contexts included. Register them after AddDbContext/AddDbContextPool, so they
pick up its options' lifetime. Your DbContext registration then carries only your own settings:
services.AddDbContextPool<AppDbContext>(o => o.UseNpgsql(cs));
services.AddGMOutbox<OutboxMessage, AppDbContext>(configuration, relay: true);
UseGMMessaging(sp) still works, and interceptors already attached are skipped, so calling it as well does no harm. You
need it only for a DbContext those registrations don't know about. A context whose options are built outside
dependency injection isn't covered either: the startup check names it when a payload protector is registered.
The outbox relay
- Published means confirmed. RabbitMQ publisher confirms are on (
RabbitMq:PublisherConfirms), so a message is marked published only after the broker has accepted it; a rejected or lost publish throws and is retried. - Missing routes fail loudly. An event type with no publish route (not in
ProducerTopology:Routes) fails withNoRouteExceptionand is retried, then dead-lettered, instead of being dropped silently. - Retries with backoff. A failed publish waits
RetryBaseDelaySeconds, doubling each time up toRetryMaxDelaySeconds. - A broker outage doesn't use up retries. When the broker can't take a message, the message isn't at fault, so
the relay doesn't count an attempt. That covers an unreachable broker, a closed connection or channel, a publish
refused because a queue is full, and a timeout. It records the error and tries again after a backoff that grows
while the outage lasts, up to
RetryMaxDelaySeconds. It hands the rest of the batch back and waits for the next poll instead of spinning. So a broker restart or a network partition delays events but doesn't dead-letter them. Only failures caused by the message itself (no route, an unknown event type, a serialization error) count towardsMaxRetries. - Unroutable messages are kept. RabbitMQ drops a message published to an exchange with no
matching queue (for example before any consumer has bound one), and still confirms it. GM topology
gives every exchange it declares an alternate exchange,
gm.unrouted(RabbitMq:UnroutedExchange), so such messages land in thegm.unroutedqueue instead of disappearing. Watch that queue (the RabbitMQ UI shows its depth) and move messages back once the consumer's queue exists. - Dead-letters. After
MaxRetriesfailures the message getsDeadLetteredAtUtc, is logged at error level, and is no longer tried. Fix the cause, then requeue it throughIOutboxAdministration(see "Dead letters and backlog"). - Keeps order per key. Messages with the same ordering key are published one at a time, in the order they were raised; see "Ordered events".
- Publishes at once. A save that adds outbox messages wakes the relay instead of it waiting for its next poll,
in this process directly and in other processes over a small RabbitMQ fanout exchange
(
OutboxRelay:WakeUpExchange).AddGMOutbox(withrelay: falsein a process that only saves events) attaches this signal and GM's other save interceptors to the DbContext itself; see "Interceptors". If RabbitMQ is unreachable, relays simply poll. - Runs in several processes. Each message is leased to one relay (
LockedBy,LockedUntilUtc) with a compare-and-set update before it is published, so an API and a worker can both run the relay. A message is published twice only if a relay stops mid-publish and its lease (LeaseSeconds) runs out; the consumer's inbox absorbs that. - Keeps the table small. Published messages are deleted after
RetentionDays(0 keeps them).
The project that calls AddGMOutboxRelay needs the GM.Messaging package, not just
GM.Messaging.Persistence. If you'd rather keep the persistence layer on GM.Messaging.Persistence
alone, call AddGMOutboxRelay from the host's Program.cs, or pass a flag into your persistence
registration (AddPersistence(configuration, runOutboxRelay: true)) to pick which hosts relay.
Handle an event once: ProcessAsync
public static class OrderPlacedHandler
{
public static Task Handle(OrderPlaced message, IInboxProcessor inbox, AppDbContext db, CancellationToken ct) =>
inbox.ProcessAsync(message, () =>
{
db.Add(Invoice.For(message.OrderId));
return Task.CompletedTask; // ProcessAsync saves and commits
}, ct);
}
- The handler gets the inbox's
DbContext.AddGMInboxrecords theDbContext, andAddGMMessaginghas Wolverine take it from the message's scope (its generated code can't build aDbContextitself), sodbabove is the same instance the inbox uses. No Wolverine setup needed. - One transaction. The inbox row and everything the handler saves through the same
DbContextcommit together. Changes made elsewhere (another database, an HTTP call) aren't covered; a handler that changes nothing through thatDbContextis logged once, as a hint. - One consumer name per service. Set
Messaging:Inbox:ConsumerName(default: the entry assembly's name) and callProcessAsync(message, handler, ct)/IngestAsync(message, ct)without a name, so a typo can't create a second consumer. - Failures are retried. When the handler throws, its changes roll back, the error and attempt count are recorded on the row, and the exception reaches Wolverine, whose retry policy runs the handler again. Only a successful run marks the event handled.
- Duplicates are skipped, including two deliveries arriving at the same moment: the second waits on the first's row lock and then skips.
- Inside a transaction you opened yourself, the row joins it and you commit.
Handle it later: IngestAsync and the inbox worker
For work that shouldn't block the consumer (sending email, calling a slow API), store the event and let the inbox worker handle it:
builder.Services.AddGMInboxWorker<InboxMessage, AppDbContext>();
builder.Services.AddGMInboxHandler<UserRegistered, SendWelcomeEmail>();
// In the Wolverine handler:
public Task Handle(UserRegistered message, IInboxProcessor inbox, CancellationToken ct) =>
inbox.IngestAsync(message, "notifications", ct);
public class SendWelcomeEmail(IEmailSender email) : IInboxHandler<UserRegistered>
{
public Task HandleAsync(UserRegistered e, CancellationToken ct) => email.SendWelcomeAsync(e.Email, ct);
}
The worker leases rows like the relay (so it can run in several processes), runs up to
MaxConcurrency handlers at once, retries failures with backoff, and dead-letters a row after
MaxAttempts. InboxMessage.Requeue() makes a dead-lettered row pending again. Set ConsumerNames
to handle only some consumers' rows.
- Leases are renewed per row. A batch is claimed at once, but its rows wait for a free handler slot. So as each
handler starts, the worker renews that row's lease to a full
LeaseSeconds, and only if it still holds an unexpired lease. A row whose lease ran out while it waited is left to whichever instance claimed it, so the handler doesn't run in two places. - Size
LeaseSecondsfor your slowest handler. The lease isn't renewed while a handler runs. - No idle wait after a full batch. After a full batch the worker claims the next one straight away, so a backlog
drains at the handlers' pace. It waits
PollingIntervalSecondsonly when there was less to do.
Event names
Each row stores the event's name: its [IntegrationEventName] if it has one, otherwise its full
type name (namespace and class, without the assembly). Moving an event to another assembly is safe.
Renaming the class or its namespace strands rows already written, unless the event keeps its old name
through [IntegrationEventName("…")]. Rows written by 1.x (assembly-qualified names) still resolve.
Event versions
Payloads are read leniently (property names case-insensitive, unknown properties ignored), so adding a property is safe. For an incompatible change, raise the event's version and register an upcaster from the previous one; rows still in the outbox or inbox are converted before they're read:
[IntegrationEventName("orders.order-shipped", Version = 2)] // v2 renamed Courier to Carrier
public record OrderShipped(Guid OrderId, string Carrier) : IntegrationEvent;
services.AddGMIntegrationEvents(events => events
.AddAssembly(typeof(OrderShipped).Assembly)
.AddUpcaster<OrderShipped>(fromVersion: 1, v1 =>
{
v1["Carrier"] = v1["Courier"]?.DeepClone();
v1.Remove("Courier");
return v1;
}));
The version also travels on the wire: GM.Messaging sets Wolverine's serializer to one that sends each
event's version in the gm-event-version header, so a consumer already at version 2 upcasts a version-1
event a not-yet-upgraded producer sent. The outbox, the inbox and the wire use the same JSON options
(ConfigureJson above). A missing upcaster fails the message (retried, then dead-lettered) with an
error naming the step.
By default these settings live in the shared, process-wide IntegrationEventRegistry.Shared. Running the
same AddGMIntegrationEvents setup in two hosts of one process, as integration tests often do, is fine;
registering a different upcaster for the same step, or different JSON options, throws instead of
silently changing how the other host reads events. A host that needs settings of its own gets its own
registry, which the relay, the inbox, the inbox worker and the wire serializer then use. Rows are written
with it too: OutboxMessage.From(registry, evt), or the usual OutboxMessage.From(evt) on a DbContext with
GM's interceptors (attached by AddGMOutbox/AddGMInbox), whose payload interceptor re-encodes such rows with the host's settings before they're saved.
Without the interceptor the host logs a warning at startup:
services.AddGMIntegrationEvents(e => e.ConfigureJson(o => o.PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower), isolated: true);
// in a command handler, with IntegrationEventRegistry injected
db.Add(OutboxMessage.From(registry, new OrderPlaced(orderId)));
Protecting payloads
Outbox and inbox rows hold the event as JSON. To keep personal data out of the database in clear text,
register a protector; AddGMOutbox and AddGMInbox add GM.Messaging's interceptor to their DbContext. The relay and the inbox worker read
protected rows back, and rows written before protection was on stay readable. The broker still carries
the plain event, which consumers need. A DbContext with outbox or inbox rows but without the interceptor (one built outside dependency
injection) stops startup when a protector is registered, rather than storing payloads in clear text.
services.AddGMPayloadDataProtection() // ASP.NET Core Data Protection
.PersistKeysToDbContext<KeysDbContext>(); // keys every instance can reach
services.AddDbContext<AppDbContext>(o => o.UseNpgsql(cs)); // AddGMOutbox / AddGMInbox attach the interceptors
// or your own: services.AddGMPayloadProtection<MyKmsProtector>(); (IMessagePayloadProtector)
Personal data in events
First, keep it out: most events need ids, not details. UserRegistered(UserId) is enough when the
consumer that emails the user can look the address up; then the broker, dead-letter queues and old
messages hold nothing to leak or erase.
When an event must carry it (the address an OTP goes to, say), mark those properties [SensitiveData].
GM.Messaging encrypts their values (AES-256-GCM) wherever the event is written (outbox and inbox rows,
messages on RabbitMQ, so dead letters and admin tools too) and decrypts them for handlers. Ids, types and
routing stay readable, and properties of nested objects and list items can be marked too. Writing such an
event without a key ring fails instead of sending the value in clear text.
public sealed record OtpRequested([property: DataSubject] Guid UserId, [property: SensitiveData] string Destination) : IntegrationEvent;
services.AddGMSensitiveData(configuration); // Messaging:SensitiveData from a secret store
"Messaging": { "SensitiveData": { "CurrentKeyId": "2026-11", "Keys": { "2026-11": "<openssl rand -base64 32>" } } }
Every producer and consumer of these events needs the same key ring. Consumers that don't use GM.Messaging
see ciphertext for these fields. Hosts in one process that share the default registry must use the same
keys (a different key ring throws); give a host its own with AddGMIntegrationEvents(..., isolated: true).
Keeping keys out of configuration. A key's value can be dp: plus the key protected with ASP.NET Core
Data Protection, so configuration holds no raw key and only processes with the Data Protection keys can use
it. Point GM.Messaging at those keys and it uses its own Data Protection, leaving the app's alone; without
DataProtection:KeysPath it uses the app's (services.AddDataProtection(), whose keys Azure Key Vault, AWS
KMS and others can protect through their Data Protection packages).
"SensitiveData": {
"CurrentKeyId": "2026-12",
"Keys": { "2026-11": "<base64>", "2026-12": "dp:CfDJ8..." },
"DataProtection": { "KeysPath": "/run/secrets/gm-data-protection", "ApplicationName": "my-system" }
}
POST /admin/messaging/sensitive-data/keys returns a new key already in its dp: form (the raw key never
leaves the process), ready to add under Keys. In code, use SensitiveDataKeyProtection.NewProtectedKey().
Or fetch the keys from a key management service yourself: implement ISensitiveDataKeyRing and register
AddGMSensitiveData<TKeyRing>().
Rotating keys. Add a key, make it CurrentKeyId, deploy, then re-wrap subject keys (below) with
ISubjectKeyStore.RewrapAsync() or POST /admin/messaging/subject-keys/rewrap. Keep the old key while
messages encrypted with it may still be read (outbox, inbox, dead letters), then remove it.
Crypto-shredding. With subject keys, the sensitive values of an event whose [DataSubject] property
names a person are encrypted with that person's own key (stored wrapped by the key ring). Forgetting the
person deletes the key, and every copy of their data, in any message, row or backup, becomes unreadable:
handlers then see null (or the type's default) for those fields.
// A database every service that reads the personal data can reach; the database and the subject_keys table
// are created at startup when missing, so there's no DbContext or migration to write.
// The key ring and the per-person key database in one call; only the provider is yours to choose.
services.AddGMSensitiveData(configuration, subjectKeys: (db, connectionString) => db.UseNpgsql(connectionString));
// Or separately: AddGMSensitiveData(configuration) and then
services.AddGMSubjectKeys(db => db.UseNpgsql(configuration.GetConnectionString("SubjectKeys")));
await subjectKeys.ForgetAsync(userId.ToString()); // ISubjectKeyStore, e.g. on an erasure request
To keep the table in your own DbContext and migrations instead, add new SubjectKeyConfiguration("messaging")
to its model and register AddGMSubjectKeys<AppDbContext>(). When the subject is a property the event
inherits, name it on the class:
[DataSubject(nameof(UserId))] // IntegrationEvent.UserId
public sealed record UserRegistered([property: SensitiveData] string Email) : IntegrationEvent;
Every service that reads those events must reach the same subject_keys table; keys are cached for
SubjectKeyCacheSeconds (60), the longest a forgotten person's data stays readable in a running process.
The relay and the inbox worker load the keys a batch needs in one asynchronous query, and
await subjectKeys.EnsureKeyAsync(subject) in a command handler creates a new person's key before
OutboxMessage.From needs it, so writing their first event doesn't create it synchronously. The admin
endpoints expose DELETE /admin/messaging/subject-keys/{subject} for erasure requests. For another store
(a key vault, a cache), implement ISubjectKeyStore and register it instead of AddGMSubjectKeys.
Erasure that reaches every service. Deleting the key covers every event, but not the copies a service
made in its own tables (an email address it stored to send to). With AddGMSubjectErasureEvents, erasing a
subject also writes SubjectForgottenIntegrationEvent (name gm.subject-forgotten) to your outbox in the same
call. Route it like any other event, and services that keep copies delete them when they handle it:
services.AddGMSubjectErasureEvents<OutboxMessage, AppDbContext>(); // the service that takes erasure requests
await erasure.EraseAsync(userId.ToString()); // ISubjectErasure; the admin DELETE uses it too
The event is written even when the key was already gone, so retrying a half-done erasure still reaches everyone. Handle it idempotently.
Ordered events
Give events an ordering key, by implementing IOrderedIntegrationEvent or with
OutboxMessage.From(evt, orderingKey: "order-42"). The relay publishes messages with the same key one
at a time, in order, even across several relays: a message waits until the earlier ones with its key are
published. Keys don't slow each other down. To keep the order on the consumer side, handle the queue
sequentially (set "Sequential": true on its ConsumerTopology binding).
- Commit order, not build order. On PostgreSQL and SQL Server, GM's ordering interceptor makes a save that adds ordered
messages take a per-key transaction lock (
pg_advisory_xact_lock,sp_getapplock). It also stamps them with the database's clock inside that lock. So two transactions writing the same key take turns, and their events are published in the order they committed, whichever built its event first and whatever each pod's clock says.- A save outside a transaction gets one, committed with the save.
- Under a retrying execution strategy (
EnableRetryOnFailure), GM can't open one itself, so only the database clock is used. Wrap such saves instrategy.ExecuteAsync(...)withBeginTransactionAsyncto get the lock too.
- A dead letter holds its key. If a message with a key is dead-lettered, later messages with that key wait until
it's requeued (after the fix) or purged, so nothing overtakes it. The waiting messages age in the backlog, which the
health check reports. Set
OutboxRelay:DeadLettersHoldOrderingKeytofalseto let the rest of the key flow on instead.
Dead letters and backlog
IOutboxAdministration (registered by AddGMOutboxRelay, or AddGMOutboxAdministration in a
process without the relay) and IInboxAdministration (registered by AddGMInbox) list dead-lettered
messages, requeue one or all, purge them, and report the backlog (pending count, oldest pending,
dead-lettered count). The GM.Messaging.AspNetCore package maps all of it, and the broker tools below, as
HTTP endpoints for whatever the service registered. They require authorization unless you opt out:
app.MapGMMessagingAdmin(o => o.AuthorizationPolicy = "ops"); // /admin/messaging/outbox, /inbox, /broker/...
// o.AllowAnonymous = true only for local development
The endpoints carry OpenAPI names, summaries and response types, so they show up typed in Swagger, and a
queue that doesn't exist answers 404. Broker message bodies can hold personal data, so the endpoints hide
them by default ([hidden: N characters]); show them masked, or as they are, when operators need them.
Masked bodies hide what the event contract marks [SensitiveData] (found from the message type), any
ciphertext, and further property names you list:
services.AddGMBrokerMessageRedaction("notes"); // optional: extra property names to mask
app.MapGMMessagingAdmin(o => { o.AuthorizationPolicy = "ops"; o.MessageBodies = BrokerMessageBodies.Redacted; });
The GM.HealthChecks.Messaging package turns the backlog into a health check.
When Wolverine gives up on an integration event and moves it to the error queue, GM also dead-letters
its unhandled ProcessAsync inbox rows, so IInboxAdministration lists the failure next to the message in
the broker's dead-letter queue. GM's default failure policy does this; add it to failure rules of your
own with AndMarkGMInboxDeadLettered():
opts.OnException<TimeoutException>().RetryTimes(2).Then.MoveToErrorQueue().AndMarkGMInboxDeadLettered();
Such rows have RetriedByInboxWorker = false: fix the cause and replay them from the broker (below),
which hands them to the handler again. Requeueing in IInboxAdministration applies to ingested rows only.
Messages in RabbitMQ that nothing handled have their own tool, IBrokerQueueAdministration
(registered by AddGMMessaging): count and peek a queue, send dead letters (messages a consumer still
failed after Wolverine's retries, in ConsumerTopology:DeadLetterQueue) back to the queue they came
from, and move unrouted messages (in gm.unrouted) to their exchange once a consumer's queue exists.
A BrokerMessageFilter picks messages by type, original queue, routing key, payload text
(BodyContains) or age (DeadLetteredBefore), caps how many, and with DryRun only counts; messages
it leaves alone stay in the queue in their order. The tool keeps one RabbitMQ connection for all calls.
Outside dependency injection (tests, scripts), BrokerQueues.Connect(messagingOptions) gives the same tool;
dispose it when done.
app.MapGet("/admin/broker/dead-letters", (IBrokerQueueAdministration broker) => broker.PeekAsync("gm.dead-letters"));
app.MapPost("/admin/broker/dead-letters/replay", (IBrokerQueueAdministration broker, string? type, bool dryRun) =>
broker.ReplayDeadLettersAsync("gm.dead-letters", new BrokerMessageFilter { MessageType = type, DryRun = dryRun }));
app.MapPost("/admin/broker/unrouted/move", (IBrokerQueueAdministration broker) => broker.MoveAsync("gm.unrouted", "gm.events"));
Throughput, scheduling, limits and large events
Consumer throughput. ConsumerTopology:PrefetchCount sets how many unacknowledged messages RabbitMQ sends each
consumer ahead. ConsumerTopology:MaxParallelMessages sets how many a queue's handlers process at once. A binding
can override either. A Sequential queue handles one message at a time, so it can't also have MaxParallelMessages.
"ConsumerTopology": {
"PrefetchCount": 50,
"Bindings": [ { "Exchange": "gm.events", "Kind": "Topic", "Queue": "billing.invoices", "RoutingKey": "invoice.#", "MaxParallelMessages": 8 } ]
}
Scheduled events. To publish an event later (a reminder, a deadline), schedule its outbox row. It's saved in the same transaction as everything else and published at the relay's first poll after that time:
var reminder = OutboxMessage.From(new PaymentReminderDue(orderId));
reminder.PublishAt(DateTimeOffset.UtcNow.AddDays(3));
db.Add(reminder);
A scheduled event can't have an ordering key, because it would hold back every later event with that key. The
backlog counts scheduled events separately (Scheduled), and they count as late only once they're due.
Queue limits. RabbitMq:MaxQueueLength and MaxQueueBytes cap each consumer queue, and a binding can override
them (MaxLength, MaxLengthBytes). A full queue refuses new messages (x-overflow: reject-publish).
- What the relay does. With publisher confirms, the relay sees the refusal and treats it like a broker outage: the event stays in the outbox and is retried with backoff, without using up its retries, until the consumer catches up. The broker queue health check reports the full queue.
- Other queues get copies. On a topic or fanout exchange, RabbitMQ still delivers the message to the other queues
bound for it while refusing it for the full one, so each retry gives those consumers another copy. Consumers that
handle through the inbox (
ProcessAsync,IngestAsync) skip the repeats. A consumer outside GM.Messaging sees duplicates, so cap only queues whose events have a single consumer, or whose consumers deduplicate.
DeadLetterRetentionHours and UnroutedRetentionHours make RabbitMQ drop messages older than
that from those two shared queues; every service must use the same values. RabbitMQ won't change these on an
existing queue, so the startup check names a queue whose limits changed, as it does for the queue type.
Large events. An event above LargeMessages:MaxMessageBytes (16 MiB, RabbitMQ's default limit) is refused when
it's saved to the outbox, with the reason, so the request that raised it fails. Before, it would have been committed
and then refused by the broker. Events that reach the relay some other way are dead-lettered rather than retried.
For events that are legitimately large, LargeMessages:OffloadAboveBytes with FileSystemPath (a directory every
producer and consumer shares) sends their bodies through a claim check. The relay stores the body, sends only a
token in the gm-claim-check header, and consumers load the body back before handling. Bodies older than
RetentionDays (14) are deleted. For another store, such as blob storage, register a Wolverine IClaimCheckStore
instance before AddGMMessaging (services.AddSingleton<IClaimCheckStore>(store)) and leave FileSystemPath unset.
GM stores the body before the send, not during serialization. Wolverine's RabbitMQ sender copies headers before it
serializes the body, so a token added while serializing would never reach the broker.
Observability
Add the GM.Messaging source and meter (MessagingTelemetry.Name) to OpenTelemetry:
builder.Services.AddOpenTelemetry()
.WithTracing(t => t.AddSource(MessagingTelemetry.Name))
.WithMetrics(m => m.AddMeter(MessagingTelemetry.Name));
The outbox stores the W3C trace context of the request that raised each event, and the relay publishes it in that trace, so a request, its published event and the consumer's handling show up as one trace. Metrics cover published, failed and dead-lettered messages, pending counts, publish lag, handled and duplicate inbox messages, and cleanup.
The outbox and inbox metrics show work that hasn't reached the broker. gm.messaging.broker.queue.messages (tags
queue, role: consumer, dead_letter, unrouted) shows work waiting on it, and gm.messaging.broker.queue.limit
gives a capped queue's max length. That covers a slow or stopped consumer, a queue filling towards MaxQueueLength,
and dead or unrouted messages piling up. Each service reads its consumer queues and the dead-letter and unrouted queues
every RabbitMq:QueueMonitorIntervalSeconds (30 s; 0 turns it off), over the broker tools' connection, and keeps the
latest readings in BrokerQueueDepths. GM.HealthChecks' broker queue check reads the same readings.
Upgrading from 4.12
Nothing to change.
- No more
UseGMMessaging(sp).AddGMOutbox,AddGMOutboxAdministration,AddGMOutboxRelayandAddGMInboxattach GM's save interceptors to their DbContext themselves (EF CoreConfigureDbContext), so you can dropUseGMMessaging(sp)from yourAddDbContext/AddDbContextPool. Register them after the DbContext. - Keeping it is safe. If you keep
UseGMMessaging(sp), interceptors already attached are skipped.
Upgrading from 4.11
Nothing to change. New in 4.12, each replacing several registrations:
AddGMOutbox<TOutbox, TDbContext>(configuration, relay)registers outbox administration, erasure events, and either the relay or the wake-up.AddGMSensitiveData(configuration, subjectKeys: (db, cs) => db.UseNpgsql(cs))sets up the key ring and the per-person key database, onConnectionStrings:SubjectKeys.
The individual registrations still work.
Upgrading from 4.10
Fixes from an independent review. One step is needed if you use quorum queues; the rest is new behaviour:
- Quorum queues need one step. A quorum
gm.dead-lettersandgm.unroutedare now declared withx-delivery-limit: -1. RabbitMQ 4 otherwise drops a message returned to a quorum queue 20 times, and the admin tools peek by taking and returning messages. If you setQueueType: Quorumon 4.8 to 4.10, the startup check names those two queues. Drain and delete them once, and the services recreate them. - Broker outages don't use up retries. A broker outage, or a full
reject-publishqueue, defers the event without counting a retry, so it no longer dead-letters events. - A dead letter holds its key. A dead-lettered message with an ordering key now holds the key until it's requeued
(
OutboxRelay:DeadLettersHoldOrderingKey). - Ordered messages follow commit order. Saves of ordered messages take a per-key lock and use the database clock (PostgreSQL, SQL Server).
- Inbox worker. It renews each row's lease as the handler starts, and doesn't wait after a full batch.
- The size guard covers every writer.
AddGMOutboxWakeUpnow registers the save-time size guard too, so services that write events without relaying them (APIs) get it.
Upgrading from 4.9
Nothing to change. New in 4.10: broker queue depths (BrokerQueueDepths, the gm.messaging.broker.queue.messages and
.limit metrics), read every RabbitMq:QueueMonitorIntervalSeconds, for dashboards and GM.HealthChecks' broker queue
check.
Upgrading from 4.8
Nothing to change unless you use the new settings, and none of them is on by default except the size guard. That guard
refuses events above 16 MiB, which RabbitMQ refused anyway. New in 4.9 (see Throughput, scheduling, limits and large
events): PrefetchCount and MaxParallelMessages, OutboxMessage.PublishAt, queue length limits and retention, the
size guard, and claim-check offloading. MessageBacklog gained Scheduled, and its Pending and oldest age leave out
events that aren't due yet.
Upgrading from 4.7
Nothing to change: queues keep the broker's default type unless you set Messaging:RabbitMq:QueueType. New in 4.8:
quorum queues (QueueType: "Quorum", or per binding), with a startup check that names any existing queue of
another type instead of failing with PRECONDITION_FAILED. To move existing classic queues, see Quorum
queues above.
Upgrading from 4.6
Nothing to change. New in 4.7: AddGMSubjectErasureEvents<TOutboxMessage, TDbContext>() and ISubjectErasure. Erasing a subject
then also publishes SubjectForgottenIntegrationEvent (gm.subject-forgotten) through your outbox, so services delete their
own copies of the person's data. The admin DELETE /subject-keys/{subject} uses it when registered.
OutboxMessage.Create<TOutbox>(registry, evt) builds an outbox row of your own type for any event.
Upgrading from 4.5
Nothing to change. New in 4.6, each replacing code you'd otherwise write:
AddGMSubjectKeys(db => db.UseNpgsql(...))keeps subject keys in GM.Messaging's ownSubjectKeysDbContextand creates the database and table at startup. No DbContext, design-time factory or migration of your own.AddGMSubjectKeys<TDbContext>()still works for a table in your own model.[DataSubject(nameof(UserId))]on an event class marks an inherited property as the subject, with no redeclaring.Messaging:SensitiveData:DataProtection:KeysPath(andApplicationName) givesdp:keys their own Data Protection, so you don't configure the app's.POST /admin/messaging/sensitive-data/keys(orSensitiveDataKeyProtection.NewProtectedKey()) makes a newdp:key, with no tool of your own to protect one.
Upgrading from 4.4
Nothing to change. New in 4.5: re-wrapping subject keys after a rotation (RewrapAsync, admin endpoint),
key rings from Data Protection (dp: keys) or your own ISensitiveDataKeyRing, batch prefetch and
EnsureKeyAsync for subject keys, mutual TLS and EXTERNAL authentication, private-CA trust for the
management API, and redaction driven by [SensitiveData]. A host that gives the shared registry a key ring
different from another host's in the same process now fails instead of silently using the first one.
ISubjectKeyStore gained EnsureKeyAsync, PrefetchAsync and RewrapAsync with default implementations.
Upgrading from 4.3
Nothing to change unless you use the new features: [SensitiveData] / [DataSubject] with
AddGMSensitiveData and AddGMSubjectKeys (encrypted personal data in events, crypto-shredding), and
RabbitMq:TlsCaCertificatePath. Integration events now always go through the event registry on the wire
(the same JSON as before), and a message without a version header is read as the current version.
Upgrading from 4.2
Nothing to change. In 4.3:
UseGMMessaging(sp)on a DbContext adds every GM interceptor (outbox signal, per-host registry, payload protection);AddGMOutboxSignalandUseGMPayloadProtectionstill work.- Startup fails when a payload protector is registered but a DbContext with outbox or inbox rows lacks the interceptor (payloads would have been stored in clear text); it warns for a host with its own registry.
- The admin endpoints hide broker message bodies by default. Set
MessageBodies = BrokerMessageBodies.Shownfor the previous behaviour, orRedactedwithAddGMBrokerMessageRedaction(...).
Upgrading from 4.1
Nothing to change. In 4.2:
- Inbox rows are dead-lettered when Wolverine actually moves a message to its error queue, instead of by
counting attempts, so custom retry policies and redeliveries after a crash no longer throw it off.
InboxOptions.BrokerDeliveryAttemptsstays as an opt-in fallback;AddGMMessagingno longer sets it. If you replaced GM's failure policy, addAndMarkGMInboxDeadLettered()to your rules. UpgradeExchangesrestores an exchange's bindings whenRabbitMq:ManagementUriis set.- New: per-host
IntegrationEventRegistry(isolated: true), payload protection, the GM.Messaging.AspNetCore admin endpoints, andBodyContains/DeadLetteredBeforebroker filters.
Upgrading from 4.0
Nothing to change. Behaviour to know about in 4.1:
- Inbox rows of
ProcessAsynchandlers that fail on the broker's last delivery are now dead-lettered (shown byIInboxAdministrationand the backlog health check).RequeueAsync/RequeueAllDeadLetteredAsyncskip them; replay them from the broker's dead-letter queue. IBrokerQueueAdministrationgained filtered overloads; the old ones remain. If you implement the interface yourself (a test fake), implement the three new members.- Registering a different upcaster for the same event and version, or configuring different JSON options twice, now throws.
Upgrading from 3.x
No database changes. 4.0 declares every exchange in ProducerTopology / ConsumerTopology with an
alternate-exchange argument, and RabbitMQ refuses to redeclare an existing exchange with different
arguments. A service that finds such an exchange stops at startup with a message naming it. Either
delete those exchanges once (the services recreate them, and consumers rebind their queues, at startup),
set Messaging:RabbitMq:UpgradeExchanges to true in one service to have it delete and recreate them
(with Messaging:RabbitMq:ManagementUri set it restores their bindings at once; without it, restart the
consumers, which rebind, and move anything that landed in gm.unrouted back), or keep
the old behaviour by setting Messaging:RabbitMq:UnroutedExchange to "" in every producer and
consumer. Producers and consumers of the same exchange must use the same setting.
Also new in 4.0, without breaking changes for callers:
- Wolverine's default serializer is GM's (same JSON as storage, versions in a header). Payloads are still plain JSON.
- Handlers can take the inbox's
DbContextdirectly; remove anyCodeGeneration.AlwaysUseServiceLocationFor<YourDbContext>()you added (it's harmless if left). IBrokerQueueAdministration,AddGMIntegrationEvents, and cross-process wake-ups (AddGMOutboxWakeUp,OutboxRelay:WakeUpExchange).OutboxSignal.Notify()now also tells other processes; useWakeUp()to wake only this process's relay.
Upgrading from 2.x
3.0 adds columns again, so add an EF Core migration in each service that maps the tables:
outbox_messages:event_version(default 1),ordering_key, and theix_outbox_messages_orderingindex.inbox_messages:event_version(default 1).
Also changed:
IInboxProcessorhas overloads without a consumer name (custom implementations must add them).OutboxMessage.FromandPopulatetake an optionalorderingKey, andInboxMessage.Createan optionaleventVersion(recompile; source-compatible).- Payloads are written with
IntegrationEventSerializer.Optionsand read case-insensitively.
Upgrading from 1.x
2.0 adds columns to both tables, so add an EF Core migration in each service that maps them:
outbox_messages:next_attempt_at_utc,dead_lettered_at_utc,locked_by,locked_until_utc,trace_parent, and theix_outbox_messages_pendingindex.inbox_messages:attempts,ingested,next_attempt_at_utc,dead_lettered_at_utc,locked_by,locked_until_utc,trace_parent, and theix_inbox_messages_pendingindex.
Also changed:
IInboxStore<T>,IOutboxDbContext<T>andAddGMInboxProcessor<T>()are gone;AddGMInboxandAddGMOutboxRelaywork on theDbContextdirectly.IInboxProcessoris scoped (it shares the handler'sDbContext), so resolve it per message or scope, not from the root provider.- New rows store the event's full type name instead of its assembly-qualified name. Existing rows still resolve.
- Published outbox messages (after 7 days) and handled inbox rows (after 30 days) are now deleted.
Set
RetentionDaysto 0 to keep them. - Rows ingested by 1.x have
ingested = falseafter the migration, so the inbox worker ignores them. Finish them with your existing poller, or setingested = trueon them.
Local infrastructure
A docker-compose.yml is included to run PostgreSQL + RabbitMQ locally:
docker compose up gm.messaging.postgres gm.messaging.rabbitmq
Requirements
- .NET 10 SDK
- RabbitMQ (and a relational database for the outbox/inbox).
Contributing & releases
Versioning is automated from Conventional Commits — see CONTRIBUTING.md. All four packages release together under one version.
License
MIT — see LICENSE.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- GM.Messaging.Domain (>= 4.13.0)
- Microsoft.AspNetCore.DataProtection (>= 10.0.10)
- Microsoft.AspNetCore.DataProtection.Extensions (>= 10.0.10)
- Microsoft.EntityFrameworkCore (>= 10.0.10)
- Microsoft.EntityFrameworkCore.Relational (>= 10.0.10)
- Microsoft.Extensions.Configuration.Binder (>= 10.0.10)
- Microsoft.Extensions.DependencyInjection.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Hosting.Abstractions (>= 10.0.10)
- Microsoft.Extensions.Options (>= 10.0.10)
- System.Security.Cryptography.Pkcs (>= 10.0.15)
- System.Security.Cryptography.Xml (>= 10.0.15)
NuGet packages (1)
Showing the top 1 NuGet packages that depend on GM.Messaging.Persistence:
| Package | Downloads |
|---|---|
|
GM.Messaging
A lightweight messaging setup for .NET built on Wolverine and RabbitMQ. One call (AddGMMessaging) wires up the message bus with config-driven RabbitMQ connection, resilience (retry + circuit breaker) and producer/consumer topology, plus an outbox relay that retries with backoff, dead-letters, leases messages so it can run in several processes, cleans up old rows and continues traces. Part of the GM.Messaging family. |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 5.0.0 | 106 | 11/17/2026 |
| 4.14.0 | 174 | 11/17/2026 |
| 4.13.0 | 97 | 11/17/2026 |
| 4.12.0 | 97 | 11/17/2026 |
| 4.11.0 | 99 | 11/17/2026 |
| 4.10.0 | 193 | 11/17/2026 |
| 4.9.0 | 97 | 11/17/2026 |
| 4.8.0 | 100 | 11/17/2026 |
| 4.7.0 | 100 | 11/17/2026 |
| 4.6.0 | 172 | 11/17/2026 |
| 4.5.0 | 115 | 11/16/2026 |
| 4.4.0 | 134 | 11/16/2026 |
| 4.3.0 | 148 | 11/16/2026 |
| 4.2.0 | 129 | 11/16/2026 |
| 4.1.0 | 204 | 11/16/2026 |
| 4.0.0 | 123 | 11/16/2026 |
| 3.0.0 | 164 | 11/16/2026 |
| 2.0.1 | 100 | 11/16/2026 |
| 2.0.0 | 107 | 11/16/2026 |
| 1.3.1 | 131 | 11/15/2026 |