| Clean Architecture Toolkit for .NET | |||||
|
Clean Architecture Toolkit is a clean, modular architecture foundation for .NET applications. It embraces SOLID principles and focuses on maintainability, testability, and scalability through clear boundaries and pluggable design. |
|
||||
|
| HOME | REPO OVERVIEW | DEMO | |
|||||
Cat.EventProcessor is the receiving side of the messaging modules. Not all work comes in over HTTP. A message on the bus is work that starts on its own. The module listens to the bus and turns every incoming message into a service request that your core handles. The worker holds no business logic.
The message is the service request. There is no mapping step. You write one ICatServiceRequestHandler for the message type, like for any other service request, and the decorators of Cat.ServiceRequests (logging, validation, tracing) work for it too.
The module settles every message for you. A handler that succeeds completes the message. A failure that will not change dead letters it, and a failure that may go away abandons it, so the bus delivers it again. Azure Service Bus is the first broker. Every broker gets its own project.
- NuGet packages
- Namespaces
- Dependencies
- Usage
- Declare the message types
- Write the handler
- Register the module
- Settings per message type
- What happens to a message
- Create the subscriptions
- Change the options of Azure Service Bus
- Try a message in development
- Add the health check
- The inbox
- Competing consumers and shutdown
- Traces
- Who sent the message and what may it do
- What is not built yet
| Package | Role | Install | Downloads |
|---|---|---|---|
| Cat.EventProcessor.Abstractions | Contracts | dotnet add package gmarginet.Cat.EventProcessor.Abstractions |
|
| Cat.EventProcessor | Implementation | dotnet add package gmarginet.Cat.EventProcessor |
|
| Cat.EventProcessor.Authorization | Optional token and policy check | dotnet add package gmarginet.Cat.EventProcessor.Authorization |
|
| Cat.EventProcessor.AzureServiceBus | Azure Service Bus listener | dotnet add package gmarginet.Cat.EventProcessor.AzureServiceBus |
|
| Cat.EventProcessor.Minimal.Api | Optional development endpoint | dotnet add package gmarginet.Cat.EventProcessor.Minimal.Api |
|
| Cat.EventProcessor.Inbox.EFCore | Optional inbox for EF Core | dotnet add package gmarginet.Cat.EventProcessor.Inbox.EFCore |
| Namespace | What it holds |
|---|---|
| Cat.EventProcessor.Abstractions | The incoming message, the settlement, the receiver, the contract of the inbox store and the settings per message type. |
| Cat.EventProcessor | The receiver, the failure table, the builder for the message types and the module registration. |
| Cat.EventProcessor.Authorization | The token validation, the user of the message and the policy check. |
| Cat.EventProcessor.AzureServiceBus | The Azure Service Bus listener, the health check and the creation of the subscriptions. |
| Cat.EventProcessor.Minimal.Api | The development endpoint that simulates an incoming message. |
| Cat.EventProcessor.Inbox.EFCore | The inbox table and its store. |
| Module | Why Cat.EventProcessor needs it |
|---|---|
| Cat.Messaging | The message types, the name convention and the Azure Service Bus connection. |
| Cat.ServiceRequests | The message is the service request, and its handler runs through the service request processor. |
| Cat.Responses | The response of the handler decides what happens to the message. |
| Cat.Users | Only the optional authorization project. It sets the user of the message. |
A message type that the service receives is a message and a service request at the same time. The Contracts project of the sender references Cat.Messaging.Abstractions and Cat.ServiceRequests.Abstractions:
using Cat.Messaging.Abstractions;
using Cat.ServiceRequests.Abstractions;
public class OrderPlaced : CatEventMessage, ICatServiceRequest
{
public Guid OrderId { get; set; }
public string OrderNumber { get; set; } = string.Empty;
}Every service that receives the message writes its own handler. An event with several receivers works, because every service has its own handlers:
public class OrderPlacedHandler : ICatServiceRequestHandler<OrderPlaced>
{
public async Task<ICatResponse> HandleAsync(OrderPlaced message, CancellationToken cancellationToken = default)
{
await ReserveStockAsync(message.OrderId, cancellationToken);
return CatResponse.Ok();
}
}Write the handler so that it can run twice for the same message. The bus delivers a message at least once. See The inbox for a handler that cannot do that.
The module needs no project of its own. The API project of the service registers it with extension methods. It declares the message types, registers the service request module, the connection to the bus and the listener. The subscription name is the name this service uses on the topic of every event:
using Cat.EventProcessor.AzureServiceBus.Extensions;
using Cat.EventProcessor.Extensions;
using Cat.Messaging.AzureServiceBus.Extensions;
using Cat.Messaging.Extensions;
using Cat.ServiceRequests.Extensions;
builder.Services.AddCatServiceRequestsModule(typeof(OrderPlacedHandler));
builder.Services.AddCatMessagingModule();
builder.Services.AddCatMessagingAzureServiceBus(options =>
options.ConnectionString = builder.Configuration.GetConnectionString("servicebus")!);
builder.Services.AddCatEventProcessor(builder.Configuration, processor => processor
.Receive<OrderPlaced>()
.Receive<ReserveStock>(options => options.MaxConcurrentCalls = 4));
builder.Services.AddCatEventProcessorAzureServiceBus(options => options.SubscriptionName = "inventory");An event is read from the subscription of the service on its topic. A command message is read from its queue. The names follow the convention of Cat.Messaging, so [CatMessage(Name = "...")] works on the receiving side too. The listener starts one Azure Service Bus processor for every declared message type, and stops them together.
Set them in code, as in the example above. Whoever runs the service can also set them in the configuration, and the configuration wins, so no build is needed:
{
"CatEventProcessor": {
"Messages": {
"OrderPlaced": {
"MaxConcurrentCalls": 8,
"FailureOverrides": { "NotFound": "Complete" }
}
}
}
}MaxConcurrentCalls: how many messages of this type run at the same time. The default of the bus applies when it is empty. For a message type that needs sessions it is the number of sessions at the same time. A session handles one message at a time, so it is still the number of messages at the same time. The default of the Azure Service Bus SDK is 8 sessions.Policy: the authorization policy of this message type, see the chapter about who sent the message.FailureOverrides: what to do with a failed response of a given type. See the table below.
The listener turns the response of the handler into an action on the message:
| The handler returns | What the listener does | Why |
|---|---|---|
| Succeeded | Completes the message. | Done. |
BadRequest, NotFound, Conflict, Forbidden or Unauthorized |
Dead letters it at once, with the error codes as the reason and the error messages as the description. | Trying again gives the same answer. |
InternalServerError, or an exception |
Abandons it, so the bus delivers it again. After the maximum number of deliveries the bus dead letters it. | The cause may go away, such as a database that is down for a moment. |
A message that fails the authorization is dead lettered with the reason Unauthorized or Forbidden, see the chapter about who sent the message. A body that is not JSON, or does not fit the message type, is dead lettered with the reason InvalidMessage. A content type other than application/json is dead lettered with the reason UnsupportedContentType.
The bus alone retries. The module adds no retry loop of its own, because two retry layers multiply. The bus uses its own settings, with a default of 10 deliveries. It redelivers at once, so a dependency that is down gets hit up to ten times in a second.
Override the table per message type with FailureOverrides. The key is the name of the response type, and the value is Complete, Abandon or DeadLetter.
Nothing is created when the service starts. A subscription belongs to the service that receives the message, so the service creates its own. The topic comes from the sender, see Cat.Messaging. The topic must exist already:
var topology = app.Services.GetRequiredService<ICatEventProcessorTopology>();
var created = await topology.EnsureSubscriptionsAsync();It creates the subscription of this service on the topic of every declared event, and skips what exists. It needs the manage right on the namespace, so call it from a deployment step or a test, and not from the service itself. A sample with the Azure CLI:
az servicebus topic subscription create --resource-group <group> --namespace-name <namespace> --topic-name OrderPlaced --name inventoryThe listener starts from the defaults of Azure Service Bus and sets only what it needs. You can change any other option with a callback on CatEventProcessorServiceBusOptions. The listener builds the SDK options with its own values first, and then calls your callback, so your callback wins. The callbacks for the client, the administration client, the queues and the topics are on the options of Cat.Messaging, see its module readme.
| Callback | Changes | When it runs |
|---|---|---|
ConfigureProcessor |
ServiceBusProcessorOptions |
For every message type that does not need sessions, when the listener starts. |
ConfigureSessionProcessor |
ServiceBusSessionProcessorOptions |
For every message type that needs sessions, when the listener starts. |
ConfigureSubscription |
CreateSubscriptionOptions |
For every subscription that EnsureSubscriptionsAsync creates. |
The two processor callbacks get the message type as the first argument, so you can tune one message type:
builder.Services.AddCatEventProcessorAzureServiceBus(options =>
{
options.SubscriptionName = "inventory";
options.ConfigureProcessor = (messageType, processor) =>
{
if (messageType == typeof(OrderPlaced))
{
processor.PrefetchCount = 20;
}
};
options.ConfigureSubscription = subscription => subscription.MaxDeliveryCount = 5;
});Things to know:
- The callback wins over the settings per message type. The listener sets
MaxConcurrentCalls, or the number of sessions, from the declaration and the configuration first. A callback that sets it again replaces that value. - The listener settles every message itself. It sets
AutoCompleteMessagesto false after your callback runs, so a callback cannot turn it on. The table in What happens to a message depends on it. - A subscription gets the defaults. The listener sets only the names of the topic and the subscription.
The API can map a development endpoint. It exists only in the Development environment. In any other environment nothing is mapped:
using Cat.EventProcessor.Minimal.Api.Extensions;
app.MapCatEventProcessor();GET /dev/event-processorlists the declared message types with their queue or topic.POST /dev/event-processor/{messageType}takes the JSON of a message and runs it through the same pipeline as a real message. The answer says what the listener would do with the message:Complete,AbandonorDeadLetter, with the reason.- The
Authorizationheader of the request becomes the token of the message. Send the same bearer token as for the API to try a message type with an authorization rule. Without the header the message has no token, and such a type answersDeadLetterwith the reasonUnauthorized.
Nothing is settled on the bus, because there is no real message.
The endpoints have the OpenAPI group name dev, so they are not in the default OpenAPI document. A client generated from that document, such as the one from NSwag, never gets them. To see them in an API explorer, add a document named dev and map it. That document holds the development endpoints and all the normal endpoints, and it is served at /openapi/dev.json:
builder.Services.AddOpenApi("dev");
app.MapOpenApi();A listener can stop because of a fatal error, such as a deleted queue or a missing permission, while the API still answers. The health check tells the platform:
builder.Services.AddHealthChecks().AddCatEventProcessorHealthCheck();It is unhealthy when a listener has stopped, and degraded for a minute after a listener reported an error while it keeps processing.
Use the inbox only when a handler cannot run twice for the same message. It stores the ID of a handled message in the service database, in the same SaveChanges as the work of the handler. A repeated message hits the key of the table and is skipped.
Add the Cat.EventProcessor.Inbox.EFCore package. Map the table in the context that the handlers use and register the store:
using Cat.EventProcessor.Inbox.EFCore;
using Cat.EventProcessor.Inbox.EFCore.Extensions;
public class InventoryDbContext : DbContext
{
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.AddCatInbox();
}
}
builder.Services.AddCatEventProcessorInbox<InventoryDbContext>();Add a migration afterwards, so the CatInbox table exists. Then switch the inbox on for the message types that need it:
builder.Services.AddCatEventProcessor(builder.Configuration, events => events
.Receive<OrderPlaced>(options => options.UseInbox = true)
.Receive<StockCounted>());What to know:
- Per message type.
UseInboxis false by default. Only a message type with it on gets the inbox, so you pick the types and not every handler. - Configuration. The configuration of the service wins over the code, like the other settings:
CatEventProcessor:Messages:OrderPlaced:UseInbox. - Where it runs. The receiver does the work, not a decorator. Before the handler runs it asks the store for the message ID. A repeated message completes without running the handler. A new message gets its row in the context of the handler, so the
SaveChangesof the handler saves the row with the work, or neither is saved. This works because the receiver runs the handler in its own scope, throughICatCurrentScopeServiceRequestProcessorofCat.ServiceRequests. A processor of your own that you register only asICatServiceRequestProcessoropens a scope per call, so the row would never be saved. RegisterICatCurrentScopeServiceRequestProcessortoo. - A handler that starts a command. Start it with
ICatCurrentScopeCommandProcessorofCat.Commands, so the command runs in the scope of the handler and saves the row with its work.ICatCommandProcessoropens a new scope for every command, so the command handler saves with another context than the receiver. The row of the receiver is then never saved, and a repeated message runs the handler again, with no error. WithUseInboxon, use the current scope processor, let the handler save through the context of its own scope, or write the handler so that it can run twice and leaveUseInboxoff. - Two copies at once. The key of the table fails the second save, the message is abandoned, and the next delivery finds the row and skips it.
- No message ID. A message without an ID is handled without the inbox.
- Missing store. The listener fails at startup when a message type has
UseInboxon and noICatInboxStoreis registered. - Other stores.
ICatInboxStorelives inCat.EventProcessor.Abstractions, so you can register a store of your own. - The sender helps. It sets the message ID, and duplicate detection of the queue or topic can drop a repeated message.
Several instances of the same service can listen to the same subscription, and the bus gives a message to one of them. The module never assumes one instance. The order of messages is not guaranteed, unless the message type needs sessions, see Cat.Messaging.
When the service stops, the listener stops taking new messages and finishes the messages that are running.
The trace of the sender continues in the service. The Azure Service Bus SDK reads the Diagnostic-Id of the message and starts the activity of the handling from it, so the second half of a flow joins the same trace. The module adds nothing for this. Register the Azure.Messaging.ServiceBus source in the OpenTelemetry setup of the service.
A message can carry the token of its sender. The optional project Cat.EventProcessor.Authorization reads it, sets the user of the message and checks a policy per message type before the handler runs. Without the project a message runs without a user and without a policy, as before.
builder.Services.AddAuthentication().AddJwtBearer(options =>
{
options.Authority = "https://keycloak.example/realms/shop";
options.Audience = "cat-gateway";
options.MapInboundClaims = false;
});
builder.Services.AddAuthorization(options => options.AddPolicy("PriceAdmin", policy => policy.RequireRole("admin")));
builder.Services.AddCatEventProcessor(builder.Configuration, events => events
.Receive<PriceChanged>(options => options.Policy = "PriceAdmin")
.Receive<StockCounted>(options => options.Require(policy => policy.RequireRole("warehouse")))
.Receive<OrderPlaced>());
builder.Services.AddCatEventProcessorAuthorization();How it works:
- The token. It travels in the message property
Authorization, with the valueBearer <token>, as in HTTP. The sender puts it there, see the chapter about the token inCat.Messaging. It is not part of the message body. - The validation. The receiver validates the token with the JWT bearer options of the service, so one setup serves the endpoints and the messages. Register the scheme,
Bearerby default, also when the service has no HTTP authentication. Use another scheme withAddCatEventProcessorAuthorization(options => options.AuthenticationScheme = "..."). SetMapInboundClaimstofalse, as above, so the claim names stay as they are in the token. - The lifetime. It is checked against the time the bus received the message, and not against the clock now, because a message can be hours old when it is handled. The clock skew of the JWT bearer options applies.
- Scheduled messages. A message with a scheduled time carries the token of the sending system, not of a user, because a token of a user is expired when the bus delivers the message. The receiver skips the lifetime check for such a message and checks the signature, the issuer and the audience. The cost is that a copied token of a system stays valid for scheduled messages as long as its signing key is valid.
- The user. The receiver sets
ICatUserofCat.Usersfor the scope of the message, from the claims of the token. A message without a token runs as the anonymous user, so a policy that needs a login denies it. - The policy.
Policynames a policy of ASP.NET Core.Requiretakes a callback that builds a rule with the policy builder of ASP.NET Core, so every rule of ASP.NET Core works. A message must pass the named policy and every rule. The check runs before the inbox, so a message that is not allowed leaves no trace in the inbox. A handler that needs theHttpContextdoes not work for a message. - A message that fails. A token that is not valid gets the reason
Unauthorized. A caller that is known and not allowed getsForbidden, and a message without a token that a policy denies getsUnauthorized. The message is dead lettered and not retried, because trying again gives the same answer. - At startup. A policy name that does not exist stops the service at startup, and so does a policy without
AddCatEventProcessorAuthorization. The service logs a warning with the message types that have no policy and no rule. - Message types without a rule. The default is to run them, so a service keeps working while its message types get their rules. Set
CatMessageAuthorizationOptions.DenyMessagesWithoutPolicytotrueto deny them. A message type that is open on purpose then states a rule that always passes, such asoptions.Require(policy => policy.RequireAssertion(_ => true)).
Set the policy from the configuration too, with CatEventProcessor:Messages:PriceChanged:Policy. The inline rules come from code only.
The project targets the latest .NET, because it uses the JWT bearer package of ASP.NET Core. The token stays in the message, so it is in the queue, in the dead letter queue and in the outbox table. Keep the risk small:
- Use short lived tokens.
- Keep message bodies out of the logs.
- Give the queues small rights. A person with the right to receive from a queue sees the tokens in clear.
- Azure Service Bus encrypts the data at rest with keys of Microsoft. Customer-managed keys are optional and need the Premium tier.
Let a person who knows security look at this before the module is used in production.
- The arrival time of the real service. The rule for a normal message uses the time the bus received it. The emulator sets it to the scheduled time for a scheduled message, and the rule for a scheduled message does not depend on it. The documentation of Azure Service Bus does not say what the real service sets, so check it once against the real service before production use.
- Tests of the listener host. The listener host and the development endpoint have no tests of their own. The demo runs a message through the emulator, but that was checked by hand.