Often when the state of a business entity changes, you want to publish an event stating that this has happened so that other microservices are in sync. To achieve this, you would do the following:
- Receive message
- Update business entity
- Publish 'EntityUpdated' event
However, if step (2) is not idempotent and only step (3) fails, we find ourselves in a scenario where the business entity has been updated an other microservices are not aware of this change.
The Outbox Pattern solves this problem by moving the event publisher into an outbox processor. The command service then updates the entity and adds a record to the outbox messages in the same transaction. The outbox processor will poll this outbox messages for any unprocessed messages, and publish them as they arrive. This is illustrated in the diagram below:
This project demo's the Outbox Pattern using NServiceBus (In Memory) and EntityFramework (SQLite). It contains 3 apps:
| App | Purpose |
|---|---|
| OutboxPatternDemo.MedicalRecords | ASP.NET Web API that allows you to update medical records, and uses the Outbox Pattern to guarantee that update events are published at least once |
| OutboxPatternDemo.Bookings | Console App that subscribes to and de-duplicates messages, and uses a saga to book follow up appointments |
| OutboxPatternDemo.Monitoring | Runs the Particular Service Platform, allowing you to monitor events within the system |
- Implemented using NSB's built-in outbox pattern feature,
- NSB configuration in Program.cs,
- Drawback: requires NSB license.
CustomOutboxContext- the DB context for messages to be published,CustomOutboxMessageBus- the message bus writing toCustomOutboxContext,CustomOutboxProcessor- the background process reading fromCustomOutboxContextand publishing messages,- Drawback: this is intended for educational purposes only, I'd recommend using commercial software (e.g. NSB, Mass Transit) for anything other than a hobby project.
- Adds duplicates to a cirular buffer (effectively caching the last X messages which it checks against),
- Pros: efficient as in-memory,
- Cons: ineffective in high throughput / 'spikey' systems, does not scale horizontally.
- Stores duplicate keys in a distributed cache (e.g. Redis)
- Pros: scales horizontally, configurable sliding expiration,
- Cons: requires additional infrastructure, does not scale well for high throughput / large sliding expiration scenarios.
- Records message ID's in a table which it checks against,
- Pros: scales horizontally, scales well with high throughput / large sliding window scenarios,
- Cons: requires additional infrastructure, SQL DB requires management.
Please read the docs here to set up this project.