- General Description
- Changes to process-oriented architecture (part 01)
- Applied concepts
- Tutorial to run the application CiRa
- Architectural Decision Records
- Assignments
- Microservices
The project encapsulates the supply chain of the company CiRa. Built with Spring Boot, Kafka, and Camunda, our system comprises four key parts: Machine, Factory, Warehouse, and Logistics microservices, each playing a vital role in making manufacturing smarter and more efficient.
- Added description to README files
- Improved Camunda 8 BPMN processes (labels)
- We can create new machines in the docker compose file of the same microservice.
- Improved sequence diagrams
- More detailed description of the trade offs between Camunda 7 vs Camunda 8
- More detailed reflection about our challenges, and implementing experience
Event Notification is a critical feature within the system architecture, facilitating seamless communication and coordination between components. In this scenario, when an employee initiates a machine by issuing a POST request, the machine service promptly emits an event signalling its availability. This event is broadcasted to the designated 'machine-status' topic. This event-driven approach enhances system responsiveness and agility, enabling real-time updates without necessitating continuous polling or manual intervention.
Event-carried state transfer is a paradigm in distributed systems architecture where changes in state are communicated through events rather than direct data transfers. In this context, the Factory microservice is responsible for maintaining the real-time inventory levels of produced goods. Once a predefined inventory threshold is met, the Factory emits an event to the 'stock-update' topic. The Warehouse microservice, subscribed to this topic, captures and retains a synchronized copy of the inventory data. This approach offers significant advantages, including enhanced decoupling between components and reduced computational load on the Factory microservice. However, it necessitates the management of replicated data and introduces eventual consistency considerations, ensuring that all replicas converge to the same state over time. Thus, while promoting scalability and resilience, event-carried state transfer requires careful consideration of trade-offs to ensure system reliability and performance.
Trade-offs:
- Decoupling and reduced load on Factory
- Replicated data and eventual consistency
The utilization of a parallel saga pattern within our event-driven and process-oriented architecture is instrumental in ensuring robustness, scalability, and fault tolerance. By employing parallel sagas, we can concurrently execute multiple interdependent calls to our machines. This pattern facilitates asynchronous communication between services, enabling them to operate independently while maintaining consistency across the system.
The integration of the Anthology Saga pattern brings substantial advantages in managing complex workflows with multiple related processes. By adopting the Anthology Saga, we can orchestrate cohesive sequences of events across various services, ensuring consistency and reliability throughout. This pattern allows us to group related sagas under a common overarching saga, facilitating better organization and coordination of business processes. With the Anthology Saga, we can handle intricate scenarios with ease, as it provides a structured approach to managing dependencies and interactions between different saga instances. We use this pattern for our “Supply Chain” process.
In the Human Intervention pattern the system is designed to gracefully handle exceptional situations by involving human intervention when necessary. When a critical error occurs that cannot be resolved automatically or through traditional retry mechanisms, the system leads to human operators for manual resolution. This pattern ensures that complex or ambiguous errors can be addressed by human expertise, maintaining the integrity and reliability of the system. In our system this pattern is used to make a decision about a further transfer in case of a transfer truck accident.
In the Stateful Retry pattern within event-driven architecture, resilience is achieved by implementing a stateful mechanism for retrying failed service tasks. Specifically, each service task is retried up to three times upon encountering a failure. This approach aims to improve the robustness and fault tolerance of the system by allowing failed tasks to be automatically retried.
We implemented the Outbox pattern. Here, the Factory microservice listens for new machine fill levels. In the same transaction, the microservice saves the machine fill level in the database and adds a new entry to the end of the Outbox. Finally, a message relay reads the entry and sends the information further.
The Machine microservice streams three different variables, each with a generated timestamp. The microservice event processor listens to the KStream "stream-machine-fill-level," "stream-machine-production," and "stream-machine-temperature". Finally, the event processor streams a joinedStream to the topic "machine-stream". The microservice Factory is listen to the kstream.
We implemented a Json Serializer, Deserializer, and a Serdes. Therefore, we can consume specific events.
In the KStream MachineFillLevel, we implemented a router (map) to split events into separate branches. The reason for this is that we can respond differently to various values. A low machine fill level does not have high priority as a high machine fill level. We split the stream into three branches: "low-machine-fill-level", "medium-machine-fill-level", "high-machine-fill-level".
For the KStream MachineTemperature, we implemented a simple filter to ignore values below a certain threshold. Temperature values are only important in the upper range.
For the KStream "stream-machine-production," we implemented a tumbling window of 60 seconds with a grace period of 5 seconds.
Next, we added a KTable to our topology. In the stream "stream-machine-production," we want to count the number of received production events. First, we group the stream by key. The reason for this is that multiple machines can send their production data. Then, we use the previously mentioned tumbling window. We count and store the KeyValue pair under the key "machine-production-counts." Finally, we configure a suppression to emit only the "final results" from the window.
After successfully creating the KTable, we map new KeyValue pairs to a stream, using the time window as keys and the data from the KTable as values.
We decided to create the timestamp in the Machine microservice. This is intended to provide higher accuracy and directly mark the data when reading it out. To achieve this, we had to program a time extractor in the microservice event processor.
For debugging purposes, we implemented several for-each loops to print the current state of the stream.
In the end, we join the KStream MachineTemperature and MachineProduction. Therefore, we create a join window with a time difference of 60 seconds and a grace period of 10 seconds.
-
Go to the path /project/.
-
Execute the command for process-oriented architecture (a)
docker-compose up --build
- Execute the command for event-driven architecture (b)
docker-compose -f docker-compose-kstreams.yml up --build
3a. Run the StartProductionLineProcess on Camunda 8
OR
3b. Start Kafka Streams
curl --location 'localhost:4001/machine/status/toggle' \
--header 'Content-Type: application/machine+json' \
--data '{"machineProductionSpeed": 2}' && curl --location --request POST 'localhost:4001/machine/stream/production/toggle' \
--header 'Content-Type: application/machine+json'