Microservices : Distributed Transaction Management — Saga Pattern(Part 4)

As the word itself means “Sequence of Events”, this pattern is a sequence of local transactions.

In microservice architecture it’s common where one microservice calls another microservice and since Microservice architecture follows SRP, they have different DBs thus a single transaction spans multiple DBs (aka multiple Microservices) so to maintain Data consistency is a challenge.

A transaction is a single unit of work. A transaction has an entity which changes its state during the transaction. This change of state is called an event. In this context a single transaction consists of many transactions.

We saw 2PC (Two Phase Commit) pattern in last article which solves this problem up to certain extent but not all. Saga patterns ensure data consistency in distributed transaction environment with the help of compensatory transactions. Saga pattern can be implemented in two ways –

1. Saga By Orchestration
2. Saga By Choreography

Let’s understand it step by step, how these implementation works –

Saga By Orchestration

In this implementation there is an orchestrator which tells all the participating services what to do and when.

In 2 PC commit we had synchronous transactions. Here we avoid having synchronous transactions and also each transaction a local transaction which is ACID compliant within itself. Orchestrator will orchestrate different transaction based on the messages received.

Let’s take the same example that we took for 2PC pattern and try to implement that using Saga orchestrator. Here is the step-by-step illustration of the same –

1. Order service receives the request and create saga orchestrator.

2. Orchestrator creates an order in pending state.

3. Orchestrator creates a message in wallet channel to reserve the credit.

4. Wallet service which subscribes to wallet channel, receives the message.

5. Wallet services attempts to reserves the credit.

6. Wallet service publishes credit reserved or “Credit insufficient” message in the orchestrator channel.

7. Orchestrator receives the message from wallet service.

8. Orchestrator either confirm the order or cancel the order based on the message received from Wallet service.

Pros –

1. It is good for complex orchestrations, especially where new services may get introduced in the future.

2. Separation of concerns over individual participants keeps the business logic simpler.

3. It doesn’t introduce cyclic dependencies (keep in mind that do not let the participants directly call the orchestrator.)

Cons –

1. Additional coordination logic make it very complex from implementation perspective.

2. Orchestrator as an additional component creates a single point of failure. Orchestrator has to be highly available.

Saga By Choreography

In Saga implementation by choreography, all the participant transactions communicate themselves with the help of messages without having a central orchestrator. In this pattern each local transaction publishes the messages which are received by the intended recipient and in turn trigger local transactions in other services.

Let’s take the same example as of above and understand step-by-step illustration of this approach –

1. Order service receives the request and create creates an order in pending state.

2. Create order transaction publishes a message in Order channel to reserve the credit.

3. Wallet service which subscribes to Order channel, receives the message.

4. Wallet services attempts to reserves the credit.

5. Wallet service publishes credit reserved or “Credit insufficient” message in the Wallet channel.

6. Order Service receives the message from wallet service.

7. Order Service either confirm the order or cancel the order based on the message received from Wallet service.

Now when we look at both the approaches “By Orchestration” and “By Choreography”, they look very similar, however there are some key factors which helps in making a decision which one may fit a particular situation based on the pros and cons of this approach –

Pros –

1. Better fit to use cases involving fewer participants with simple flow

2. No additional service and logic as in “By Orchestration”

3. No single point of failure as no central orchestrator

Cons –

1. Cross cutting concerns among participants makes it harder to understand which participant will react to which message

2. High risk of cyclic dependency since services react to the messages produced by each other

Considerations

However, there are few things to keep in mind while using this pattern –

· We have already seen how easily this pattern can become complex for implementation and management. Recommend to always have a unique transaction id maintained throughout the flow for debug and traceability.

· Data roll back is not easy as individual services writes to their local databases. Compensating transactions should be put in place to maintain data consistency.

· Participating transactions should be idempotent to ensure data consistency. Also request payloads should be captured for resubmitting.

· Synchronous transactions should be avoided.

· Saga pattern implementation is highly dependent on the way it’s designed to act. Architects need to ensure that in case of “By Choreography” participating transactions have their context, boundary and domain model defined and in case of “By Orchestration” approach, all the data is available before the handling over the control to orchestrator.

· It imposes data durability challenges due to lack of data isolation by participants which can result in anomalies like dirty reads. Measures need to be put in place to deal with such situations

In the next article we’ll explore CQRS pattern, how to use, when to use and its trade-offs.

Till then happy reading and share your thoughts on Saga pattern in comments.