Microservices : Distributed Transaction Management — API Composition (Part 2)
In part 1 of this article you saw what is SRP with respect to Microservices, and how distributed transaction management along with SRP make things complex. We also saw few patterns to deal with these complexities. And in continuation to that, let’s explore next pattern “API Composition”.
In Microservices architecture, where each service has its own DB, it helps in loose coupling and inter-operability but also brings challenge of consuming data from multiple external source systems.
Context
Consumer has many APIs tens or even hundreds to consume. Every API has many of its own characteristics like — authentication, throttling, error handling, pagination etc. which makes it hard for consumer to connect each of the API separately and manage their’s nuances.
Solution
“API Composition” design pattern offers a solution in this context and is one of the most popular patterns specially in joining data from multiple queries from separate DBs and creating a composite view.
There are multiple ways to implement this pattern –
Aggregation Composition –
In this pattern client communicate with the composite service layer and that layer may call multiple other services to respond client. Every individual service will have its own implementation and that not necessarily to be same.
For example, some of might always go to DB to fetch latest data, some may have local cache involved. They might have different ways to handle failure and different mechanism for retries. These all will be hidden from the consumer. Consumer will not bother to integrate each of them individually rather will be integrating with just one composite service which will provide all the data based on consumer’s requirements, keeping it isolated from underlying individual service’s integration requirements.
One of the real-world problems is to present a single UI which is brining data from multiple microservices and performs some aggregation or computation on the data received. Now think of this service design pattern.
Let’s say there is an application which provides the user product information, reviews and its availability but for availability it needs to go to a third-party service which provides data in real time. One of the possible implementations using this pattern would be something like this -
Branch Composition –
In this pattern client communicated the composition service layer and in turn composition service calls other services which in turn can implement other composition patterns like aggregation and/or chain. Also, a service may call multiple service in parallel.
An individual service can be called directly by the consumer as well without going through the composite service. This is a complex implementation of API composition pattern but used widely in appropriate business use cases.
Let’s take an example of an Order placement scenario. While placing the order, once user tries to place order, a public API for address verification is called. For a verified address, it calls a composite service to confirm the order and send notification to the user. Now in turn order creation may make use of many other services to place order, while it can send the notification to user for order processing, which doesn’t need to wait for a successful order placement. Also, notification service may be called separately as well once status of an order changes.
One of the possible implementations using this pattern would be something like this -
Chained Composition –
In this pattern consumer calls the composition service which in turn call other services in a series and request to next service is dependent on the response of previous service.
This pattern is not advisable until absolutely necessary since it’ll block the consumer until all the services in the flow completes their processing successfully. Also, if this needs to be implemented then there are few things to keep in mind, chain should be small and there has to be a fallback mechanism in place to handle scenarios where one or more services in the chain are not available.
API Composition Implementation in API Gateway
In context of microservices, using a management layer provides a centralize place for all cross cutting concerns like logging, monitoring, security, throttling, caching etc. which are common to all services. API gateway covers nonfunctional aspects of the microservices, while other patterns discussed above takes care of functional concerns.
One thing to keep in mind here that API gateway is additional work in terms of infrastructure, configuration, setup etc. Also, during the round trip this will be an additional hop which will add latency to the response time, which not be significant but something worth to mention. So, If you are having only few microservices which are not expected to grow over time, API gateway might not be suitable. Weigh out the benefits of it versus the complexity and efforts it brings in before you make a decision.
More on API gateway later in some other article.
In the next article, I’ll talk about “Two Phase Commit (2PC)” which is another pattern to manage distributed transactions in Microservices architecture.
Share your thoughts in the comments.