Microservices for Business System Integration: A Bad Idea
- What are Microservices
- What is Business System Integration
- Microservices for Business System Integration
- An Example: Mulesoft
- A Saner Example
- Recommended Architecture
What are Microservices
Microservices are best described in opposition to monoliths. A monolith is one large, complex application. Microservices are a number of small, simple applications that can be combined to perform complex tasks.
- Scalability: It is easier to scale and manage small, simple services.
- Resiliency: If one of the microservices crashes it does not impact the others.
- Choice of technologies: Each microservice can choose the programming language and tech stack that are best for it instead of having to use the technology that was chosen for the monolith.
- Security and compliance: Each microservice can have granular control over permissions.
- Using services is complex: Instead of calling a function you have to use credentials and API calls.
- Testing is difficult: It is easy to test one application in isolation. With automated testing you can ensure that updates do not break any of the existing code. With microservices, though, a change could break another application and this will not be surfaced in automated testing.
- Infrastructure and Talent: If you want to use the optimal technology for a microservice you need to set up and maintain infrastructure for those technologies and also hire programmers that know how to work with all of the different languages you use.
- Debugging is difficult: With microservices it is hard to find the source of a problem. Instead of being able to start and step through debug one application, which is easy to do, you have to track the process through multiple applications, which requires a lot more time and is prone to manual error.
- Coordinating Deployments: Instead of being able to make updates and deploy them all at once with continuous integration (CI), deployments of multiple services have to be coordinated, so the rollout happens in the correct order and does not break anything.
- Speed of development: This is sometimes listed as a benefit of microservices, but in my experience setting up an application takes a lot of work that can be avoided with a monolith.
What is Business System Integration
Quickly, in case the name is not clear enough, business system integration coordinates and facilitates processes across multiple business systems, like Salesforce, Shopify, or Airtable, by passing data between them.
Microservices for Business System Integration
Business System Integration is already, in many ways dealing with a microservices architecture. All of the different business systems it is integrating are the microservices. All of these different systems have been purchased to best serve different business needs and teams. The job of business system integration is to coordinate all of these business “microservices” (although many of these business systems are more like monoliths on their own).
From this perspective the idea of using microservices for integration as well seems a bit redundant and overly complex. The benefits of microservices are minimized if not nullified altogether and the drawbacks are compounded.
Microservices Benefits applied to Business System Integration
- Scalability: This can still be a benefit, although integrations are not usually that resource intensive.
- Reliability: Microservices are still more reliable, although you can use a monolithic repository to initialize multiple services and gain reliability without the drawbacks of microservices.
- Choice of technology: Since integrations are performing the same functions there is little benefit here.
- Security and compliance: This seems true in theory, but once data is passed to another microservice control of it is lost and it is very hard to track due to the difficulty of debugging microservices. Since the business systems are generally set up for security and compliance this is usually redundant work.
Microservices Drawbacks applied to Business Systems Integration
- Using services is complex: Integrating business systems already requires understanding the business systems. With the addition of multiple microservices this introduces more complexity and required knowledge.
- Testing is difficult: Testing is already difficult in business systems integration due to it coordinating multiple separate applications. Building your integrations with a microservices architecture compounds this.
- Debugging is difficult: Again, introducing more services to a process increases the difficulty of debugging.
- Coordinating Deployments: The more separate components required by a deployment, the more complex it is.
- Speed of development: Standing up lots of independent services requires a lot of work, especially for small business systems teams that usually do not have many developer resources.
An Example: Mulesoft
Mulesoft is a SaaS integration tool that uses a microservices architecture. Their recommended way of developing an integration is to have an Experience API, Process API and System APIs . This means that if you want to build an integration you should create at a minimum of three applications. That is a lot of work and means you are dealing with so many applications the drawbacks of microservices are really felt. Further a lot of the work is redundant.
A system API is an API that coordinates with a single system. For example you create a System API for Shopify. In this application what you do is reimplement all of Shopfy’s REST APIs with Mulesoft APIs this requires you to create and maintain redundant API specifications, documentation and code.
Mulesoft is also not the best technology for creating complex business logic, so in the spirit of micorservices architecture their recommendation is to create a microservice using a technology that is good at creating business logic that you call from Mulesoft. Now your integration requires a minimum of 4 applicaitons, excluding the business systems you are actually trying to integrate.
A Saner Example
If we step away from the over engineered Mulesoft and think about how else a microservices based integration architecture could look we might end up with one application per integration. This way we get the benefits of reliability, but are not creating an insane number of applications.
We still end up with a lot of drawbacks, however:
- Reusability: many integrations will be integrating the same systems, but the code will be isolated, so it either has to be packaged and shared, or rewritten, both of which waste a lot of developer time.
- Speed of development: setting up applications takes a lot more time than adding code to an existing application.
- Lots of credentials: Credentials for business systems can frequently be quite expensive. This model requires each integration having its own set of credentials.
The best architecture I have used for integrating business systems is a monolithic repository. It enables easy reuse of code and credentials, easy debugging, easy deployment and quick development. It is also possible to avoid reliability issues with a monolith because monoliths can be used to start multiple separate applications that share the same codebase.
Business system integration is complex, requiring developers to work with many different business systems, teams and processes. Adding complexity to the method of integration through a microservices architecture compounds this complexity with little benefit.