Microservices architecture: What the gurus say about it

About a year ago I was very interested in learning as much as possible about the subject and gathered as much information as I could about it. I watched several conference talks and I read several articles from very knowledgeable and experienced people, like Martin Fowler, Fred George, Adrian Cockcroft, or Chris Richardson, in order to learn as much as possible about microservices, and this post is the result of that.

This post talks about:

SOA vs Microservices
When should we use Microservices?
Prerequisites
Characteristics
Frontend / Backend
Dangers
How to decompose a monolith
Conclusion

A micro-service architecture is an architectural concept that opposes a monolithic approach on an application architecture.

This means that instead of having an application with all its bounded contexts running in one server using in-process communication, we will have several smaller applications, corresponding to each of the application bounded contexts, running in different servers and communicating through the network, probably using HTTP.

In other words, the micro-service approach encapsulates the bounded contexts of the application, one bounded context in each micro-service, each micro service in its own server.

SOA vs Microservices

According to Martin Fowler, the term SOA has been widely used, in a very abusive way, and currently means many things, it is a very broad term. In Martin Fowler view, Microservices is a subsection of SOA.

When should we use Microservices?

As an academic, and pragmatic architect wannabe, I think this is a decision too important to make based on myths, or on what we want to try next, or on our desire to be in the cutting edge of technology. We need to base such an important decision in pragmatic trade-offs, in line with what Rachel Myers defends.

Rachel Myers, notes that an architecture should:

Make our product more resilient and fault tolerant
Encourage understanding, debugging and changing code
Help teams work together

I agree with Rachel, however I also feel that those characteristics are achievable in both monolith and micro-services architectural styles.

Martin Fowler sees several advantages to both monoliths and micro-services, which can help us decide when we should use microservices:

Advantages
Monolith		Microservice
Simplicity A monolith approach is much simpler to build, to orchestrate and deploy. A microservice approach will need heavy orchestration because it deploys to several servers, and uses other microservices APIs.		Partial deployment Using a microservice approach, we can update only a piece of the application. However, in a monolith architecture, we always need to redeploy the whole application, which involves more risks.
Consistency In a monolith application, its easier to maintain consistency of coding, error handling, etc., while in a microservices approach each microservice is fully managed by different teams, which might decide on different standards.		Availability In a microservice architecture, availability is higher, because even if one microservice fails, that means that one of the pieces of the application fails, not the whole application.
Inter-module refactoring A monolith approach makes it easier to refactor situations where several modules need to communicate or when we want to move classes from one module to another. When moving into a microservices approach, we need to have the modules boundaries very well defined!		Preserve modularity Keeping modularity and encapsulation can be quite difficult, despite SOLID rules. However, using a microservices architecture, we can further guarantee that there will be no shared state between modules.
		Multiple platforms / Heterogeneous

Microservices give you the option and flexibility to use different technologies, different programming languages, according to the business needs.

Personally, I like the pragmatic view of Eric Evans and I think of real pragmatic hard and soft pros of using micro-services, where the hard pros are the things that are not possible to do with a monolith, and the soft pros, the things that are possible in a monolith but the micro-services make it easier:

Micro-services
Hard Pros		Soft Pros
Independent Scalability Each module, being in different server nodes, can scale horizontally in a rate independent of other modules.		Preserve modularity Keeping modularity and encapsulation IS possible in both monoliths and micro-services. Nevertheless, it can be quite difficult and we have been trying to do it for decades, quite unsuccessfully, despite SOLID rules. However, using a microservices architecture, we can further guarantee that the logical partition of the application (modules) is enforced by its explicit physical partition in separate servers/nodes. This physical isolation makes it unnatural to cross the bounded contexts boundaries.
Independent tech stack Because each module is in different server nodes and use an agnostic communication language, they can use completely different programming languages, communication tools, monitoring tools, or data persistence tools. This is great for choosing the best tool for the job at hand and innovate by being early adopters of new technologies.
		Independent evolution of sub-systems A microservice can evolve and break backwards compatibility without it getting clutterred with code to support older versions, because we can leave older versions of the microservice running while needed.

I feel the hard pros are the real pragmatic reasons to use a micro-services architecture. Those are the only things we can not do with a monolith, so if we need them, we must move to a micro-services architecture, otherwise the complexity demanded by a micro-services architecture is hardly worth it.

I also think the partial deployment and partial availability provided my micro-services, although impossible to do with monoliths, they are not really key advantages (although they are advantages nevertheless).

Independently of what is our preference or desire, we should NOT start a new project with a microservice approach. In the beginning we must focus our attention on understanding the domain, and for that we can not be distracted by the huge complexity of creating a microservices ecosystem (Rebecca Parsons, Simon Brown).

Prerequisites

Continuous delivery

Ability and mindset to go faster, faster, faster

One of the reasons we want to use microservices is that we want to be able to change rapidly, so we can respond to business requirements changes faster than the competition, or in Eric Evans words, we need to acknowledge the rough and tumble of enterprises:

The reality of software development is that the start point is never one where we fully understand the domain. We deepen our understanding of the domain as we go, and we have a constant need of refactoring. So, refactoring is a need, but also a danger because code can become confusing, messy, specially if bounded contexts are not respected. Microservices reinforce the bounded contexts borders and so, they help maintain the sanity, clarity, isolation and encapsulation of the code, in decoupled and cohesive modules. If a module/microservice gets messy, the mess is contained within it, it does not spread easily beyond its borders.

We need to go faster, in all stages of the development cycle! Although this is true both in a monolith and a microservices approach, the microservices approach is said to better comply to that need.

Martin Fowler tells us we need to be able to have:

Rapid provisioning: We need to be able to deploy new machines rapidly, for development, testing, acceptance and production.
Rapid application deployment: We need to be able to automatically and rapidly deploy our services.

Fred George tells us the same thing: there is a huge drive/need to go faster and get ahead of the competition! He does a retrospective of the hardware lead times and he notes that the evolution goes from taking 6 months to get a server, back in 1990s, to 30min in 2010 using cloud services, and currently (2015) we aim to need less than a minute to have a new server up and running, using Docker.

Adrian Cockcroft, one of the keys in netflix cloud and microservices early adoption, also tells us how important it is to have early adoption of new technologies and to be very fast in provisioning new machines and deploying new versions of our application. He is a big fan of Docker, because it allows provisioning and deployment of development, testing, and production boxes, in a matter of seconds.

Sophisticated monitoring

Monitoring is critical (Rebecca Parsons), we need to know when a server is down, when something becomes unresponsive, when calls are failing, and we need the tools to quickly debug it (Martin Fowler), and we need to know it per microservice (Fred George).

Strong devops culture

We need to have devops exclusively to handle monitoring and orchestration, and we need a proximity relation with good communication between developers and devops (Martin Fowler).

In a microservices approach we have more things to deploy, monitoring needs to be more sophisticated, the amount of things that can fail is much higher. As such, having a strong devops culture is a strong requirement (Rebecca Parsons).

Characteristics

Martin Fowler and James Lewis, tell us what they found to be the set of characteristics that define a micro service, in their well known article and some of their talks (Martin Fowler, James Lewis).

What is a microservice?

My personal view on what a microservice aligns completely with the definition given by Adrian Cockcroft:

Loosely coupled service oriented architecture with bounded contexts

A bounded context is an explicit conceptual boundary around a business context. For example, in a eComm platform, we can think of “themes”, “payment providers”, “orders”, “shipments” or “app store” as bounded contexts and therefore candidates to microservices.

Some other good background information related to microservices in general is the book “Building Microservices”, authored by Sam Newman, for which we can access a free preview here.

James Lewis, also tells us that microservices should:

Be cheap to replace
Be quick to scale
Withstand failure
Allow us to go as fast as possible

How big is a microservice?

James Lewis, co-author of the known microservices article, together with Martin Fowler, states that a service should be just big enough to fit his hand, meaning that one person should be able to completely understand that microservice.

In the community, the microservices size varies a lot. Martin Fowler found cases of companies with ratios that range from 60 people per 20 services down to ratios of 4 people to 200 services. The Amazon approach, he mentions, is the two pizzas team: a microservice team should never go beyond the amount of people that we can feed with two pizzas.

Fred George says that a microservice should be “very, very small”, needing only a team size of one developer, to develop and maintain, which is in sync with James Lewis.

Personally, I agree with James Lewis, Fred George and Adrian Cockcroft. I feel a microservice should correspond to a bounded context that one person can fully understand.

This means that, the more functionality your application has, the more microservices it will have. For example Netflix has around 800 microservices!! (Fred George)

Nevertheless, either in the very beginning of the microservice life cycle or later on, a bounded context / microservice, might be too big to be understood by only one person, in which case we need to identify bounded contexts within it, and further segregate it into different services. This is in sync with both the evolutionary architecture concept and DDD, where both of them defend that an application architecture is in continuous change/refactor, as we learn more about the domain and/or the business requirements change. As Rebecca Parsons says, “the granularity question is crucial”: the most difficult thing to do when developing microservices is defining the boundaries, and we will surely be merging and segregating services as we further understand the domain and/or the business requirements change.

Componentization via services

A component is part of a system of components, where each component is independently replaceable, independently upgradeable (Martin Fowler) and independently scalable (Rebecca Parsons).
In software development we can see two types of components:
1. Libraries: which are pieces of code that we use in our application, and which we can be upgraded or replace by another library, hopefully without affecting the rest of the application. Communication is done through the language constructs. However, if the library we want to use in our application is built in another language, we can not use that component;
2. Services: part of the application which is in fact an application itself, running in its own process. Communication is done through interprocess communication, web-service calls, queue messaging, etc.. If we want to use a service component in our application, who is written in a different language, we can do it because it will be running in its own process (approach preferred by Chad Fowler);
Independent scalability, where each service can be scaled independently of the rest of the application, so if one of the application microservices has 100 times more load than the others, we can easily scale only that microservice;

Heterogeneous

Being heterogeneous, means the system is able to be built using different programming languages. This has several advantages (Martin Fowler), and Chad Fowler believes a system should be heterogeneous by default, meaning developers should not only be allowed, but be encouraged to try new technologies.

An heterogeneous system has a few benefits, namely:

Helps prevent tight coupling, because there are different programming languages;
Developers can experiment with new/different technologies, which helps in creating value and maintaining developers in the company because they don’t need to leave in order to try new things;
- RULE:The code unit we use to try a new technology should be a small code unit, a small module/microservice, so that the risk is small;
- RULE: The code unit we use to try a new technology, must be disposable.

Human resources organized around business capabilities

In the old days (and in many cases still today) software development teams would organize themselves into technology based sub teams. This means that in a project we have (ie) a DBA team, a Server side team and a UI team, who work independently of each other. This ends up bringing quality problems because both the domain knowledge and the development efforts get dispersed;

In a micro-services approach, we will have teams organized into business capabilities, for example Orders, Shipping or Catalog teams. Each team would then have members specialized in all the necessary technologies (ie. UI, Server side, DBA, QA), all the way through the end users. This allows the team to gather high domain knowledge and to focus all development efforts on a particular section of the application, their micro-service. (Martin Fowler, Eric Evans)

This approach goes together with Conway’s law, which tells us that if we want to have highly cohesive and decoupled microservices, we need to design our organization structure as a reflection of the components structure we want to have.

organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations

Melvin Conway, 1967

Products not projects

The old approach was to have a team build a set of functionality which, when finished, was given to another team to be maintained.

In a micro-services approach, a team should own a product over its full lifetime, all the way from development, into maintenance, until the product is taken off-line. This approach creates the “product mindset”, which means a strong link between the technical product and its business capabilities. It creates an on-going relationship where the question is how can the software assist its users to enhance the business capability.

Smart endpoints and dumb pipes

Again, in the old days, companies would use a Enterprise Service Bus architecture, which would both have the communication channel between endpoints and the business logic. This approach would then develop into a “spaghetti box”;

The micro-services architecture moves the business logic into the endpoints and uses a simple communication mean, like http.

Decentralized governance

Key decisions about a micro-service should be made by the people actually developing the micro-service. Key decisions are decisions, ie, about the programming language, the persistence tool, the deployment methodology, the public interface contracts, the UI, etc.

Decentralized data management

In a traditional approach, an application will have only one database, and many of the different business logic components of the application will “communicate” through the database, meaning that components read data belonging to other components directly from the database. It also means that all the components use the same data persistence, even if it is not the most appropriate to some components (Martin Fowler);

In a micro-services approach, where each business component is a micro-service, each component will have its own database, inaccessible by other micro-services. Each component data is only accessible (both for reading and writing) through the related component interface. This also means that the data persistence type can also be different between business components (Martin Fowler, Chad Fowler).

For Fred George, this is the first challenge in moving into a microservice approach.

Infrastructure automation

Continuous delivery (Martin Fowler, Rebecca Parsons, Chad Fowler, Eric Evans):

Blue/green deployment, to deploy with zero downtime;
Automation, so we can deploy to several servers with the press of a button;
Phoenix servers, as to get boxes to go down and up rapidly;
Monitoring, so we can spot when things go wrong and have monitoring tools to debug it.

Design for failure

When we have an application distributed throughout several servers, they will eventually fail, specially if in different nodes. So, we need to design our software in a way that it is prepared for those failures (Martin Fowler).

Chaos monkey is a tool, built by NetFlix, which turns servers down, in order to test the resilience of a system to this type of failures (Martin Fowler).

This is also something Rebecca Parsons finds crucial, after all we are not even using in-process communication between our services any more, we are using http communication which is not even remotely as reliable, so services communication will fail and our system must be ready for it.

Evolutionary design

The design of the overall application must not be static, it must be possible to simple to evolve the architecture according to the business needs. For example:

We can evolve (refactor) a monolith application into a microservice application by isolating and detaching a sets of business logic (bounded contexts) into a standalone microservice;
We can merge existing microservices, for example when different microservices frequently need to change at the same time;
We can segregate existing microservices, when we need and can evolve them separately, or when we realize there is significant business logic differences;
We can add a temporary feature to an application by creating a microservice who will only live for a predetermined time frame;

Frontend / Backend

There are two ways of thinking about how to structure the frontend and backend in a microservices architecture:

Break up each piece of the UI per microservice, and keep each piece together with the corresponding microservice. This approach has the benefit of having the frontend communicating in-process with the backend. However, the complexity of maintaining UI consistency across microservices is very high, if not impossible, and will lead to the need of updating several microservices simultaneously when there are cross boundaries changes to the UI, coupling microservices and eliminating the isolation and independence of the microservices provided by this architecture style. This actually makes this option an Anti-Pattern!
Break up the frontend and backend code bases, keeping the UI of an application together, which then communicates with the microservices through HTTP. This keeps the microservices decoupled from each other and provides extra decoupling between backend and frontend, while maintaining each application UI together, making it trivial to keep the UI consistent. This is the structure advised by Rachel Myers, and as far as I’m concerned its the only way of doing it.
In this case, we have two options for the communication approach between frontend and backend:
1. Many tiny HTTP async requests instead of one big request, which has the advantage of being non-blocking (approach preferred by Chad Fowler).
2. One big request to a specialized service (gateway / aggregator / cache) who then gathers all data from the microservices ecosystem, which reduces UI complexity.

Dangers

Technology flexibility must be managed

One of the advantages of the microservices is that we can use different technologies for the same purpose. We can use a different XML parser library or a different data persistence tool in every one of our microservices. But the fact that we can do it, doesn’t mean that we should. Using a multitude of technologies and libraries it can get out of hand, so we need to select a fundamental set of tools and only go outside those tools when we actually need to (Rebecca Parsons).

Interface instability must be managed

Particularly when in the beginning of the development of a microservice, its API is quite unstable, but even at later stages, when the microservice is already mature, we will need to change the API, both its input and output. When making those API changes, the process must be managed carefully because there will be software relying on that interface stability (Rebecca Parsons).

Data consistency must be assured

As, when using microservices, we will have each microservice with its own data storage and in many cases data belonging to one microservice will be partially or totally duplicated in another client microservice. That duplicated data will be updated when the client microservice receives an event triggered by the supplier when its data is changed.

The event that is triggered will go into a message queue, where it will wait until it gets picked up by the client microservice.

This means that the client microservice will be returning outdated data while it doesn’t pick up the triggered event. The data is inconsistent.

Nevertheless, eventually the data changes will be propagated to all the copies, data will be ~~great~~ consistent again, and hence the term “eventual consistency”. This term is simply an acknowledgement that data might be inconsistent for some (short) period of time.

The usage of eventual consistency has significant implications that must be addressed while developing the application, all the way from the back-end to the UX layers. (Rebecca Parsons)

How to decompose a monolith

When starting to build an application, we should still start by building a monolith, for its simplicity. Yet, we must try to build it in such a way that it is very modular, so that those components will be easy to detach into a standalone microservice (Rebecca Parsons). This, is in fact, in sync with Simon Brown’s idea of designing an application as a set of decoupled components in a single deployable unit.

When decomposing a monolith into a microservice architecture, or a set of decoupled components, we need to think of several dimensions that can support our decision:

Think about bounded contexts as defined in DDD (Rebecca Parsons, Rachel Myers)
- Each microservice should be a bounded context, code that belongs together in a conceptual business and technical view. It should typically have data and/or business logic connections between the code units in it, but few connections with external code units;
Think about business capabilities (Rebecca Parsons)
- What are the value streams that exist in the organization? the business products? the business services being delivered?
Think about what consumers need (Rebecca Parsons)
- Rather than a producer view, we can also look at a consumer view: what will the consumer want from this service? How will he use it? What will he expect?
Think about communication patterns
- What parts of the system might be using the same data? What business logic will communicate more intensely? (Reb ecca Parsons)
- Does the architecture have single points of failure, because a micro-service is a hard dependency of many other micro-services? (Rachel Myers)
Think about data architecture (Rebecca Parsons, Rachel Myers)
- Services own their data, they have their own database, and we need to think of things like “eventual consistency”. If two data structures are very dependent of each other it might be a good idea to keep it in the same microservice so we don’t have to create mechanisms to deal with “eventual consistency”;
Think about correlated change patterns (Rebecca Parsons, Rachel Myers)
- If two code units are foreseeable to be changing together, then we should keep them in the same microservice to reduce the overhead of dealing with changing APIs;
Be prepared to merge and segregate services (Rebecca Parsons)
- We will probably not get it right every time, and as we gain knowledge of the domain, we will have a better understanding of where the bounded contexts are. At the same time, business will change and we will need to adapt to that change, rapidly! So we must deign our system in such a way that we can quickly split and merge microservices;
Tolerant reader (Rebecca Parsons)
- We will always, at some point, need to make a backwards compatibility breaking change. However, we can do our best to only do it as a last resort. One way of not needing to change our service whenever another service changes is to make it so that we only need to change when they change the data we actually need;
Stateless and phoenix nodes (Rachel Myers, Chad Fowler, Eric Evans)
- Do not repeat static files (HTML, CSS, JS, Img) across apps and services;
- The UI applications should be completely separate from the micro-services ecosystem;
- Use disposable nodes;
- Use immutable deployments: never upgrade software on an existing node;
Convention over configuration (Chad Fowler)
- The architecture structure and naming should be the same throughout the microservices ecosystem;
- Create a microservice sandbox generator, so we have a starting point with the usual structure already defined;
Optimized communication between Microservices (Chad Fowler)
- Create a base HTTP REST client library, optimized for REST calls, from where we can build specific microservice clients to be used by the other microservices. This optimized client must be ported to all languages used in the microservices ecosystem;
Service discovery (Chad Fowler)
- Every microservice must know how to contact the other microservices. We can use a localized (per service) config that can be updated in all microservices at once, when there is a change in a microservice location;
Monitoring (Chad Fowler)
- Measure everything: Network, Machine, Application;
- When creating a new microservice, it must be mandatory to add all monitoring functionality to it;
Migrating from Monolith to Microservices (Chad Fowler)
- Favour many small queries over a few big queries (remove joins);
- Segregate DBs;
- Build new features as microservices, as prototypes;
- Replace code in the old system by API calls to the new microservices;
- Test under crazy load/behaviour;
Long term objectives (Chad Fowler)
- make it work;
- make it fast;
- make it cheap => AWS spot instances provides 85% to 95% savings in server nodes;
Productivity (Chad Fowler)
- Use Docker to increase development and deployment speed

Conclusion

Most of the projects don’t need a Microservices Architecture, what they need is a good architecture.

By good Architecture I mean a good structure, but also (maybe even more important) a clear definition of the structure, a clear and accurate reflection of that structure in the code, so that it is implicitly communicated to the developers, so they can see the bounded contexts borders and understand when they should cross those borders or not.

Then it is up to the developers to maintain and evolve the architectural structure. This involves strictness to stick to the plan, to the structure, to the architecture, which is not always easy, given that we are only human.