DDD.14 – Maintaining model integrity

When working in a large system, with a large model where we have several teams working on it, a section of the model will likely end up being interpreted, and adjusted in code, in different ways by the different teams.

These different interpretations and code adjustments will end up corrupting the model, leading to incoherent code and eventually bugs.

The best way to deal with this issue is to break the large model into smaller independent models. Breaking down a model is done by identifying bounded contexts and using context maps to identify their borders and relationships between them.

Bounded context

It’s common, and natural, for projects to have several different models within them.

However, the boundaries in which the different models connect are fragile and prone to lead to mistakes, unreliability and confusion. To prevent these issues, we should:

Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas. Keep the model strictly consistent within these bounds, but don’t be distracted or confused by issues outside.

Pg. 336

In different contexts, different models apply, with differences in terminology, concepts and rules. The different models, used in different contexts can and will overlap. They might relate to the same data, but represent it in a different way. I am thinking for example of having a User entity in a UserManagement context contain the whole User data, but in an Account context the User entity might be just a fraction of the whole User data.

This is specially relevant when we think of microservices, where the communication between the User microservice and the Account microservice has high cost (HTTP communication), and therefore we might need to cache/duplicate the relevant User data in the Account microservice, and rely on eventual consistency mechanisms to keep the data in sync across different microservices.

Recognizing splinters within a bounded context

Two types of problems arise when a bounded context starts to splinter:

• Duplicate concepts: The same concept is represented and implemented in more than one way. This means that when a concept changes, it has to be updated in several places, and the developers need to be aware of the several different ways of doing one same thing, as well as the subtle difference within them.
• False cognates: When two people are using the same term and they think they are talking about the same thing, but they are actually thinking of different things.

Context map

When the bounded context boundaries are not explicitly clear, those boundaries tend to get blurred by promiscuously mixing code that exists in different bounded contexts.  This leads to code that is tightly coupled and difficult to change in isolation.

Therefore, we should not directly use functionality and data structures through different bounded contexts. The bounded contexts must be encapsulated, as independent as possible. To reach this goal, bounded contexts must communicate through abstractions (interfaces) and, if necessary, translation layers or even anti-corruption layers.

To make the bounded contexts ecosystems explicitly clear, encapsulated, loosely coupled and high cohesive, we should:

1. Create a global view of all bounded contexts and their relations, using context maps, naming them and adding them to the ubiquitous language. Identify the points of contact between bounded contexts, together with the used translations and abstractions. All developers must know the boundaries and to what context any given code unit belongs to.
   Personally, I find that having diagrams of the context maps hanging in the walls of the development offices is a great way of communicating the boundaries to the team, and keeping them in mind at all times. They should include their components, contact points, translator layers, and anti-corruption layers.
2. Define teams organization to match the technical and conceptual bounded contexts we (want to) have in the project (Conway’s law).
3. Personally I also find it natural and logic to explicitly define the bounded contexts as modules and sub-modules, in the project structure.

Continuous Integration

The bigger the bounded context, the bigger the team working on it, and the bigger the danger of model fragmentation and incoherency.

In the other hand, breaking the system in many small bounded contexts can also make it lose integration and, again, coherency.

This means that maintaining a system model unified and coherent is very hard. To solve this, we need to:

• Increase communication
• reduce complexity
• have procedures in place, so we can safely modify existing code

This means concretely:

• merge code as frequently as possible
• have a good suite of automated tests
• continuously discuss, update and extend the ubiquitous language

Patterns

Shared kernel

At some point, for some reason, we might decide that different contexts should share a subset of the model implementation. This means they might share code, data representation (entities) and data storage.

This gives us the benefit of allowing us to not need any translation layers or anti-corruption layers. In the other hand, the shared subset of the model can only be changed in a very careful and coordinated action between the teams owning the different contexts.

I can imagine this as a very specific library or even component, that is used by several modules.

Customer/Supplier development teams

Also known as server/client or upstream/downstream, this refers to the situation where one module (the supplier)  just feeds data into another module (the customer).

These two modules should refer to two different bounded contexts. In fact, just because they are two distinct modules is already a revelation that they are distinct bounded contexts.

However, there is a tight relation between these two modules. The customer module depends on the supplier module. This means that the supplier can not change freely, as these changes could break the customer. Even when using an anti-corruption layer, the changes in the supplier need to be coordinated with updates in the anti-corruption layer, so it adapts to those changes.

In other words, the teams responsible for these modules must work closely to coordinate the evolution of these modules:

1. Carefully plan module evolution together;
2. Jointly develop automated test to validate the interfaces and force the supplier to respect these tests so it can evolve without breaking those tests;
3. The customer needs are the priority while deciding the schedule of the supplier.

Crucial, for a customer/supplier relation to work, its that the involved teams work under the same management, who should be as hierarchically close as possible to the actual teams.

Conformist

When we have a customer/supplier relation but, for whatever reason, we don’t have the necessary coordination and motivation of the supplier to coordinate with the customer team, the customer team is on its own when dealing with the supplier evolution.

In such case, we have tree options:

1. Abandon the supplier:  In case there are better options or the added value of maintaining such relation is not worth it. [SEPARATE WAYS]
2. Take full responsibility for translation: If the supplier can not be abandoned but the technical quality is less than acceptable. [ANTI-CORRUPTION LAYER]
3. Adopt the foreign model: If the quality of the supplier is acceptable and compatible, we can fully adopt its model. [CONFORMIST]

The conformist approach can simplify integration enormously, as no translation nor anti-corruption layers would be needed, and it would provide the same ubiquitous language to both teams.

Shared Kernel Vs Conformist

They both deal with a situation where two bounded contexts share part of the model. However, the shared kernel is appropriate only when the teams owning the modules can coordinate and collaborate tightly. When we have a situation where there is a customer/supplier frame, and collaboration is not possible, we need to use a conformist approach, or an Anti-Corruption layer.

Anti-corruption layer

An Anti-Corruption layer should be used when integrating two systems who have different conceptual models. Its purpose is to translate data and functionality between the two systems.

This layer is a system in itself, and it should require no changes in the systems communicating through it. I see it pretty much as a middle-ware.

The Anti-Corruption layer is implemented using:

• FACADE
  An abstraction layer on top of a system API, as to limit and simplify the usage of the underlying API.
• ADAPTER
  Which allow us to connect to different subsystems/APIs through a stable interface, implemented by all adapters who connect to equivalent subsystems/APIs.
• TRANSLATOR
  A stateless object used by the adapters, and which belongs to a specific adapter. They hold the logic to perform the conversion of conceptual objects or Entities, from one subsystem to another.

Translation layer Vs Anti-corruption layer

• The translation layer is collaboratively maintained by the teams owning both bounded contexts.
• The anti-corruption layer is fully maintained by one of the team, the one owning the client module.

Separate ways

Integration is always expensive. Sometimes the benefit is small.

e-book loc. 5968

We should only integrate sub-systems if we really, REALLY, need to. We should, as much as possible, define bounded contexts that are completely independent, completely disconnected from other bounded contexts, and therefore have no need for integration.

Open host service

It is indeed a good practise to create a translation layer for every contact point between bounded contexts.

However, if a bounded context connects to many other bounded contexts, we will be putting a lot of effort on building translators.

To prevent such issues, we can simply create a coherent and clean API in our Bounded Context, supported on a set of Services that provide the functionality needed by the other Bounded Contexts.

This goes in the same direction as the Conformist approach, where the different bounded contexts accept each others model concepts, which provides high integration at a minimum cost.

Published language

The Open Host Service strategy is about creating a bounded context API, supported by Services.

However, data must flow through those Services, in some language they can all understand. This language is a textual representation of the data, its a Data Interchange Language. There are currently several examples of such languages, like XML or JSON, and there are even more specific languages based on base languages like the mentioned XML or JSON. For example, a widely used specific XML representation of pharmaceutical data.

We should use a Data Interchange Language that is the most widely used possible.

Integrating sub-modules

A large scale domain model is composed of smaller domain models.

However, the smaller domain models don’t even have to agree on their view of the global domain model. As long as they operate on their own, we can maintain their integrity.

It’s only when they need to integrate, that we might need to find common views on the models.

Several strategies can be used to integrate sub-models:

• Draw a Context Map of the current state of the model;
• Define the Bounded Contexts borders with the whole dev team;
• Guarantee that the Bounded Contexts borders and relation are known and understood by the whole dev team;
• We must carefully model aiming at small or large Bounded Contexts by considering their advantages:
  • Advantages of large models:
    • Its easier to understand one model than several models plus their mappings;
    • There is no need for translation layers, which might be complex;
    • One shared language eases communication.
  • Advantages of small models:
    • Needs less communication between developers working in the same model;
    • Its easier to manage smaller code bases and teams;
    • Smaller contexts are simpler, requiring less skills from the developers;
    • Smaller models provide more flexibility to the global model.
• Identify and alienate what is not part of our bounded context. A good starting is to exclude what can not change and what we do not need in our bounded context;
• When integrating external systems, we can consider 3 patterns:
  1. SEPARATE WAYS
     Only integrate if we really need to, otherwise keep them completely isolated;
  2. CONFORMIST
     Fully adhere and rely on the external system model;
  3. ANTI-CORRUPTION LAYER
     Completely isolate the two systems, allowing them to communicate through a middle-ware, requiring no changes to either system.
• When integrating internal systems (inside the same bounded context) we can:
  • Use a SHARED KERNEL to split relatively independent functionality into separate bounded contexts.
  • Set up CUSTOMER/SUPPLIER DEV TEAMS, if all the dependencies of the subsystems go in one direction;
  • Go SEPARATE WAYS, if for some reason its not possible to integrate the sub-systems;
  • Use a TRANSLATION LAYER, to connect both systems, which is maintained by the teams of both systems.
• Deployment of new versions must be carefully coordinated by the teams owning the bounded contexts that connect, and as such the Bounded Contexts boundaries must be defined with these issues in mind:
  • Customer/Supplier structures must be tested together to prevent regressions;
  • Long deployments must be managed carefully to minimize issues with data migration temporary inconsistencies;
  • A Shared Kernel update must be tested and verified to satisfy all client systems to prevent regressions.

In the end, like most architectural decisions, the decision of how to integrate the sub-models in our global model, relies on trade-offs.

Here, in the extremes, we are trading off decoupled logic and management of that logic, for easier and faster integration of functionality.

Refactoring Bounded Contexts ecosystem

When we already have a system in place and we want to refactor it, the first step is to identify the current situation. For this, we need to create a schema of the current configuration of the bounded contexts (sub-models) in our system.

From this clear view of the current situation, we can start breaking or merging contexts. While breaking up a bounded context is usually easy, merging them is usually quite difficult, as there can exist very different concepts and models. We will typically want to merge contexts when there is too much translation overhead or duplication.

From SEPARATE WAYS to SHARED KERNEL

1. Define one of 3 strategies:
   • Refactor one context into the other context model;
   • Create a new context with the best pieces of each of the contexts;
   • Create a completely new, deeper model, which can replace both the initial models.
2. Create a small team with developers from both the initial teams;
3. Define the new shared model:
   • Decide the code to be shared, identifying overlaps/duplications in both initial models;
   • Decide naming conventions;
4. Create a basic test suite for the shared kernel, which will grow along with the implementation;
5. Implement the new model, merging frequently and rethinking it when needed;
   • Start simple, with code that is non-critical, duplicated in both contexts. Something being translated is a good choice;
6. Integrate the original contexts with the new Shared Kernel;
7. Remove any obsolete code (translation layers, etc.).

From SHARED KERNEL to CONTINUOUS INTEGRATION

If the Shared Kernel continues to grow, maybe we should completely merge the Bounded Contexts. This, however, involves not only the code but also the teams structure, their work-flow and their language.

1. Create the same work-flow in both teams;
2. Set-up Continuous Integration in both teams;
3. Circulate developers in both teams, as means to sharing and spreading knowledge;
4. Distil each model individually;
5. Start merging the core domain, doing it as fast as possible, prioritizing it above most new development, so we don’t lose the momentum. Beware not to leave out any specialized functionality needed by the users;
6. As the Shared Kernel grows, increase the integration frequency until we have continuous integration;
7. The goal is to, in the end, have one big team, or two small teams who circulate members and do continuous integration.

Phasing out a legacy system

Phasing out a legacy system means, most of the times, replacing it for one or several newer systems.

1. Identify a small piece of the legacy system that can be moved to one of the new systems;
2. Identify additions needed in the Anti-Corruption layer;
3. Implement;
4. Deploy;
5. Identify obsolete code in the Anti-Corruption layer and remove it;
6. Identify obsolete code in the legacy system and, if possible, remove it.

From OPEN HOST SERVICE to PUBLISHED LANGUAGE

Sometimes, when in an Open Host Service architecture, we will have several ad-hoc protocols specific to each communication situation. There is no common, standard communication protocol.

As the need to integrate with more systems grows, the amount of communication specificities grows as well: Scalability and maintainability are at risk.

Therefore we need a common, standard, communication Interchange Language that any system con use to communicate with our system, we need a published language:

1. If possible, use an industry standard language;
2. If its not possible to use an industry standard language, create your own:
   1. Start by making the core domain as clear as possible;
   2. Use the core domain as the base for the new interchange language;
   3. If possible, use a general purpose Interchange Language Syntax (XML, JSON) as a base for our new Interchange Language;
   4. Publish the language, making it known to all involved in the systems collaboration;
   5. Publish any architectural changes as well;
   6. Build translation layers for all collaborating systems;
   7. Switch to the new Interchange Language.

The end goal is that new integrations are made easier, moving the most of the integration effort and responsibility over to the other systems.

This post is part of a set of posts with my personal notes about all the chapters in the book “Domain Driven Design” by Eric Evans. I will do this as I read through the book, and take notes on the concepts I personally find more relevant.