Commit da018815 authored by Fabio Pitino's avatar Fabio Pitino
Browse files

Add domain layer isolation page, replacing packages extraction

Introduce domain_layer.md as the single source of truth for isolating the
domain layer — exclusive data ownership, boundary enforcement, decoupling
shared models, and the path to get there — and remove the older
packages_extraction.md it supersedes.

- Repoint the decomposition overview to "Isolate the domain layer".
- Move the feature-category-to-bounded-context mapping guidance into
  bounded_contexts.md.
- Condense the hexagonal monolith's application-domain section to point to
  the new page.

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent df36f3a6
Loading
Loading
Loading
Loading
+1 −1
Changes for content/handbook/engineering/architecture/design-documents/modular_monolith/_index.md: 1 added line, 1 removed line.
Original line number Diff line number Diff line
@@ -179,7 +179,7 @@ add more important details as we move forward towards the goal:
1. [Extract the transport layer into gems](transport_layer.md), pulling the
   Web, REST, GraphQL, and Sidekiq transport into transport gems. This
   enables runtime profiles such as API-only or Sidekiq-only nodes.
1. [Separate domains into modules](packages_extraction.md) that will reflect product structure.
1. [Isolate the domain layer](domain_layer.md) so each domain owns its data and is reached only through its public API.
1. Start a training program for team members on how to work with decoupled domains (TODO)
1. Build tools that will make it easier to build decoupled domains through inversion of control (TODO)
1. [Introduce hexagonal architecture within the monolith](hexagonal_monolith/index.md)
+21 −0
Changes for content/handbook/engineering/architecture/design-documents/modular_monolith/bounded_contexts.md: 21 added lines, 0 removed lines.
Original line number Diff line number Diff line
@@ -36,6 +36,27 @@ In May 2024 we [defined and enforced bounded contexts](decisions/002_bounded_con
   and new bounded contexts need to adhere to the characteristics defined previously.
1. Enforce the list of bounded contexts so that no new top-level namespaces can be used aside from the authorized ones.

## Mapping bounded contexts to feature categories

Bounded contexts are anchored on feature categories. A feature category represents a
product area large enough for a module to be **deep**, which keeps us from proliferating
small top-level modules. Anchoring on feature categories also keeps the codebase aligned
with the product's [ubiquitous language](https://docs.gitlab.com/ee/development/software_design.html#use-ubiquitous-language-instead-of-crud-terminology)
and helps new contributors — GitLab team members and the wider community — navigate the
code.

A team can be responsible for multiple feature categories, and therefore own the vision
for multiple bounded contexts. When a feature category changes ownership, remapping the
bounded context to its new owners is cheap.

Feature categories and bounded contexts are not always one-to-one:

- If multiple feature categories are strongly related, they may be grouped under a single
  bounded context.
- If a feature category is only relevant within a parent feature category, it may be folded
  into the parent's bounded context — for example, build artifacts exist within the
  Continuous Integration feature category and may be merged under a single bounded context.

## Iterations

1. [Extract libraries out of the `lib/` directory](https://gitlab.com/gitlab-org/gitlab/-/blob/4c6e120069abe751d3128c05ade45ea749a033df/doc/development/gems.md).
+261 −0
Changes for content/handbook/engineering/architecture/design-documents/modular_monolith/domain_layer.md: 261 added lines, 0 removed lines.
Original line number Diff line number Diff line
---
title: "Isolating the domain layer"
status: proposed
creation-date: "2026-06-12"
authors: [ "@fabiopitino", "@ayufan" ]
coach: [ ]
approvers: [ ]
owning-stage: ""
toc_hide: true
---

This page is the single source of truth for isolating the **application domain**
the business logic at the core of the [hexagonal monolith](hexagonal_monolith/index.md)
into bounded, independently-owned modules.

It covers domain code only. Extracting cross-cutting **platform** code is covered in
[Extracting cross-cutting libraries into gems](library_extraction.md); extracting the
**transport layer** (Web, REST, GraphQL, Sidekiq) is covered in
[Extracting the transport layer into gems](transport_layer.md). How domains are
identified and named is covered in [Defining bounded contexts](bounded_contexts.md).

Domain isolation is the hardest part of modularization. The platform and transport
layers have relatively clean seams; the domain is where the coupling actually lives.

## What domain isolation means

A domain is isolated when:

- **Nothing crosses its boundary except through its public API.** Other domains call
  a documented, intentional interface — they never reach into internals, AR models,
  or private services.
- **It exclusively owns its data.** The domain owns its database tables and the
  ActiveRecord models that map them. No other domain queries those tables or holds
  references to those models.

A well-designed domain module is also **deep** — it encapsulates a large amount of
internal logic, state, and data behind that small interface — and **cohesive**, acting as
the single source of truth for the feature it describes. It follows the
[guideline on naming namespaces](https://docs.gitlab.com/ee/development/software_design.html#use-namespaces-to-define-bounded-contexts)
and uses [ubiquitous language](https://docs.gitlab.com/ee/development/software_design.html#use-ubiquitous-language-instead-of-crud-terminology)
rather than CRUD terminology. How domains are identified and mapped to feature categories
is covered in [Defining bounded contexts](bounded_contexts.md).

Everything below is about the work required to get there.

## The hard part: cross-domain coupling

Our modular-monolith research surfaced the coupling that makes this difficult. In the
current codebase:

- **Policies depend on other policies** — for example `PipelinePolicy` delegates to
  `ProjectPolicy`.
- **Services depend on shared components** — publishing events, sending mail, system
  notes, auditing, and so on.
- **Domains depend on each other's AR models**, both explicitly (`Ci::Pipeline.find_by_id`)
  and implicitly through associations (`security_scan.pipeline`).
- **Circular dependencies** between domains are common.

This is why domain isolation cannot be a mechanical file move the way library and
transport extraction largely can. Most of the effort is *refactoring away the coupling*,
not relocating code.

## Owning the data

Exclusive data ownership is the core of isolation. Each domain owns its tables and the
AR models that map them, and is the only code allowed to query them.

Two enforcement problems follow:

**Cross-domain AR associations.** Today any domain can call `project.ci_pipelines` and
walk straight into CI's data. Isolation requires removing cross-domain associations and
going through the owning domain's public API instead — `Ci::Pipeline.all(project)` rather
than `project.ci_pipelines`. A prerequisite is
[Taming Omniscient Classes](https://docs.gitlab.com/development/software_design/#taming-omniscient-classes):
classes like `Project` and `User` accrete associations and methods from every domain and
must be slimmed down.

**Direct model references.** Even with associations gone, any code can reference
`Ci::Pipeline` directly. One option is to keep the AR model private to the domain — for
example `Ci::Internal::Pipeline` — so it cannot be referenced from outside, exposing
pipeline data only through the `Ci::*` public interface.

### Repository-pattern option

A more structured variant uses AR strictly as a persistence/repository layer, separate
from the domain object:

```ruby
# Persistence only — scopes and AR persistence, nothing else. Private to the domain.
Ci::Repository::Pipeline

# Domain object wrapping the record, exposing behaviour. The public type.
Ci::Pipeline   # #builds, #cancelable?, ...
```

The AR model is never used outside its domain object. This makes the domain trivially
stubbable in tests and removes the AR dependency from callers. The cost is significant:
it is a large, invasive refactor regardless of whether we ultimately extract gems.

## Cross-domain data and shared models

The thorniest question is what happens to shared entities like `Project` and `User`,
which nearly every domain touches.

**Strip shared models to identity objects.** If a cross-domain accessor returns the full
`Project` AR object, it drags in that object's entire surface and re-couples the domains.
Shared data should be reduced to a minimal identity object, not the full model.

**Domain-specific representations.** Rather than every domain consuming a shared
`Project`, a domain can wrap the data it needs behind its own facade:

```ruby
project = Ci::Internal::Project.new(project_id) # PORO or record local to the CI domain
project.variables       # internal to CI
project.public_builds?  # CI-specific project setting
project.pipelines       # pipelines queried via Ci::Project, not Project
```

This keeps `project.pipelines` inside the CI domain instead of on the global `Project`.
A related idea is per-domain owner objects — for example `Ci::JobOwner` wrapping a
`user_id` and delegating to `User` only for identity-specific concerns.

**Prefer direct dependencies over monolith callbacks.** Where a domain needs data from
another domain, prefer a direct, declared dependency on that domain's public interface
over wiring callbacks back into the monolith. Direct dependencies are trackable — we can
render a cross-domain dependency graph and use it in CI to run only the affected tests —
whereas callbacks hide the coupling.

**A stable client interface, swappable to gRPC later.** A domain can be treated as
external from the start: it exposes a Ruby client interface and keeps everything else
private. The backend behind that interface can later move to gRPC without changing
callers.

**Avoid chatty domains.** Once boundaries are enforced we will find components that are
too dependent on each other — Rails makes everything available all the time, so a single
business transaction reaches across many domains. Avoiding chatty cross-domain calls
(a real problem once domains become services) means denormalizing or duplicating data,
keeping local representations, and applying interface segregation — for example a slim
`Ci::Internal::User` that wraps a basic identity object instead of the full `User`.

> The shared "scaffolding" every domain needs — `Project`/`User` lookups, permission
> checks, and similar — might instead live in a coarse `gitlab-platform`/`gitlab-core`
> gem that every domain depends on, rather than being injected per-domain. That
> trade-off is tracked as an
> [open question on the library-extraction page](library_extraction.md#open-questions).

## What must be resolved before extraction

Domain isolation is mostly unanswered design work. Before extracting a domain — into a
gem **or** a package — we need reproducible guidelines, so the refactor is not left to
per-engineer or per-agent interpretation, for at least:

- What happens to cross-domain **AR associations**.
- What logic lives on the **AR model** versus the **domain object**.
- What data is allowed to **cross a boundary**, and in what shape.
- What to do when a caller receives a data object but needs a method on a **domain
  object** (for example `Ci::Pipeline#merge_request`) — without creating chatty domains
  or over-serializing responses (a current pain in REST).
- Who **owns database migrations**, and where they live.
- Whether `Project` is a **shared model** or becomes a `Projects` **domain**.
- How **EE and JH** extensions are isolated alongside the domain — the same extension
  problem the library and transport layers face.
- How **tests** decouple. Factories are global and heavily coupled. As domains decouple,
  each should expose a public interface for specs/factories, or callers should stub
  rather than build cross-domain records:

  ```ruby
  RSpec.describe Ci::Pipeline do
    let(:user)    { Gitlab::Platform::Specs.create_user }
    let(:project) { Gitlab::Platform::Specs.create_project }
  end
  ```

The glue code that bridges domains during the transition could be very large. We likely
need a proof of concept to learn what it actually contains before committing to a
strategy.

## Where domain code lives: `domains/` vs `gems/`

Domain code should live under a dedicated `domains/` directory, separate from the `gems/`
directory used for cross-cutting libraries. Both can use the gem format, but separating
them buys us:

- No flat sprawl of mixed-responsibility gems under a single `gems/`.
- A clear distinction between **domain code** and **generic code**.
- The ability to track and visualise **cross-domain** dependencies separately from
  dependencies on generic libraries — an architecture-level diagram of domain
  relationships.
- An enforceable rule: a library may depend on another library, but **never on a
  domain**. Static analysis can enforce this in CI.

Domains follow [`config/bounded_contexts.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/bounded_contexts.yml)
and use their bounded-context namespace directly (`Ci::`, `Packages::`) — not a `Gitlab::`
prefix.

Optionally, a large domain can be broken into nested, private sub-domain gems:

```ruby
# domains/ci/ci.gemspec  (the public domain gem)
spec.add_dependency "ci-pipelines"
spec.add_dependency "ci-runners"
spec.add_dependency "ci-catalog"
```

## How we get there: gems, Packwerk, or both

**This is not decided.** We have not agreed whether domain decomposition uses Ruby gems,
[Packwerk](https://github.com/Shopify/packwerk) packages, or a combination — because the
decoupling challenges above dominate the choice of mechanism.

The trade-off:

- **Gems** give hard isolation and explicit, declared dependencies. But a gem extraction
  is **atomic**: the code must be fully isolated in a single step. That is painful for
  large, complex, tightly-coupled domains that can only be untangled gradually.
- **Packwerk (optional).** Static analysis can assist gradual, guided isolation *in
  place* — drawing a package boundary without moving files, surfacing privacy and
  dependency violations, and letting a domain's owners work the violations down over
  time. With RBS type signatures it can even catch implicit dependencies such as
  `security_scan.pipeline` when `Ci::Pipeline` is not an allowlisted constant. Once a
  domain is sufficiently isolated this way, a later gem extraction is far simpler.

The two are not mutually exclusive: an extracted gem can carry its own Packwerk
configuration to keep enforcing what it may depend on. Packwerk is a **complementary,
optional** tool for the gradual path — not a committed approach, and not a replacement
for gems.

### An optional gradual path with Packwerk

If we take the gradual route, the steps below describe how it could work.
They are **one option**, not a decision:

1. **Namespace first.** Put all classes and modules for a
   [bounded context](bounded_contexts.md) under the same namespace. Without a rough
   understanding of the domains no plan is possible; consistent namespacing is the
   prerequisite.
1. **Prepare Rails for Packwerk** — a once-off step: make the autoloader work with
   Packwerk's directory layout (as in
   [this PoC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/129254/diffs#note_1512982957)),
   run [Danger-Packwerk](https://github.com/rubyatscale/danger-packwerk) in CI, and
   optionally a pre-commit/pre-push check.
1. **Move files into a package** — create the package and move files in iteratively.
   Constants autoload whether under `app/` or `lib/` inside a package. Code is split
   between the package and the Rails tree during this phase, so move quickly.
1. **Enforce boundaries** — require packages to
   [declare dependencies explicitly](https://github.com/Shopify/packwerk/blob/main/USAGE.md#enforcing-dependency-boundary)
   and depend only on a package's
   [public interface](https://github.com/rubyatscale/packwerk-extensions#privacy-checker).
   Enforcing privacy *after* moving files reveals the real coupling between constants and
   domains as recorded violations (like RuboCop TODOs).
1. **Work off the recorded violations** — a long-term phase the domain's DRIs nurture:
   make over-coupled constants private, move a constant to a better-fitting package, or
   merge packages that are too coupled, using the dependency diagram to guide the design.

Once the tooling exists, emerging domains can be implemented as packages from the start
and get isolation and a clear interface immediately.

## Related

- [Defining bounded contexts](bounded_contexts.md) — how domains are identified and named.
- [Hexagonal Rails Monolith](hexagonal_monolith/index.md) — the three-layer model.
+8 −28
Changes for content/handbook/engineering/architecture/design-documents/modular_monolith/hexagonal_monolith/index.md: 8 added lines, 28 removed lines.
Original line number Diff line number Diff line
@@ -43,37 +43,17 @@ flowchart TD
### Application domain

The application core (functional domains) is composed of all the code that describes the business logic, policies and data
that is unique to GitLab product.
It is divided into separate top-level [bounded contexts](../bounded_contexts.md).
A bounded-context is represented in the form of a module.
This follows the existing [guideline on naming namespaces](https://docs.gitlab.com/ee/development/software_design.html#use-namespaces-to-define-bounded-contexts)
but puts more structure to it.

Modules should:

- Be deep enough to encapsulate a lot of the internal logic, state and data.
- Have a public interface that is as small as possible, safe to use by other bounded contexts and well documented.
- Be cohesive and represent the SSoT (single source of truth) of the feature it describes.

Feature categories represent a product area that is large enough for the module to be deep, so we don't have a proliferation
of small top-level modules. It also helps the codebase to follow the
[ubiquitous language](https://docs.gitlab.com/ee/development/software_design.html#use-ubiquitous-language-instead-of-crud-terminology).
A team can be responsible for multiple feature categories, hence owning the vision for multiple bounded contexts.
While feature categories can sometimes change ownership, this change of mapping the bounded context to new owners
is very cheap.
Using feature categories also helps new contributors, either as GitLab team members of members of the wider community,
to navigate the codebase.

If multiple feature categories are strongly related, they may be grouped under a single bounded context.
If a feature category is only relevant in the context of a parent feature category, it may be included in the
parent's bounded context. For example: Build artifacts existing in the context of Continuous Integration feature category
and they may be merged under a single bounded context.
that is unique to the GitLab product. It is divided into separate top-level [bounded contexts](../bounded_contexts.md),
each represented as a module that owns its data and exposes a small, well-documented public interface.

The application domain has no knowledge of outer layers like the application adapters and only depends on the
platform code. This makes the domain code to be the SSoT of the business logic, be reusable and testable regardless
whether the request came from the WebUI or REST API.
platform code. This makes the domain code the SSoT of the business logic, reusable and testable regardless of
whether the request came from the WebUI or REST API. If an inner layer needs something from an outer layer, that
is solved with inversion of control, especially dependency injection.

If a dependency between an outer layer and an inner layer is required (domain code depending on the interface of an adapter), this can be solved using inversion of control techniques, especially dependency injection.
Isolating the domain — exclusive data ownership, boundary enforcement, decoupling shared models, and how we get
there — is documented in full on its own page, which is the single source of truth:
[Isolating the domain layer](../domain_layer.md).

### Application adapters / Transport layer

+0 −52
Changes for content/handbook/engineering/architecture/design-documents/modular_monolith/packages_extraction.md: 0 added lines, 52 removed lines.
Original line number Diff line number Diff line
---
title: "Convert domain module into packages"
status: proposed
creation-date: "2023-09-29"
authors: [ "@fabiopitino" ]
coach: [ ]
approvers: [ ]
owning-stage: ""
toc_hide: true
---

The general steps of refactoring existing code to modularization could be:

1. Use the same namespace for all classes and modules related to the same [bounded context](bounded_contexts.md).

   - **Why?** Without even a rough understanding of the domains at play in the codebase it is difficult to draw a plan.
     Having well namespaced code that everyone else can follow is also the pre-requisite for modularization.
   - If a domain is already well namespaced and no similar or related namespaces exist, we can move directly to the
     next step.
1. Prepare Rails development for Packwerk packages. This is a **once off step** with maybe some improvements
   added over time.

   - We will have the Rails autoloader to work with Packwerk's directory structure, as demonstrated in
     [this PoC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/129254/diffs#note_1512982957).
   - We will have [Danger-Packwerk](https://github.com/rubyatscale/danger-packwerk) running in CI for merge requests.
   - We will possibly have Packer check running in Lefthook on pre-commit or pre-push.
1. Move file into a Packwerk package.

   - This should consist in creating a Packwerk package and iteratively move files into the package.
   - Constants are auto-loaded correctly whether they are in `app/` or `lib/` inside a Packwerk package.
   - This is a phase where the domain code will be split between the package directory and the Rails directory structure.
     **We must move quickly here**.
1. Enforce namespace boundaries by requiring packages declare their [dependencies explicitly](https://github.com/Shopify/packwerk/blob/main/USAGE.md#enforcing-dependency-boundary)
   and only depend on other packages' [public interface](https://github.com/rubyatscale/packwerk-extensions#privacy-checker).

   - **Why?** Up until now all constants would be public since we have not enforced privacy. By moving existing files
     into packages without enforcing boundaries we can focus on wrapping a namespace in a package without being distracted
     by Packwer privacy violations. By enforcing privacy afterwards we gain an understanding of coupling between various
     constants and domains.
   - This way we know what constants need to be made public (as they are used by other packages) and what can
     remain private (taking the benefit of encapsulation). We will use Packwerk's recorded violations (like RuboCop TODOs)
     to refactor the code over time.
   - We can update the dependency graph to see where it fit in the overall architecture.
1. Work off Packwerk's recorded violations to make refactorings. **This is a long term phase** that the DRIs of the
   domain need to nurture over time. We will use Packwerk failures and the dependency diagram to influence the modular design.

   - Revisit wheteher a class should be private instead of public, and crate a better interface.
   - Move constants to different package if too coupled with that.
   - Join packages if they are too coupled to each other.

Once we have Packwerk configured for the Rails application (step 2 above), emerging domains could be directly implemented
as Packwerk packages, benefiting from isolation and clear interface immediately.