Skip to content

Move Architecture Blueprints to Public Handbook

We'd like to move the Architecture pages from the docs site to our public handbook. This includes the top level page and all sub pages.

We're moving from: https://docs.gitlab.com/ee/architecture/

To: https://handbook.gitlab.com/handbook/engineering/

Also need to verify and update any links in https://handbook.gitlab.com/handbook/engineering/architecture/workflow/

Why?

The primary driver of this work is to avoid architecture pages showing up in search results through the docs site.

Action items

  1. ✅ Add all existing coaches as code owners for the /engineering/architecture folder.
  2. ✅ Introduce a shortcode in the public handbook project to print the design documents list so it matches the current list in the docs site.
  3. ✅ Add notes to the "old" design docs page and the design docs template that we'll migrate the content
  4. ✅ Rename the existing gitlab-com label architecture blueprint to Architecture Evolution Design Doc and update the description.
  5. ✅ Add a note to danger so team members know that design docs will be migrated
  6. ✅ Add the new design documents list page
  7. ✅ Update the documentation (Architecture workflow) to reflect the repository changes.
  8. ✅ Migrate a single design document to test how things work (to ensure we don't break design docs that are often referenced).
  9. ✅ Enable danger review with the architecture plugin in the public handbook. Tahnks @splattael
  10. We decided to migrate the content in roughly 4 chunks (divided by status), starting with implemented and rejected documents. Therefore we need a short freeze for changes to design documents, which we'll share in the slack channels #architecture, #engineering-fyi, #handbook and on the handbook itself.
    1. ✅ First posting in Slack
    2. ✅ Comment on all open design doc MRs so that authors are aware of the change
    3. The migration of content will involve a MR that adds the new content to the handbook and another MR that removes the content from the gitlab project and sets up redirects.
      1. ✅ Set up redirect for the first design doc
    4. ✅ Migrate the first document
    5. ✅ Announce migration of all rejected and implemented docs
    6. ✅ Migrate all rejected and implemented docs
    7. ✅ Announce migration of all other docs with a date range
    8. ✅ Migrate all other docs
    9. ✅ Notify all authors of MRs that add new design docs about how they can add their content in the handbook.
    10. After the migration replace all remaining links to */blueprints/* with local links.

Future iterations

  1. Add review roulette that takes the coach from the design docs front matter.
  2. Check whether all front matter keys exist.

Migration process

This is to document the migration progress. See this list with all design documents from the gitlab project sorted by creation date:

Design document folder Creation Date Migrated? Redirect?
email_ingestion ✅ ✅
cloud_native_gitlab_pages 2019-05-16 ✅ ✅
feature_flags_development 2020-06-10 ✅ ✅
cloud_native_build_logs 2020-08-26 ✅ ✅
container_registry_metadata_database 2020-09-29 ✅ ✅
image_resizing 2020-10-21 ✅ ✅
gitlab_to_kubernetes_communication 2020-12-03 ✅ ✅
graphql_api 2021-01-07 ✅ ✅
ci_scale 2021-01-21 ✅ ✅
consolidating_groups_and_projects 2021-02-07 ✅ ✅
database_testing 2021-02-08 ✅ ✅
composable_codebase_using_rails_engines 2021-05-19 ✅ ✅
ci_data_decay 2021-09-10 ✅ ✅
object_storage 2021-11-18 ✅ ✅
runner_scaling 2022-01-19 ✅ ✅
cells 2022-09-07 ❗ Open MRs gitlab-org/gitlab!148010 (closed) gitlab-org/gitlab!146768 (merged) gitlab-org/gitlab!156678 (merged) gitlab-org/gitlab!151365 (closed) ✅ Separate MR for cells migration !7085 (merged) 👷 Separate MR for cells migration gitlab-org/gitlab!159216 (merged)
rate_limiting 2022-09-08 ✅ ✅
ci_pipeline_components 2022-09-14 ✅ ✅
work_items 2022-09-28 ✅ ✅
runner_tokens 2022-10-27 ✅ ✅
observability_metrics 2022-11-09 ✅ ✅
remote_development 2022-11-15 ✅ ✅
gitlab_agent_deployments 2022-11-23 ✅ ✅
secret_detection 2022-11-25 ✅ ✅
code_search_with_zoekt 2022-12-28 ✅ Migration MR until here (w/o cells) !7003 (merged) ✅ Removal MR until here (w/o cells) gitlab-org/gitlab!158915 (merged)
clickhouse_ingestion_pipeline 2023-01-10 ✅ ✅
ci_builds_runner_fleet_metrics 2023-01-25 ✅ ✅
gitaly_plugins 2023-02-01 ✅ ✅
clickhouse_usage 2023-02-02 ✅ ✅
clickhouse_read_abstraction_layer 2023-02-23 ✅ ✅
model_experiments_and_registry 2023-03-04 ✅ ✅
gitlab_events_platform 2023-03-06 ✅ ✅
runner_admission_controller 2023-03-07 ✅ ✅
permissions 2023-03-10 ✅ ✅
gitlab_ci_events 2023-03-15 ✅ ✅
object_pools 2023-03-30 ✅ ✅
organization 2023-04-05 ✅ ✅
gitlab_ml_experiments 2023-04-13 ✅ ✅
repository_backups 2023-04-26 ✅ ✅
ci_pipeline_processing 2023-05-15 ✅ ✅
git_data_offloading 2023-05-19 ✅ ✅
modular_monolith 2023-05-22 ✅ Part of "rest MR" !7051 (merged) ❗ Open MRs gitlab-org/gitlab!135802 ✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
gitaly_adaptive_concurrency_limit 2023-05-30 ✅ ✅
gitaly_transaction_management 2023-05-30 ✅ Migration MR until here !7024 (merged) ✅ Migration MR until here gitlab-org/gitlab!158944 (merged)
ai_context_management 2023-06-03 ✅ Part of "rest MR" !7051 (merged) ❗ Merged MR gitlab-org/gitlab!158265 (merged) ✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
container_registry_metadata_database_self_managed_rollout 2023-06-09 ✅ ✅
gitaly_handle_upload_pack_in_http2_server 2023-06-15 ✅ ✅
observability_tracing 2023-06-20 ✅ ✅
ai_gateway 2023-07-14 ✅ ✅
ssh_certificates 2023-07-28 ✅ ✅
runway 2023-07-31 ✅ ✅
bundle_uri 2023-08-04 ✅ ✅
secret_manager 2023-08-07 ✅ ✅
gitlab_services 2023-08-18 ✅ ✅
gitlab_steps 2023-08-23 ✅ Part of "rest MR" !7051 (merged) ❗ Open MRs gitlab-org/gitlab!138092 (closed) ✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
google_artifact_registry_integration 2023-08-31 ✅ ✅
transfer_data 2023-09-07 ✅ ✅
capacity_planning 2023-09-11 ✅ ✅
observability_for_self_managed 2023-09-11 ✅ ✅
activity_pub 2023-09-12 ✅ ✅
cloud_connector 2023-09-28 ✅ ✅
rapid_diffs 2023-10-10 ✅ ✅
cdot_orders 2023-10-12 ✅ ✅
gitlab_housekeeper 2023-10-18 ✅ ✅
google_cloud_platform_integration 2023-10-26 ✅ ✅
observability_logging 2023-10-29 ✅ ✅
feature_flags_usage_in_dev_and_ops 2023-11-01 ✅ ✅
pipeline_execution_policy 2023-11-23 ✅ ✅
ci_gcp_secrets_manager 2023-11-29 ✅ ✅
security_policies_database_read_model 2023-12-07 ✅ ✅
tailwindcss 2023-12-21 ✅ Migration MR until here !7028 (merged) ✅ Removal MR until here gitlab-org/gitlab!159079 (merged)
ci_build_speed 2024-01-12 ✅ ✅
gitlab_duo_rag 2024-01-25 ✅ ✅
disaster_recovery 2024-01-29 ✅ ✅
cdot_plan_managment 2024-02-07 ✅ ✅
autoflow 2024-02-16 ✅ ✅
gitlab_rag 2024-02-20 ✅ ✅
custom_models 2024-03-29 ✅ ✅
gitlab_xray_rag 2024-04-23 ✅ ✅
duo_workflow 2024-05-17 ✅ ✅
pipeline_mini_graph 2024-05-27 ✅ ✅
epss 2024-06-19 ✅ Migration MR until here !7051 (merged) ✅ Removal MR until here gitlab-org/gitlab!159113 (merged)
database NotFound ✅ Technically not a design document with proper metadata (and not accessible via list page). Follow-up issue https://gitlab.com/gitlab-org/gitlab/-/issues/472215 ✅
database_scaling NotFound ✅ Technically not a design document with proper metadata (and not accessible via list page). Follow-up issue https://gitlab.com/gitlab-org/gitlab/-/issues/472216
notes_table_partitioning 2023-12-22 ✅ Wasn't listed on the blueprints page because it wasn't in a separate folder. !7085 (merged) 👷 gitlab-org/gitlab!159216 (merged)
backup_and_restore ✅ !7085 (merged) 👷 gitlab-org/gitlab!159216 (merged)

Migrate a design doc to the handbook

❗ Before you start see this list of open MRs that change existing design docs. It's also worth double checking that no new MRs have been started that don't have migration labels. Here's a list of MRs that add new design documents.

Use this script to copy content one design doc folder at a time from the gitlab project to the handbook project and do some adjustments. It does the following:

  1. Copy folder
  2. Rename index.md to _index.md
  3. Remove h1 headline
  4. Add former headline as title front matter attribute
  5. Add toc_hide: true to front matter

Expand to see `migrate.sh` script

 
#!/bin/bash

# Usage:
# ./migrate.sh image_resizing

# Check if a sub path argument is provided
if [ -z "$1" ]; then
  echo "Usage: $0 <design_doc>"
  exit 1
fi

# Your GDK path to blueprints folder
SRC_DIR=~/development/gitlab-development-kit/gitlab/doc/architecture/blueprints
# Your handbook path to design documents folder
DEST_DIR=~/development/handbook/content/handbook/engineering/architecture/design-documents

# Get the sub path from the argument
DESIGN_DOC=$1

echo "Copy document ${DESIGN_DOC}"
cp -R "${SRC_DIR}/${DESIGN_DOC}" "${DEST_DIR}/${DESIGN_DOC}"

echo "Move index.md to _index.md"
mv "${DEST_DIR}/${DESIGN_DOC}/index.md" "${DEST_DIR}/${DESIGN_DOC}/_index.md"

INDEX_FILE="${DEST_DIR}/${DESIGN_DOC}/_index.md"

echo "Add write permission for ${INDEX_FILE}"
chmod g+w "$INDEX_FILE"

# Extract the headline 1 and remove it from the file
HEADLINE=$(sed -n 's/^# \(.*\)/\1/p' "$INDEX_FILE")
sed -i '' '/^# /d' "$INDEX_FILE"

echo "Found and removed headline: ${HEADLINE}"

echo "Add title and toc_hide to front matter"

awk -v title="$HEADLINE" 'BEGIN {found=0} 
    {print} 
    /^---/ && !found {print "title: " title; print "toc_hide: true"; found=1}' "$INDEX_FILE" > "${INDEX_FILE}.tmp" && mv "${INDEX_FILE}.tmp" "$INDEX_FILE"

echo "---"
echo "TODOs:"
echo "1. Add {{< design-document-header >}} to document"
echo "2. Replace all handbook.gitlab.com links with local links."
echo "3. Replace all documentation links with absolute links."
echo "---"

echo "Occurrences of handbook.gitlab.com links that need transforming:"
grep -n "https://.*handbook.gitlab.com" "$INDEX_FILE" || true
echo ""

echo "Occurrences of documentation links that need transforming:"
grep -Eon '\[.*\]\([^)]*\)' "$INDEX_FILE" | grep -v -E '\]\(https?://[^)]*\)' || true

echo "---"

code "$INDEX_FILE"

Usage: ./migrate.sh email_ingestion where email_ingestion is the folder name of a design doc.

Do the following manually after the automated step:

  1. Add {{< design-document-header >}} shortcode after frontmatter to render the design docs header
  2. Move tod_hide: true to the bottom of front matter (just because it's nicer 😛
  3. Replace all handbook.gitlab.com links with local links. The script gives you a nice list of links to work-off
  4. Replace all documentation links with absolute links. For example browse the original document and copy the absolute links. The script gives you a list of occurrences.

Remove a design doc from the GitLab repo and add a redirect

In the GitLab repository do the following:

  1. Open the index.md file of the document in your text editor

  2. Replace the whole content of the file with the following (ensure you replace email-ingestion with the correct path) (taken from the docs):

    ---
redirect_to: 'https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/email-ingestion/'
remove_date: '2025-07-08'
---

This document was moved to [another location]  (https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/email-ingestion/).

<!-- This redirect file can be deleted after <2025-07-08>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->

  3. Do the same for all subpages and ADR records that might exist.

  4. Remove all images in that design docs folder.

❗ Once a MR is set up (and reviewed), ask @phillipwells for the final Technical Writing review and also leave a message in the #docs Slack channel, so that any technical writer can jump in and help. Something along these lines:

Hello everyone :wave: this is another removal and redirect MR for
the architecture design documents migration. [LINK TO MR]

Phillipp Wells can you or someone else from your team please review
and merge this? :bow: Thanks a lot in advance :slight_smile:

Migration issue: https://gitlab.com/gitlab-com/content-sites/handbook/-/issues/279

Notable templating changes for MR authors

  1. Raise a new MR in the public handbook
  2. There are a few notable changes in the template:

    1. Rename your index.md to _index.md. Other files don't need the prepended _. If your design doc uses subfolders which contain index.md files, rename these files to _index.md, too.

    2. Don't use h1s (# ). Instead use the title: Your title frontmatter attribute

    3. Add this entry to frontmatter which prevents the main document page from appearing in the left sidebar navigation. We don't want to have 60+ entries there.

      toc_hide: true

    4. If it exists remove any description attributes from frontmatter.

    5. By default the Hugo template displays a list of subpages with an excerpt below the document if subpages exist. That's a nice feature if you only have a few subpages. If your design document has a lot of subpages you can disable this behavior by adding this to the frontmatter of the _index.md page.

      no_list: true

    6. Add this shortcode directly after frontmatter before any text (blank lines are fine). This displays the design doc header on the detail page which contains authors, coaches, stage etc. and the disclaimer. You don't need to add it to subpages unless you also want to display the meta information there.

      {{< design-document-header >}}

    7. Use absolute links to link to the documentation

    8. Use relative links to link to handbook pages. For example if you want to link to https://handbook.gitlab.com/handbook/values/ use ../../../../values/ or ../../../../values/_index.md from the _index.md file.

    9. If you use images and you're editing a subpage, make sure to include an extra ../ in the image path. Hugo doesn't change the image path and because the URL rendering doesn't use *.html endings you need to go up one folder to make the image visible.

    10. Double-check the appearance and links in your design docs. It's possible we made a mistake during the migration. Thanks 🙌

Links

cc: @gl-docsteam

Edited by Nick Nguyen
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information