Move Architecture Blueprints to Public Handbook

We'd like to move the Architecture pages from the docs site to our public handbook. This includes the top level page and all sub pages.

We're moving from: https://docs.gitlab.com/ee/architecture/

To: https://handbook.gitlab.com/handbook/engineering/

Also need to verify and update any links in https://handbook.gitlab.com/handbook/engineering/architecture/workflow/

Why?

The primary driver of this work is to avoid architecture pages showing up in search results through the docs site.

Action items

✅ Add all existing coaches as code owners for the /engineering/architecture folder.
✅ Introduce a shortcode in the public handbook project to print the design documents list so it matches the current list in the docs site.
✅ Add notes to the "old" design docs page and the design docs template that we'll migrate the content
✅ Rename the existing gitlab-com label architecture blueprint to Architecture Evolution Design Doc and update the description.
✅ Add a note to danger so team members know that design docs will be migrated
✅ Add the new design documents list page
✅ Update the documentation (Architecture workflow) to reflect the repository changes.
✅ Migrate a single design document to test how things work (to ensure we don't break design docs that are often referenced).
✅ Enable danger review with the architecture plugin in the public handbook. Tahnks @splattael
We decided to migrate the content in roughly 4 chunks (divided by status), starting with implemented and rejected documents. Therefore we need a short freeze for changes to design documents, which we'll share in the slack channels #architecture, #engineering-fyi, #handbook and on the handbook itself.
1. ✅ First posting in Slack
2. ✅ Comment on all open design doc MRs so that authors are aware of the change
3. The migration of content will involve a MR that adds the new content to the handbook and another MR that removes the content from the gitlab project and sets up redirects.
  1. ✅ Set up redirect for the first design doc
4. ✅ Migrate the first document
5. ✅ Announce migration of all rejected and implemented docs
6. ✅ Migrate all rejected and implemented docs
7. ✅ Announce migration of all other docs with a date range
8. ✅ Migrate all other docs
9. ✅ Notify all authors of MRs that add new design docs about how they can add their content in the handbook.
10. After the migration replace all remaining links to */blueprints/* with local links.

Future iterations

Add review roulette that takes the coach from the design docs front matter.
Check whether all front matter keys exist.

Migration process

This is to document the migration progress. See this list with all design documents from the gitlab project sorted by creation date:

Design document folder	Creation Date	Migrated?	Redirect?
email_ingestion		✅	✅
cloud_native_gitlab_pages	2019-05-16	✅	✅
feature_flags_development	2020-06-10	✅	✅
cloud_native_build_logs	2020-08-26	✅	✅
container_registry_metadata_database	2020-09-29	✅	✅
image_resizing	2020-10-21	✅	✅
gitlab_to_kubernetes_communication	2020-12-03	✅	✅
graphql_api	2021-01-07	✅	✅
ci_scale	2021-01-21	✅	✅
consolidating_groups_and_projects	2021-02-07	✅	✅
database_testing	2021-02-08	✅	✅
composable_codebase_using_rails_engines	2021-05-19	✅	✅
ci_data_decay	2021-09-10	✅	✅
object_storage	2021-11-18	✅	✅
runner_scaling	2022-01-19	✅	✅
cells	2022-09-07	❗ Open MRs gitlab-org/gitlab!148010 gitlab-org/gitlab!146768 (merged) gitlab-org/gitlab!156678 (merged) gitlab-org/gitlab!151365 ✅ Separate MR for cells migration !7085 (merged)	👷 Separate MR for cells migration gitlab-org/gitlab!159216 (merged)
rate_limiting	2022-09-08	✅	✅
ci_pipeline_components	2022-09-14	✅	✅
work_items	2022-09-28	✅	✅
runner_tokens	2022-10-27	✅	✅
observability_metrics	2022-11-09	✅	✅
remote_development	2022-11-15	✅	✅
gitlab_agent_deployments	2022-11-23	✅	✅
secret_detection	2022-11-25	✅	✅
code_search_with_zoekt	2022-12-28	✅ Migration MR until here (w/o cells) !7003 (merged)	✅ Removal MR until here (w/o cells) gitlab-org/gitlab!158915 (merged)
clickhouse_ingestion_pipeline	2023-01-10	✅	✅
ci_builds_runner_fleet_metrics	2023-01-25	✅	✅
gitaly_plugins	2023-02-01	✅	✅
clickhouse_usage	2023-02-02	✅	✅
clickhouse_read_abstraction_layer	2023-02-23	✅	✅
model_experiments_and_registry	2023-03-04	✅	✅
gitlab_events_platform	2023-03-06	✅	✅
runner_admission_controller	2023-03-07	✅	✅
permissions	2023-03-10	✅	✅
gitlab_ci_events	2023-03-15	✅	✅
object_pools	2023-03-30	✅	✅
organization	2023-04-05	✅	✅
gitlab_ml_experiments	2023-04-13	✅	✅
repository_backups	2023-04-26	✅	✅
ci_pipeline_processing	2023-05-15	✅	✅
git_data_offloading	2023-05-19	✅	✅
modular_monolith	2023-05-22	✅ Part of "rest MR" !7051 (merged) ❗ Open MRs gitlab-org/gitlab!135802	✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
gitaly_adaptive_concurrency_limit	2023-05-30	✅	✅
gitaly_transaction_management	2023-05-30	✅ Migration MR until here !7024 (merged)	✅ Migration MR until here gitlab-org/gitlab!158944 (merged)
ai_context_management	2023-06-03	✅ Part of "rest MR" !7051 (merged) ❗ Merged MR gitlab-org/gitlab!158265 (merged)	✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
container_registry_metadata_database_self_managed_rollout	2023-06-09	✅	✅
gitaly_handle_upload_pack_in_http2_server	2023-06-15	✅	✅
observability_tracing	2023-06-20	✅	✅
ai_gateway	2023-07-14	✅	✅
ssh_certificates	2023-07-28	✅	✅
runway	2023-07-31	✅	✅
bundle_uri	2023-08-04	✅	✅
secret_manager	2023-08-07	✅	✅
gitlab_services	2023-08-18	✅	✅
gitlab_steps	2023-08-23	✅ Part of "rest MR" !7051 (merged) ❗ Open MRs gitlab-org/gitlab!138092	✅ Part of "rest MR" gitlab-org/gitlab!159113 (merged)
google_artifact_registry_integration	2023-08-31	✅	✅
transfer_data	2023-09-07	✅	✅
capacity_planning	2023-09-11	✅	✅
observability_for_self_managed	2023-09-11	✅	✅
activity_pub	2023-09-12	✅	✅
cloud_connector	2023-09-28	✅	✅
rapid_diffs	2023-10-10	✅	✅
cdot_orders	2023-10-12	✅	✅
gitlab_housekeeper	2023-10-18	✅	✅
google_cloud_platform_integration	2023-10-26	✅	✅
observability_logging	2023-10-29	✅	✅
feature_flags_usage_in_dev_and_ops	2023-11-01	✅	✅
pipeline_execution_policy	2023-11-23	✅	✅
ci_gcp_secrets_manager	2023-11-29	✅	✅
security_policies_database_read_model	2023-12-07	✅	✅
tailwindcss	2023-12-21	✅ Migration MR until here !7028 (merged)	✅ Removal MR until here gitlab-org/gitlab!159079 (merged)
ci_build_speed	2024-01-12	✅	✅
gitlab_duo_rag	2024-01-25	✅	✅
disaster_recovery	2024-01-29	✅	✅
cdot_plan_managment	2024-02-07	✅	✅
autoflow	2024-02-16	✅	✅
gitlab_rag	2024-02-20	✅	✅
custom_models	2024-03-29	✅	✅
gitlab_xray_rag	2024-04-23	✅	✅
duo_workflow	2024-05-17	✅	✅
pipeline_mini_graph	2024-05-27	✅	✅
epss	2024-06-19	✅ Migration MR until here !7051 (merged)	✅ Removal MR until here gitlab-org/gitlab!159113 (merged)
database	NotFound	✅ Technically not a design document with proper metadata (and not accessible via list page). Follow-up issue https://gitlab.com/gitlab-org/gitlab/-/issues/472215
database_scaling	NotFound	✅ Technically not a design document with proper metadata (and not accessible via list page). Follow-up issue https://gitlab.com/gitlab-org/gitlab/-/issues/472216
notes_table_partitioning	2023-12-22	✅ Wasn't listed on the blueprints page because it wasn't in a separate folder. !7085 (merged)	👷 gitlab-org/gitlab!159216 (merged)
backup_and_restore		✅ !7085 (merged)	👷 gitlab-org/gitlab!159216 (merged)

Migrate a design doc to the handbook

❗ Before you start see this list of open MRs that change existing design docs. It's also worth double checking that no new MRs have been started that don't have migration labels. Here's a list of MRs that add new design documents.

Use this script to copy content one design doc folder at a time from the gitlab project to the handbook project and do some adjustments. It does the following:

Copy folder
Rename index.md to _index.md
Remove h1 headline
Add former headline as title front matter attribute
Add toc_hide: true to front matter

Expand to see `migrate.sh` script

#!/bin/bash

# Usage:
# ./migrate.sh image_resizing

# Check if a sub path argument is provided
if [ -z "$1" ]; then
  echo "Usage: $0 <design_doc>"
  exit 1
fi

# Your GDK path to blueprints folder
SRC_DIR=~/development/gitlab-development-kit/gitlab/doc/architecture/blueprints
# Your handbook path to design documents folder
DEST_DIR=~/development/handbook/content/handbook/engineering/architecture/design-documents

# Get the sub path from the argument
DESIGN_DOC=$1

echo "Copy document ${DESIGN_DOC}"
cp -R "${SRC_DIR}/${DESIGN_DOC}" "${DEST_DIR}/${DESIGN_DOC}"

echo "Move index.md to _index.md"
mv "${DEST_DIR}/${DESIGN_DOC}/index.md" "${DEST_DIR}/${DESIGN_DOC}/_index.md"

INDEX_FILE="${DEST_DIR}/${DESIGN_DOC}/_index.md"

echo "Add write permission for ${INDEX_FILE}"
chmod g+w "$INDEX_FILE"

# Extract the headline 1 and remove it from the file
HEADLINE=$(sed -n 's/^# \(.*\)/\1/p' "$INDEX_FILE")
sed -i '' '/^# /d' "$INDEX_FILE"

echo "Found and removed headline: ${HEADLINE}"

echo "Add title and toc_hide to front matter"

awk -v title="$HEADLINE" 'BEGIN {found=0} 
    {print} 
    /^---/ && !found {print "title: " title; print "toc_hide: true"; found=1}' "$INDEX_FILE" > "${INDEX_FILE}.tmp" && mv "${INDEX_FILE}.tmp" "$INDEX_FILE"

echo "---"
echo "TODOs:"
echo "1. Add {{< design-document-header >}} to document"
echo "2. Replace all handbook.gitlab.com links with local links."
echo "3. Replace all documentation links with absolute links."
echo "---"

echo "Occurrences of handbook.gitlab.com links that need transforming:"
grep -n "https://.*handbook.gitlab.com" "$INDEX_FILE" || true
echo ""

echo "Occurrences of documentation links that need transforming:"
grep -Eon '\[.*\]\([^)]*\)' "$INDEX_FILE" | grep -v -E '\]\(https?://[^)]*\)' || true

echo "---"

code "$INDEX_FILE"

Usage: ./migrate.sh email_ingestion where email_ingestion is the folder name of a design doc.

Do the following manually after the automated step:

Add {{< design-document-header >}} shortcode after frontmatter to render the design docs header
Move tod_hide: true to the bottom of front matter (just because it's nicer 😛
Replace all handbook.gitlab.com links with local links. The script gives you a nice list of links to work-off
Replace all documentation links with absolute links. For example browse the original document and copy the absolute links. The script gives you a list of occurrences.

Remove a design doc from the GitLab repo and add a redirect

In the GitLab repository do the following:

Open the index.md file of the document in your text editor

Replace the whole content of the file with the following (ensure you replace email-ingestion with the correct path) (taken from the docs):

---
redirect_to: 'https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/email-ingestion/'
remove_date: '2025-07-08'
---

This document was moved to [another location]  (https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/email-ingestion/).

<!-- This redirect file can be deleted after <2025-07-08>. -->
<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->

Do the same for all subpages and ADR records that might exist.
Remove all images in that design docs folder.

❗ Once a MR is set up (and reviewed), ask @phillipwells for the final Technical Writing review and also leave a message in the #docs Slack channel, so that any technical writer can jump in and help. Something along these lines:

Hello everyone :wave: this is another removal and redirect MR for
the architecture design documents migration. [LINK TO MR]

Phillipp Wells can you or someone else from your team please review
and merge this? :bow: Thanks a lot in advance :slight_smile:

Migration issue: https://gitlab.com/gitlab-com/content-sites/handbook/-/issues/279

Notable templating changes for MR authors

Raise a new MR in the public handbook
There are a few notable changes in the template:
1. Rename your index.md to _index.md. Other files don't need the prepended _. If your design doc uses subfolders which contain index.md files, rename these files to _index.md, too.
2. Don't use h1s (# ). Instead use the title: Your title frontmatter attribute
3. Add this entry to frontmatter which prevents the main document page from appearing in the left sidebar navigation. We don't want to have 60+ entries there.
```
toc_hide: true
```
4. If it exists remove any description attributes from frontmatter.
5. By default the Hugo template displays a list of subpages with an excerpt below the document if subpages exist. That's a nice feature if you only have a few subpages. If your design document has a lot of subpages you can disable this behavior by adding this to the frontmatter of the _index.md page.
```
no_list: true
```
6. Add this shortcode directly after frontmatter before any text (blank lines are fine). This displays the design doc header on the detail page which contains authors, coaches, stage etc. and the disclaimer. You don't need to add it to subpages unless you also want to display the meta information there.
```
{{< design-document-header >}}
```
7. Use absolute links to link to the documentation
8. Use relative links to link to handbook pages. For example if you want to link to https://handbook.gitlab.com/handbook/values/ use ../../../../values/ or ../../../../values/_index.md from the _index.md file.
9. If you use images and you're editing a subpage, make sure to include an extra ../ in the image path. Hugo doesn't change the image path and because the URL rendering doesn't use *.html endings you need to go up one folder to make the image visible.
10. Double-check the appearance and links in your design docs. It's possible we made a mistake during the migration. Thanks 🙌