Fix the lost-secrets SQL that fails on current schemas

What does this MR do?

#632677 (closed) reports that UPDATE ci_runners SET token = null, token_encrypted = null; in the lost secrets file steps fails because ci_runners no longer has a token column. I ran every SQL block in that section against the schema on master, and two more statements fail the same way. This MR corrects all three:

Statement on the page Why it fails now Changed to
UPDATE ci_runners SET token = null, token_encrypted = null; The token column was dropped in 18.4 (!202720 (merged)). UPDATE ci_runners SET token_encrypted = null;
UPDATE ci_builds SET token_encrypted = null; ci_builds is a partition of p_ci_builds and moved to the gitlab_partitions_dynamic schema in 18.1 (!191002 (merged)), which is not on the search path. UPDATE p_ci_builds SET token_encrypted = null;
TRUNCATE integrations, ..., web_hook_logs CASCADE; web_hook_logs was dropped in 18.0 (!189420 (merged)). TRUNCATE refuses the whole list when one table doesn't exist, so none of the integration and webhook tables is emptied either. The same list with web_hook_logs_daily, the partitioned table that replaced it.

Both corrected UPDATE statements target a partitioned parent and reach every partition: ci_runners is partitioned by runner type, and p_ci_builds holds every builds partition, whichever schema it lives in. The other statements in the section run unchanged on the current schema.

What this MR leaves alone, and why:

  • Only these three statements change. The headings and the rest of the steps are as they were.
  • The translated pages under doc-locale/ (fr-fr, ja-jp and ko-kr) contain the same three statements, and the Japanese page also keeps an older UPDATE ci_builds SET token = null, token_encrypted = null; for GitLab 15.3 and earlier. I didn't edit them, because .gitlab/CODEOWNERS assigns that directory to the localization maintainers.
  • UPDATE p_ci_builds SET token_encrypted = null; updates every row in every partition, finished jobs included. A build's token is already cleared when the job finishes (build.remove_token! in app/workers/ci/build_finished_worker.rb:44), so WHERE token_encrypted IS NOT NULL would limit the write to the jobs that still hold one, and p_ci_builds already has a partial index on that condition (p_ci_builds_token_encrypted_partition_id_idx) that PostgreSQL could use to find them instead of reading every row. I kept the statement as broad as it was so that this MR only corrects names, and I can add the condition if you prefer it.
How I checked, against db/structure.sql on master at 6af6ab35

I loaded db/structure.sql into an empty PostgreSQL 17 database. It holds the p_ci_builds and web_hook_logs_daily parents but none of their partitions, which the partition manager creates at runtime (lib/gitlab/database/partitioning/partition_manager.rb:74) in gitlab_partitions_dynamic, the schema both partition classes name (multiple_numeric_list_partition.rb:25 for the builds, time_partition.rb:103 for the hook logs). So I created gitlab_partitions_dynamic.ci_builds (partition 100), gitlab_partitions_dynamic.ci_builds_101 and one web_hook_logs_daily partition, and inserted one row with a token in each builds partition and each runner type, one web hook (with foreign key checks off for that insert, because the database holds no project) and one web hook log. Then I ran every SQL block of the section in page order with psql, with <GROUPID> and <PROJECTID> set to 1, once as the page reads on master and once as it reads on this branch. The search path was the default, "$user", public.

On master, the three statements fail and every token, the web hook and its log are still there:

UPDATE ci_runners SET token = null, token_encrypted = null;
ERROR:  column "token" of relation "ci_runners" does not exist
UPDATE ci_builds SET token_encrypted = null;
ERROR:  relation "ci_builds" does not exist
TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs CASCADE;
ERROR:  relation "web_hook_logs" does not exist

                partition                | rows | tokens_left
-----------------------------------------+------+-------------
 gitlab_partitions_dynamic.ci_builds     |    1 |           1
 gitlab_partitions_dynamic.ci_builds_101 |    1 |           1

        partition         | rows | tokens_left
--------------------------+------+-------------
 group_type_ci_runners    |    1 |           1
 instance_type_ci_runners |    1 |           1
 project_type_ci_runners  |    1 |           1

 web_hooks_rows | web_hook_logs_daily_rows
----------------+--------------------------
              1 |                        1

On this branch, every statement succeeds and nothing is left:

UPDATE ci_runners SET token_encrypted = null;
UPDATE 3
UPDATE p_ci_builds SET token_encrypted = null;
UPDATE 2
TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs_daily CASCADE;
NOTICE:  truncate cascades to table "dingtalk_tracker_data"
NOTICE:  truncate cascades to table "slack_integrations_scopes"
TRUNCATE TABLE

                partition                | rows | tokens_left
-----------------------------------------+------+-------------
 gitlab_partitions_dynamic.ci_builds     |    1 |           0
 gitlab_partitions_dynamic.ci_builds_101 |    1 |           0

        partition         | rows | tokens_left
--------------------------+------+-------------
 group_type_ci_runners    |    1 |           0
 instance_type_ci_runners |    1 |           0
 project_type_ci_runners  |    1 |           0

 web_hooks_rows | web_hook_logs_daily_rows
----------------+--------------------------
              0 |                        0

The plans of the two corrected updates:

EXPLAIN (COSTS OFF) UPDATE ci_runners SET token_encrypted = null;
 Update on ci_runners
   Update on instance_type_ci_runners ci_runners_1
   Update on group_type_ci_runners ci_runners_2
   Update on project_type_ci_runners ci_runners_3
   ->  Append
         ->  Seq Scan on instance_type_ci_runners ci_runners_1
         ->  Seq Scan on group_type_ci_runners ci_runners_2
         ->  Seq Scan on project_type_ci_runners ci_runners_3

EXPLAIN (COSTS OFF) UPDATE p_ci_builds SET token_encrypted = null;
 Update on p_ci_builds
   Update on ci_builds p_ci_builds_1
   Update on ci_builds_101 p_ci_builds_2
   ->  Append
         ->  Seq Scan on ci_builds p_ci_builds_1
         ->  Seq Scan on ci_builds_101 p_ci_builds_2

The other statements of the section (ci_group_variables, ci_variables, projects, namespaces and application_settings) succeeded in both runs.

Closes #632677 (closed)

Author's checklist

If you are a GitLab team member and only adding documentation, do not add any of the following labels:

  • ~"frontend"
  • ~"backend"
  • ~"type::bug"
  • ~"database"

These labels cause the MR to be added to code verification QA issues.

Reviewer's checklist

Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.

If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.

  • If the content requires it, ensure the information is reviewed by a subject matter expert.
  • Technical writer review items:
    • Ensure docs metadata is present and up-to-date.
    • Ensure the appropriate labels are added to this MR.
    • Ensure a release milestone is set.
    • If relevant to this MR, ensure content topic type principles are in use, including:
      • The headings should be something you'd do a Google search for. Instead of Default behavior, say something like Default behavior when you close an issue.
      • The headings (other than the page title) should be active. Instead of Configuring GDK, say something like Configure GDK.
      • Any task steps should be written as a numbered list.
      • If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
  • Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.

Merge request reports

Loading
Loading