Fix the lost-secrets SQL that fails on current schemas
What does this MR do?
#632677 (closed) reports that UPDATE ci_runners SET token = null, token_encrypted = null; in the lost secrets file steps fails because ci_runners no longer has a token column. I ran every SQL block in that section against the schema on master, and two more statements fail the same way. This MR corrects all three:
| Statement on the page | Why it fails now | Changed to |
|---|---|---|
UPDATE ci_runners SET token = null, token_encrypted = null; |
The token column was dropped in 18.4 (!202720 (merged)). |
UPDATE ci_runners SET token_encrypted = null; |
UPDATE ci_builds SET token_encrypted = null; |
ci_builds is a partition of p_ci_builds and moved to the gitlab_partitions_dynamic schema in 18.1 (!191002 (merged)), which is not on the search path. |
UPDATE p_ci_builds SET token_encrypted = null; |
TRUNCATE integrations, ..., web_hook_logs CASCADE; |
web_hook_logs was dropped in 18.0 (!189420 (merged)). TRUNCATE refuses the whole list when one table doesn't exist, so none of the integration and webhook tables is emptied either. |
The same list with web_hook_logs_daily, the partitioned table that replaced it. |
Both corrected UPDATE statements target a partitioned parent and reach every partition: ci_runners is partitioned by runner type, and p_ci_builds holds every builds partition, whichever schema it lives in. The other statements in the section run unchanged on the current schema.
What this MR leaves alone, and why:
- Only these three statements change. The headings and the rest of the steps are as they were.
- The translated pages under
doc-locale/(fr-fr,ja-jpandko-kr) contain the same three statements, and the Japanese page also keeps an olderUPDATE ci_builds SET token = null, token_encrypted = null;for GitLab 15.3 and earlier. I didn't edit them, because.gitlab/CODEOWNERSassigns that directory to the localization maintainers. UPDATE p_ci_builds SET token_encrypted = null;updates every row in every partition, finished jobs included. A build's token is already cleared when the job finishes (build.remove_token!inapp/workers/ci/build_finished_worker.rb:44), soWHERE token_encrypted IS NOT NULLwould limit the write to the jobs that still hold one, andp_ci_buildsalready has a partial index on that condition (p_ci_builds_token_encrypted_partition_id_idx) that PostgreSQL could use to find them instead of reading every row. I kept the statement as broad as it was so that this MR only corrects names, and I can add the condition if you prefer it.
How I checked, against db/structure.sql on master at 6af6ab35
I loaded db/structure.sql into an empty PostgreSQL 17 database. It holds the p_ci_builds and web_hook_logs_daily parents but none of their partitions, which the partition manager creates at runtime (lib/gitlab/database/partitioning/partition_manager.rb:74) in gitlab_partitions_dynamic, the schema both partition classes name (multiple_numeric_list_partition.rb:25 for the builds, time_partition.rb:103 for the hook logs). So I created gitlab_partitions_dynamic.ci_builds (partition 100), gitlab_partitions_dynamic.ci_builds_101 and one web_hook_logs_daily partition, and inserted one row with a token in each builds partition and each runner type, one web hook (with foreign key checks off for that insert, because the database holds no project) and one web hook log. Then I ran every SQL block of the section in page order with psql, with <GROUPID> and <PROJECTID> set to 1, once as the page reads on master and once as it reads on this branch. The search path was the default, "$user", public.
On master, the three statements fail and every token, the web hook and its log are still there:
UPDATE ci_runners SET token = null, token_encrypted = null;
ERROR: column "token" of relation "ci_runners" does not exist
UPDATE ci_builds SET token_encrypted = null;
ERROR: relation "ci_builds" does not exist
TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs CASCADE;
ERROR: relation "web_hook_logs" does not exist
partition | rows | tokens_left
-----------------------------------------+------+-------------
gitlab_partitions_dynamic.ci_builds | 1 | 1
gitlab_partitions_dynamic.ci_builds_101 | 1 | 1
partition | rows | tokens_left
--------------------------+------+-------------
group_type_ci_runners | 1 | 1
instance_type_ci_runners | 1 | 1
project_type_ci_runners | 1 | 1
web_hooks_rows | web_hook_logs_daily_rows
----------------+--------------------------
1 | 1On this branch, every statement succeeds and nothing is left:
UPDATE ci_runners SET token_encrypted = null;
UPDATE 3
UPDATE p_ci_builds SET token_encrypted = null;
UPDATE 2
TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs_daily CASCADE;
NOTICE: truncate cascades to table "dingtalk_tracker_data"
NOTICE: truncate cascades to table "slack_integrations_scopes"
TRUNCATE TABLE
partition | rows | tokens_left
-----------------------------------------+------+-------------
gitlab_partitions_dynamic.ci_builds | 1 | 0
gitlab_partitions_dynamic.ci_builds_101 | 1 | 0
partition | rows | tokens_left
--------------------------+------+-------------
group_type_ci_runners | 1 | 0
instance_type_ci_runners | 1 | 0
project_type_ci_runners | 1 | 0
web_hooks_rows | web_hook_logs_daily_rows
----------------+--------------------------
0 | 0The plans of the two corrected updates:
EXPLAIN (COSTS OFF) UPDATE ci_runners SET token_encrypted = null;
Update on ci_runners
Update on instance_type_ci_runners ci_runners_1
Update on group_type_ci_runners ci_runners_2
Update on project_type_ci_runners ci_runners_3
-> Append
-> Seq Scan on instance_type_ci_runners ci_runners_1
-> Seq Scan on group_type_ci_runners ci_runners_2
-> Seq Scan on project_type_ci_runners ci_runners_3
EXPLAIN (COSTS OFF) UPDATE p_ci_builds SET token_encrypted = null;
Update on p_ci_builds
Update on ci_builds p_ci_builds_1
Update on ci_builds_101 p_ci_builds_2
-> Append
-> Seq Scan on ci_builds p_ci_builds_1
-> Seq Scan on ci_builds_101 p_ci_builds_2The other statements of the section (ci_group_variables, ci_variables, projects, namespaces and application_settings) succeeded in both runs.
Related issues
Closes #632677 (closed)
Author's checklist
- Optional. Consider taking the GitLab Technical Writing Fundamentals course.
- Follow the:
- If you're adding a new page, add the product availability details under the H1 topic title.
- If you are a GitLab team member, request a review based on:
- The documentation page's metadata.
- The associated Technical Writer.
If you are a GitLab team member and only adding documentation, do not add any of the following labels:
~"frontend"~"backend"~"type::bug"~"database"
These labels cause the MR to be added to code verification QA issues.
Reviewer's checklist
Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.
If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.
- If the content requires it, ensure the information is reviewed by a subject matter expert.
- Technical writer review items:
- Ensure docs metadata is present and up-to-date.
- Ensure the appropriate labels are added to this MR.
- Ensure a release milestone is set.
- If relevant to this MR, ensure content topic type principles are in use, including:
- The headings should be something you'd do a Google search for. Instead of
Default behavior, say something likeDefault behavior when you close an issue. - The headings (other than the page title) should be active. Instead of
Configuring GDK, say something likeConfigure GDK. - Any task steps should be written as a numbered list.
- If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
- The headings should be something you'd do a Google search for. Instead of
- Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.