Add a cache column for the number of changed files in an MR diff
What does this MR do?
Preparatory work for modifying the external storage migration to work at large scale.
We need to know if a merge request diff has any merge request diff files when performing the external storage migration. Right now, we do this with a subselect (SELECT id from merge_request_diffs WHERE (select * from merge_request_dif_files where merge_request_diffs.id = merge_request_diff_files.id)
or so). This turns out to be expensive - it would be much better to have an indexable column to work against.
This MR adds such a column along with a backfill migration to fill it in for the existing 72M merge request diff rows. A future MR will modify the partial index and change the scheduling queries to make use of it.
Update query
UPDATE "merge_request_diffs" SET files_count = (
SELECT count(*)
FROM merge_request_diff_files
WHERE merge_request_diff_files.merge_request_diff_id = merge_request_diffs.id
)
WHERE "merge_request_diffs"."id" BETWEEN 1 AND 21262 AND "merge_request_diffs"."id" >= 18768 AND "merge_request_diffs"."id" < 20010
Note: uncached execution (DB lab)
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Closes #227570 (closed)