Skip to content

Bump allowed number of queries to unblock patch release

Michael Kozonorequested to merge
bump-allowed-queries-in-destroy-all-expired-service into 14-7-stable-ee-patch-6

What does this MR do and why?

Describe in detail what your merge request does and why.

Fix CI failures in !83054 (merged).

Bump the allowed number of queries during Ci::JobArtifacts::DestroyAllExpiredService.

The new query which is causing the number of queries to cross the limit is: UPDATE "ci_job_artifacts" SET "expire_at" = NULL WHERE 1=0. The where clause looks funny because there are zero wrongly expired artifacts, yet update_all is called anyway, on a relation built with an empty array.

The new query is called at app/services/ci/job_artifacts/destroy_batch_service.rb:142.

We are modifying only the tests, in order to minimize risk while unblocking a patch release.

Related to https://gitlab.com/gitlab-org/release/tasks/-/issues/3612 and #353128 (closed) and #346261 (closed)

More context

From a Slack conversation (internal):

Click here to expand 
Here are the queries:
SELECT "ci_job_artifacts"."expire_at" FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."expire_at" < '2022-03-18 18:01:10.747533' ORDER BY "ci_job_artifacts"."expire_at" DESC LIMIT 1 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT "ci_job_artifacts"."expire_at" FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."expire_at" < '2022-03-18 18:01:10.747533' AND "ci_job_artifacts"."expire_at" <= '2022-03-17 00:00:00' ORDER BY "ci_job_artifacts"."expire_at" DESC LIMIT 1 OFFSET 100 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT "ci_job_artifacts"."project_id", "ci_job_artifacts"."file_type", "ci_job_artifacts"."size", "ci_job_artifacts"."created_at", "ci_job_artifacts"."updated_at", "ci_job_artifacts"."expire_at", "ci_job_artifacts"."file", "ci_job_artifacts"."file_store", "ci_job_artifacts"."file_sha256", "ci_job_artifacts"."file_format", "ci_job_artifacts"."file_location", "ci_job_artifacts"."id", "ci_job_artifacts"."job_id", "ci_job_artifacts"."locked" FROM "ci_job_artifacts" INNER JOIN "ci_builds" ON "ci_builds"."id" = "ci_job_artifacts"."job_id" AND "ci_builds"."type" = 'Ci::Build' INNER JOIN "ci_pipelines" ON "ci_pipelines"."id" = "ci_builds"."commit_id" WHERE "ci_job_artifacts"."expire_at" < '2022-03-18 18:01:10.747533' AND "ci_job_artifacts"."expire_at" <= '2022-03-17 00:00:00' AND "ci_pipelines"."locked" = 0 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_coverage_regex", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."emails_disabled", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id" FROM "projects" WHERE "projects"."id" = 6 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."source_id" = 6 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT "project_statistics".* FROM "project_statistics" WHERE "project_statistics"."project_id" = 6 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
UPDATE "ci_job_artifacts" SET "expire_at" = NULL WHERE 1=0 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
DELETE FROM "security_findings" WHERE "security_findings"."id" IN (SELECT "security_findings"."id" FROM "security_findings" INNER JOIN "security_scans" ON "security_scans"."id" = "security_findings"."scan_id" WHERE "security_scans"."build_id" = 6) /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SAVEPOINT active_record_2 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
DELETE FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."id" = 3 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
RELEASE SAVEPOINT active_record_2 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/
SELECT 1 AS one FROM "geo_nodes" LIMIT 1 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/

mkozono  1 hour ago
When I run this on 14-7-stable-ee, there is one less query. The new query in the patch branch is:
UPDATE "ci_job_artifacts" SET "expire_at" = NULL WHERE 1=0 /*application:test,correlation_id:059b9cdd4259f766c1228d0c97f38c43,db_config_name:main*/

mkozono  44 minutes ago
Side note, the test says within(1).of(10) and there used to be 11 queries. So since the new branch adds 1 query, the test now fails.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Michael Kozono

Merge request reports