Adding decompress_archive_file_timeout to application settings
What does this MR do and why?
Currently the hardcoded timeout of 210seconds for decompressing archived files is too short for some customers. If we were to move this to application settings, self-managed users can set their own timeouts depending on how long they want to wait.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
| Before | After |
|---|---|
 |
 |
How to set up and validate locally
- Ensure import/export still works.
- Timeout is changeable, by changing the timeout limit in the UI and then going into rails console to check
Gitlab::CurrentSettings.current_application_settings.decompress_archive_file_timeoutor the new database columndecompress_archive_file_timeoutshould reflect the new changed timeout value
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #421432 (closed)
Merge request reports
Activity
changed milestone to %16.4
assigned to @mfanGitLab
3 Warnings 
b1345422: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines. 7f39c2ed: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines. Please add a merge request subtype to this merge request. 2 Messages 
This merge request adds or changes files that require a review from the Database team. This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge. This merge request requires a database review. To make sure these changes are reviewed, take the following steps:
- Ensure the merge request has database and databasereview pending labels. If the merge request modifies database files, Danger will do this for you.
- Prepare your MR for database review according to the docs.
- Assign and mention the database reviewer suggested by Reviewer Roulette.
The following files require a review from the Database team:
db/migrate/20230814181359_add_decompress_archive_file_timeout_to_application_setting.rbdb/schema_migrations/20230814181359db/structure.sql
Documentation review
The following files require a review from a technical writer:
-
doc/administration/settings/account_and_limit_settings.md(Link to current live version) -
doc/api/settings.md(Link to current live version)
The review does not need to block merging this merge request. See the:
-
Metadata for the
*.mdfiles that you've changed. The first few lines of each*.mdfile identify the stage and group most closely associated with your docs change. - The Technical Writer assigned for that stage and group.
- Documentation workflows for information on when to assign a merge request for review.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Piotr Skorupa (
@pskorupa) (UTC+2)Andy Soiron (
@Andysoiron) (UTC+2)database Charlie Ablett (
@cablett) (UTC+12)Andy Soiron @Andysoiron) (UTC+2)frontend Rajan Mistry (
@ramistry) (UTC+5.5)Himanshu Kapoor (
@himkp) (UTC+7)To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-reviewjob that generated this comment.Generated by
Edited by Ghost UserPlease start the
trigger-omnibus-and-follow-up-e2ejob in theqastage and ensure the tests infollow-up-e2e:package-and-test-eepipeline are passing before this MR is merged. (The E2E test pipeline is computationally intensive and we cannot afford running it automatically for all pushes/rebases. Therefore, this job must be triggered manually after significant changes at least once.)If you would like to run all E2E tests, please apply the pipeline:run-all-e2e label and trigger a new pipeline. This will run all tests in
e2e:package-and-testpipeline.The E2E test jobs are allowed to fail due to flakiness. For the list of known failures please refer to the latest pipeline triage issue.
Once done, please apply the
For any questions or help in reviewing the E2E test results, please reach out on the internal #quality Slack channel.
added 173 commits
-
d828f0e2...6fed0664 - 172 commits from branch
master - 8918f20e - Adding decompress_archive_file_timeout to application settings
-
d828f0e2...6fed0664 - 172 commits from branch
- A deleted user
added database databasereview pending labels
- A deleted user
added Data WarehouseImpact Check label
Allure report
allure-report-publishergenerated test report!e2e-test-on-gdk:
expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Manage | 13 | 0 | 1 | 1 | 14 | ❗ | | Govern | 34 | 0 | 0 | 0 | 34 | ✅ | | Create | 36 | 2 | 0 | 2 | 38 | ❌ | | Plan | 47 | 0 | 0 | 0 | 47 | ✅ | | Data Stores | 20 | 0 | 0 | 0 | 20 | ✅ | | Verify | 8 | 0 | 0 | 0 | 8 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 158 | 2 | 1 | 3 | 161 | ❌ | +-------------+--------+--------+---------+-------+-------+--------+e2e-package-and-test:
expand test summary
+-----------------------------------------------------------------------+ | suites summary | +------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------+--------+--------+---------+-------+-------+--------+ | Create | 151 | 0 | 35 | 12 | 186 | ❗ | | Manage | 158 | 1 | 12 | 27 | 171 | ❌ | | Configure | 1 | 0 | 0 | 0 | 1 | ✅ | | Package | 0 | 0 | 2 | 0 | 2 | ➖ | | Plan | 8 | 0 | 0 | 0 | 8 | ✅ | | Data Stores | 4 | 0 | 0 | 0 | 4 | ✅ | | Monitor | 8 | 0 | 0 | 0 | 8 | ✅ | | Framework sanity | 0 | 0 | 2 | 0 | 2 | ➖ | | Govern | 4 | 0 | 0 | 0 | 4 | ✅ | +------------------+--------+--------+---------+-------+-------+--------+ | Total | 334 | 1 | 51 | 39 | 386 | ❌ | +------------------+--------+--------+---------+-------+-------+--------+Edited by Ghost User- A deleted user
added frontend label
Database migrations (on the main database)
Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230814181359 - AddDecompressArchiveFileTimeoutToApplicationSetting Regular 2.2 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Migration: 20230814181359 - AddDecompressArchiveFileTimeoutToApplicationSetting
- Type: Regular
- Duration: 2.2 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 18.7 ms 18.7 ms 18.7 ms 0 ALTER TABLE "application_settings" ADD "decompress_archive_file_timeout" integer DEFAULT 210 NOT NULL
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
1 0.0 ms 0.0 ms 0.0 ms 1 SELECT $1::regtype::oid
Histogram for AddDecompressArchiveFileTimeoutToApplicationSetting
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 4 0.1 seconds - 1 second 0 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
No other migrations pending on GitLab.com
Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-2222216-10922352-main2023-08-16T20:06:50Z 2023-08-16T16:17:51Z 2023-08-17 08:12:32 +0000 database-testing-2222216-10922352-ci2023-08-16T20:06:50Z 2023-08-12T15:50:05Z 2023-08-17 08:12:32 +0000 Database migrations (on the ci database)
1 Warnings 20230814181359 - AddDecompressArchiveFileTimeoutToApplicationSetting had a query that
exceeded timing guidelines. Run time should not exceed 100ms, but it was 102.52ms. Please consider
possible options to improve the query performance.ALTER TABLE "application_settings" ADD
"decompress_archive_file_timeout" integer DEFAULT 210 NOT NULLMigrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).
Migration Type Total runtime Result DB size change 20230814181359 - AddDecompressArchiveFileTimeoutToApplicationSetting Regular 2.7 s +0.00 B Runtime Histogram for all migrations
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 - Type: Regular
- Duration: 2.7 s
- Database size change: +0.00 B
Calls Total Time Max Time Mean Time Rows Query 1 102.5 ms 102.5 ms 102.5 ms 0 ALTER TABLE "application_settings" ADD "decompress_archive_file_timeout" integer DEFAULT 210 NOT NULL
1 0.0 ms 0.0 ms 0.0 ms 1 SELECT $1::regtype::oid
2 0.0 ms 0.0 ms 0.0 ms 2 SELECT pg_backend_pid()
Histogram for AddDecompressArchiveFileTimeoutToApplicationSetting
Query Runtime Count 0 seconds - 0.01 seconds 0 0.01 seconds - 0.1 seconds 3 0.1 seconds - 1 second 1 1 second - 5 seconds 0 5 seconds - 15 seconds 0 15 seconds - 5 minutes 0 5 minutes + 0 Other information
Other migrations pending on GitLab.com
Migration Type Total runtime Result DB size change 20230728171609 - AddCiJobAnnotationsPlanLimits Regular 3.0 s +0.00 B 20230803125434 - AddHasMergeRequestOnVulnerabilityReadsTrigger Regular 2.7 s +0.00 B 20230804064817 - BackfillGoogleCloudLoggingName Regular 2.8 s +0.00 B 20230804065052 - AddNotNullToGcpConfigName Regular 2.5 s +0.00 B 20230807101745 - AddActiveToAuditEventsStreamingHeaders Regular 2.6 s +8.00 KiB [note] 20230807105131 - AddActiveToInstanceAuditEventsStreamingHeaders Regular 2.6 s +0.00 B 20230808140338 - AddFluxResourceColumnToEnvironments Regular 3.4 s +0.00 B 20230809165212 - AddPathPrefixAndBuildRefToPagesDeployments Regular 2.7 s +0.00 B 20230809165213 - AddIndexToPathPrefixAndBuildRefToPagesDeployments Regular 4.2 s +8.00 KiB [note] 20230809192256 - AddFileSizeLimitToPlanLimits Regular 2.6 s +0.00 B 20230810132301 - AddHasRemediationsToVulnerabilityReads Regular 2.6 s +0.00 B 20230814055259 - AddPipelineIdAndExportTypeToDependencyListExports Regular 2.6 s +0.00 B 20230814055310 - AddIndexPipelineIdToDependencyListExports Regular 3.0 s +8.00 KiB [note] 20230701053315 - EnsureAgainBackfillForCiPipelineVariablesPipelineIdIsFinished Post deploy 2.7 s +0.00 B 20230712052619 - DropIndexDeploymentsOnProjectIdAndStatus Post deploy 3.6 s +0.00 B 20230712054057 - DropIndexDeploymentsOnProjectIdSha Post deploy 3.5 s +0.00 B 20230712055956 - DropIndexDeploymentsOnEnvironmentIdAndIidAndProjectId Post deploy 3.6 s +0.00 B 20230726024322 - AddNotValidForeignKeyForCiPipelineVariablesPipelineId Post deploy 3.6 s +8.00 KiB [note] 20230727102936 - DropPreparedAtIndex Post deploy 4.2 s +8.00 KiB [note] 20230727103144 - AddPreparedAtCreatedAtIndex Post deploy 4.0 s +0.00 B 20230727132342 - PrepareIndexOnVulnerabilityOccurrencesUuidAsync Post deploy 3.0 s +0.00 B 20230728122928 - PrepareIndexOnVulnerabilityOccurrencesUuidIncludingVulnerabilityIdAsync Post deploy 2.7 s +0.00 B 20230804053643 - AddTicketWorkItemType Post deploy 2.6 s +0.00 B 20230804121704 - RemoveNamespacesUserDetailsEnterpriseGroupIdFk Post deploy 3.0 s +0.00 B 20230804121705 - RemoveNamespacesUserDetailsProvisionedByGroupIdFk Post deploy 3.0 s +0.00 B 20230809090349 - EnsureIdUniquenessForPCiBuildsV2 Post deploy 3.1 s +0.00 B 20230809133249 - IndexSbomOccurrencesOnProjectIdComponentIdAndInputFilePath Post deploy 3.2 s +8.00 KiB [note] 20230810122746 - EnsureSnippetUserMentionsBigintBackfillIsFinishedForSelfHosts Post deploy 2.5 s +0.00 B 20230810123044 - SwapSnippetUserMentionsNoteIdToBigintForSelfHosts Post deploy 2.5 s +0.00 B 20230811103654 - EnsureVumBigintBackfillIsFinishedForSelfHosts Post deploy 2.4 s +0.00 B 20230811103941 - SwapVulnerabilityUserMentionsNoteIdToBigintForSelfHosts Post deploy 2.5 s +0.00 B Clone details
Clone ID Clone Created At Clone Data Timestamp Expected Removal Time database-testing-2222216-10922352-main2023-08-16T20:06:50Z 2023-08-16T16:17:51Z 2023-08-17 08:12:32 +0000 database-testing-2222216-10922352-ci2023-08-16T20:06:50Z 2023-08-12T15:50:05Z 2023-08-17 08:12:32 +0000
Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic
Edited by Ghost User- A deleted user
added database-testing-automation label
added 1 commit
- 586b0b88 - Adding decompress_archive_file_timeout to application settings
changed milestone to %16.3
changed milestone to %16.4
