Unified Backups: Introduce `backup_information.json` metadata

What does this MR do and why?

This introduces a new metadata format for the Unified Backups: backup_information.json.

The change from old file in YAML to new file being in JSON is to allow defining a JSON Schema to validate the file in a later stage.

This also makes the two backups incompatible, which removes the need to implement a different validation mechanism on the old one.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

When a Backup is performed, it should include on the root folder a file: backup_information.json with the following keys:

• metadata_version
• backup_id
• created_at
• gitlab_version

{
  "metadata_version": 2,
  "backup_id": "1712944480_2024_04_12_16.11.0-pre",
  "created_at": "2024-04-12T17:54:40Z",
  "gitlab_version": "16.11.0-pre"
}

Related to #427337 (closed)

changed milestone to %16.11

added Category:Backup/Restore of GitLab instances GitLab Free devopssystems featureaddition groupgeo sectioncore platform typefeature workflowin dev + 1 deleted label

assigned to @brodock

changed the description

added backend label

	1 Warning
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.

	1 Message
	CHANGELOG missing: If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog. If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@serenafang (UTC-5, 7 hours behind author)	@hustewart (UTC-4, 6 hours behind author)
~"Backend Static Code Analysis"	Reviewer review is optional for ~"Backend Static Code Analysis"	@ddieulivol (UTC+2, same timezone as author)

Please check reviewer's status!

• Reviewer is available!
• Reviewer is unavailable!

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

added 54 commits

• 60055294...7dc52246 - 52 commits from branch master
• bced5c79 - Introduce `:silence_output` to prevent stdout and stderr when testing
• f2ecb106 - Introduce a way to export Backup Metadata into JSON

Compare with previous version

changed the description

changed the description

added 3 commits

• 6f69c5c7 - Introduce `:silence_output` to prevent stdout and stderr when testing
• 256523d2 - Introduce a way to export Backup Metadata into JSON
• f9ef9f82 - Write metadata when creating a backup

Compare with previous version

added 2 commits

• 6bbf4cbe - Introduce a way to export Backup Metadata into JSON
• ff125b2e - Write metadata when creating a backup

Compare with previous version

requested review from @mkozono

@mkozono could you review / merge this change? This introduces a new backup metadata in the Unified Backup version. We are not carrying over most of the stuff from the old one, this seems to be the required fields at this point.

added 2 commits

• 418fc6dc - Introduce a way to export Backup Metadata into JSON
• 964fa685 - Write metadata when creating a backup

Compare with previous version

mentioned in issue #427337 (closed)

added 2 commits

• a7c303e6 - Ensure error is raised and handled when metadata fails
• 44444835 - Exit 1 when error happens during backup

Compare with previous version

@mkozono could you please give it another look?

requested review from @mkozono

approved this merge request

added pipeline:mr-approved label

@mkozono, thanks for approving this merge request.

This is the first time the merge request has been approved. To ensure we don't only run predictive pipelines, and we don't break master, a new pipeline will be started shortly.

Please wait for the pipeline to start before resolving this discussion and set auto-merge for the new pipeline. See merging a merge request for more details.

resolved all threads

enabled an automatic merge when the pipeline for 5e2c277a succeeds

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-package-and-test: test report for 44444835

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Systems     | 10     | 0      | 0       | 0     | 10    | ✅     |
| Package     | 0      | 0      | 2       | 0     | 2     | ➖     |
| Govern      | 6      | 0      | 0       | 0     | 6     | ✅     |
| Create      | 16     | 0      | 6       | 0     | 22    | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Data Stores | 4      | 0      | 0       | 0     | 4     | ✅     |
| Plan        | 8      | 0      | 0       | 0     | 8     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 52     | 0      | 8       | 0     | 60    | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

merged

mentioned in commit e257f03e

mentioned in incident gitlab-org/quality/engineering-productivity/master-broken-incidents#5875 (closed)

mentioned in incident gitlab-org/quality/engineering-productivity/master-broken-incidents#5877 (closed)

added workflowstaging-canary label and removed workflowin dev label

added workflowcanary label and removed workflowstaging-canary label

added workflowstaging label and removed workflowcanary label

added workflowproduction label and removed workflowstaging label

added workflowpost-deploy-db-staging label and removed workflowproduction label

added releasedcandidate label

mentioned in task #456789 (closed)

added releasedpublished label and removed releasedcandidate label

added pipelinetier-3 label