Backup-utility s3cmd rinclude flag not work result in backup files deleted.
Summary
In backup-utility script, the get_existing_backups
function use s3cmd
tool to get existing backup archive files in s3 storage when storage backend was s3. The command was listed below:
s3cmd ${S3_CMD_BACKUP_OPTION} ls s3://$BACKUP_BUCKET_NAME --rinclude '^\d{10}_\d{4}_\d{2}_\d{2}_.+_gitlab_backup.tar$' | awk '{print $4}' | LC_ALL=C sort
but --include
flags was not work in s3cmd ls
subcommand. The test result was listed below:
s3cmd ls s3://oppo-3k-1308186353/p-gitlab2-backup-1253431691/ --rinclude '^\d{10}_\d{4}_\d{2}_\d{2}_.+_gitlab_backup.tar$' | awk '{print $4}' | LC_ALL=C sort
s3://oppo-3k-1308186353/p-gitlab2-backup-1253431691/
s3://oppo-3k-1308186353/p-gitlab2-backup-1253431691/1711298001_2023_07_29_11_01-jh_gitlab_backup.tar
s3://oppo-3k-1308186353/p-gitlab2-backup-1253431691/2023_07_29_11_01-jh_gitlab_backup.tar
s3://oppo-3k-1308186353/p-gitlab2-backup-1253431691/2023_07_29_11_22-jh_gitlab_backup.tar
This command would list all files in bucket, not the standard file which format is timestamp_gitlab_backup.tar
. So if someone upload some test file into this bucket, it may result the real backup archive file list on the top, then they will be deleted by cleanup function in backup-utility tool, cleanup function was used to delete backup archive files which exceed the maximum backup file counts.
(Summarize the bug encountered, concisely as possible)
Steps to reproduce
- create some non-standard test files like
2023_07_29_11_01-jh_gitlab_backup.tar
,test.txt
; - upload them to s3 bucket
- use backup-utility to clean up backends.
(Please provide the steps to reproduce the issue)
Configuration used
(Please provide a sanitized version of the configuration used wrapped in a code block (```yaml))
(Paste sanitized configuration here)
Current behavior
backup archive files was deleted.
Expected behavior
get_existing_backups function can identify the real backup archive file.
Versions
- Chart: (tagged version | branch | hash
git rev-parse HEAD
) - Platform:
- Cloud: (GKE | AKS | EKS | ?)
- Self-hosted: (OpenShift | Minikube | Rancher RKE | ?)
- Kubernetes: (
kubectl version
)- Client:
- Server:
- Helm: (
helm version
)- Client:
- Server:
Relevant logs
(Please provide any relevate log snippets you have collected, using code blocks (```) to format)