Restore error: "gtar: .: Cannot unlink: Invalid argument"

Summary

Backup restore fails with gtar: .: Cannot unlink: Invalid argument on systems running RHEL/EL-patched GNU tar (tar-1.34-9.el9_7 or later).

Reported on the forums: https://forum.gitlab.com/t/cant-restore-18-7-1/132444

Root Cause

RHSA-2026:0067 patches GNU tar for CVE-2025-45582 (a two-archive symlink path traversal). The CVE fix itself uses openat2(RESOLVE_BENEATH) to jailify the extraction directory and does not restrict --unlink-first. However, the same advisory bundles a separate fix (RHEL-136277) that changes how tar handles unlink("."). On XFS (and possibly other filesystems), unlink(".") now returns EINVAL, and the patched tar propagates this as a fatal error instead of ignoring it.

GitLab backup archives contain ./ as the root entry. When --unlink-first is passed, tar attempts unlink(".") on that entry, triggering the fatal error on patched systems.

CVE: https://nvd.nist.gov/vuln/detail/CVE-2025-45582
Red Hat advisory: https://access.redhat.com/errata/RHSA-2026:0067
GNU tar maintainer response: https://lists.gnu.org/archive/html/bug-tar/2025-08/msg00012.html

Affected file: gems/gitlab-backup-cli/lib/gitlab/backup/cli/utils/tar.rb (lines 63-72)

Affected Versions

GitLab 18.5.5+
Confirmed on EL9 and EL10 (Rocky Linux, RHEL, AlmaLinux) with tar-1.34-9.el9_7 or later
Not affected: Ubuntu, Debian, or other distros whose tar packages do not include the RHEL-136277 fix

Workaround

Downgrade tar package to version 1.34-7.el9 or earlier.

Error Log

2026-01-23 09:30:10 UTC -- Deleting backup and restore PID file at [/opt/gitlab/embedded/service/gitlab-rails/tmp/backup_restore.pid] ... done
rake aborted!
Backup::Error: Restore operation failed: gtar: .: Cannot unlink: Invalid argument
gtar: Exiting with failure status due to previous errors
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/targets/files.rb:104:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/tasks/task.rb:31:in `restore!'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/restore/process.rb:30:in `execute!'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:101:in `run_restore_task'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:168:in `block in run_all_restore_tasks'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:165:in `each_value'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:165:in `run_all_restore_tasks'
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:68:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rb:22:in `block in restore_backup'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rb:83:in `lock_backup'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rb:19:in `restore_backup'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:14:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => gitlab:backup:restore
(See full trace by running task with --trace

Proposed solution

The backup_existing_files_dir method in lib/backup/targets/files.rb:107-126 already moves all existing files to a timestamped backup directory before tar extraction runs. The --unlink-first and --recursive-unlink flags are redundant and should be removed. The flags were added in gitlab-foss!17516 (merged).

Note — Backup CLI gem behavior change: The shared Tar#extract_cmd method is also used by the backup CLI gem's restore path (gems/gitlab-backup-cli/lib/gitlab/backup/cli/targets/files.rb). Unlike the Rails restore path, the gem's restore does not pre-move existing files (tracked in #499876 — it currently logs a warning and continues). Removing the flags means the gem path will no longer force-unlink existing files before extraction. This is acceptable because:

The gem path already lacks proper pre-move logic, so the flags only provided partial coverage.

Tar's default behavior (overwrite existing files) is sufficient for the gem's current "warn and continue" approach.

When #499876 is resolved, the gem will have proper pre-move logic matching the Rails path.

Implementation Plan

Edit gems/gitlab-backup-cli/lib/gitlab/backup/cli/utils/tar.rb
- Remove --unlink-first and --recursive-unlink from extract_cmd (lines 64-66)
Update spec spec/lib/backup/targets/files_spec.rb
- Remove assertions expecting the deleted flags (lines 76-87)

Run tests

bundle exec rspec spec/lib/backup/targets/files_spec.rb
bundle exec rspec gems/gitlab-backup-cli/spec/

Plan to Validate the Proposed Solution

Click to expand plan generated with Duo (corrected after execution)

Here is a compact end-to-end runbook you can execute locally.

Important notes from execution:

The GitLab container is Ubuntu 24.04 (not RHEL). Ubuntu's tar 1.35 is unpatched and does not exhibit the bug. Step 6 uses a wrapper script to simulate the RHEL-patched tar behavior instead of apt-get install.

Files created by root in the uploads directory will cause Errno::EACCES during restore because backup_existing_files_dir runs as the git user. Step 3 sets ownership to git:git.

End-to-end Docker validation flow

Start GitLab container

docker run --detach \
  --hostname gitlab.local \
  --publish 8929:8929 --publish 2222:22 \
  --name gitlab-restore-test \
  --restart unless-stopped \
  --volume gitlab_config:/etc/gitlab \
  --volume gitlab_logs:/var/log/gitlab \
  --volume gitlab_data:/var/opt/gitlab \
  gitlab/gitlab-ee:18.8.3-ee.0

Wait for readiness

docker exec -it gitlab-restore-test gitlab-ctl status
docker exec -it gitlab-restore-test gitlab-rake gitlab:check SANITIZE=true

Create sample data (minimum)

docker exec -it gitlab-restore-test bash -lc '
set -e
mkdir -p /var/opt/gitlab/gitlab-rails/uploads/validation-seed
echo "seed" > /var/opt/gitlab/gitlab-rails/uploads/validation-seed/file.txt
chown -R git:git /var/opt/gitlab/gitlab-rails/uploads/validation-seed
'

Why chown? The restore process calls backup_existing_files_dir which runs FileUtils.mv as the git user. If files in the uploads directory are owned by root, the move fails with Errno::EACCES before tar extraction even runs. This is unrelated to the tar flag issue being tested.

Create backup

docker exec -it gitlab-restore-test gitlab-backup create STRATEGY=copy
docker exec -it gitlab-restore-test bash -lc 'ls -1 /var/opt/gitlab/backups/*_gitlab_backup.tar'

Run baseline restore script (should pass before tar change)

docker exec -it gitlab-restore-test bash -lc '
cat >/root/validate_restore_min.sh << "EOF"
#!/usr/bin/env bash
set -euo pipefail
BID="$(ls -1 /var/opt/gitlab/backups/*_gitlab_backup.tar | sed "s#.*/##" | sed "s/_gitlab_backup.tar$//" | sort | tail -1)"
UP="/var/opt/gitlab/gitlab-rails/uploads"

mkdir -p "$UP/preexisting"
echo dummy > "$UP/preexisting/should_move.txt"
chown -R git:git "$UP/preexisting"

echo "=== Restore 1 ==="
gitlab-backup restore BACKUP="$BID" force=yes 2>&1 | tee /root/restore1.log

if [ -f "$UP/preexisting/should_move.txt" ]; then
  echo "FAIL: preexisting file was NOT moved"
  exit 1
fi

echo "=== Restore 2 (idempotency) ==="
gitlab-backup restore BACKUP="$BID" force=yes 2>&1 | tee /root/restore2.log

if grep -qE "Cannot unlink|Backup::Error|Exiting with failure status" /root/restore1.log /root/restore2.log; then
  echo "FAIL: Found error patterns in restore logs"
  exit 1
fi
echo "PASS"
EOF
chmod +x /root/validate_restore_min.sh
/root/validate_restore_min.sh
'

Simulate RHEL-patched tar behavior (the breaking condition)

The GitLab container runs Ubuntu 24.04 with unpatched GNU tar 1.35. The breakage is caused by a RHEL-specific patch (RHEL-136277) bundled in RHSA-2026:0067 that makes unlink(".") fatal. Since we cannot install the RHEL-patched tar on Ubuntu, we use a wrapper script that simulates the patched behavior: extract operations that combine --extract with --unlink-first fail with the same error message.

docker exec -it gitlab-restore-test bash -lc '
set -e

# Back up real tar
cp /usr/bin/tar /usr/bin/tar.real

# Create wrapper that simulates RHEL-patched tar behavior
cat > /usr/bin/tar << "WRAPPER"
#!/usr/bin/env bash
REAL_TAR=/usr/bin/tar.real

HAS_EXTRACT=false
HAS_UNLINK_FIRST=false

for arg in "$@"; do
  case "$arg" in
    --extract|-x) HAS_EXTRACT=true ;;
    --unlink-first|-U) HAS_UNLINK_FIRST=true ;;
  esac
done

# Simulate RHEL-patched behavior: fail when extract + unlink-first
if $HAS_EXTRACT && $HAS_UNLINK_FIRST; then
  echo "tar: .: Cannot unlink: Invalid argument" >&2
  echo "tar: Exiting with failure status due to previous errors" >&2
  exit 2
fi

exec $REAL_TAR "$@"
WRAPPER
chmod +x /usr/bin/tar

tar --version | head -1
echo "RHEL-patched tar simulator installed"
'

Show breakage on current code

docker exec -it gitlab-restore-test bash -lc '/root/validate_restore_min.sh || true'
docker exec -it gitlab-restore-test bash -lc 'grep -E "Cannot unlink|Backup::Error" -n /root/restore1.log /root/restore2.log || true'

Hotpatch tar.rb (remove the two flags)

docker exec -it gitlab-restore-test bash -lc '
set -e
FILE=/opt/gitlab/embedded/service/gitlab-rails/gems/gitlab-backup-cli/lib/gitlab/backup/cli/utils/tar.rb
cp "$FILE" "${FILE}.bak"
sed -i "/--unlink-first/d; /--recursive-unlink/d" "$FILE"
grep -n "unlink-first\|recursive-unlink" "$FILE" || echo "Flags removed successfully"
'

Re-run minimum validation after patch (should pass)

docker exec -it gitlab-restore-test bash -lc '/root/validate_restore_min.sh'

Nested-path smoke test

docker exec -it gitlab-restore-test bash -lc '
set -e
echo "gitlab_rails[\"uploads_directory\"] = \"/var/opt/gitlab/data/shared/gitlab/uploads\"" >> /etc/gitlab/gitlab.rb
gitlab-ctl reconfigure
mkdir -p /var/opt/gitlab/data/shared/gitlab/uploads/nested-seed
echo nested > /var/opt/gitlab/data/shared/gitlab/uploads/nested-seed/file.txt
chown -R git:git /var/opt/gitlab/data/shared/gitlab/uploads
gitlab-backup create STRATEGY=copy
/root/validate_restore_min.sh
'

Permission failure test (fail-fast behavior)

docker exec -it gitlab-restore-test bash -lc '
set -euo pipefail
BID="$(ls -1 /var/opt/gitlab/backups/*_gitlab_backup.tar | sed "s#.*/##" | sed "s/_gitlab_backup.tar$//" | sort | tail -1)"
UP="/var/opt/gitlab/gitlab-rails/uploads"

mkdir -p "$UP/perm-test"
echo blocked > "$UP/perm-test/file.txt"

# Make directory effectively non-movable for restore user path handling
chmod 000 "$UP/perm-test"

set +e
gitlab-backup restore BACKUP="$BID" force=yes | tee /root/restore-perm.log
RC=$?
set -e

# Cleanup perms so container is usable
chmod 755 "$UP/perm-test" || true

if [ "$RC" -eq 0 ]; then
  echo "FAIL: expected restore failure due to permission block"
  exit 1
fi

grep -Ei "permission denied|cannot|failed|error|Backup::Error" /root/restore-perm.log >/dev/null
echo "PASS: restore failed fast with actionable error"
'

Data integrity spot-check (sample checksums)

docker exec -it gitlab-restore-test bash -lc '
set -euo pipefail
UP="/var/opt/gitlab/gitlab-rails/uploads"
BID="$(ls -1 /var/opt/gitlab/backups/*_gitlab_backup.tar | sed "s#.*/##" | sed "s/_gitlab_backup.tar$//" | sort | tail -1)"

mkdir -p "$UP/integrity"
chown git:git "$UP/integrity"
for i in 1 2 3 4 5; do
  echo "sample-$i-$(date +%s)" > "$UP/integrity/file$i.txt"
  chown git:git "$UP/integrity/file$i.txt"
done

find "$UP/integrity" -type f | sort | xargs sha256sum > /root/integrity-before.sha256

gitlab-backup create STRATEGY=copy >/root/integrity-backup.log
BID2="$(ls -1 /var/opt/gitlab/backups/*_gitlab_backup.tar | sed "s#.*/##" | sed "s/_gitlab_backup.tar$//" | sort | tail -1)"

# Mutate files so restore must actually recover old content
for i in 1 2 3 4 5; do echo "mutated-$i" > "$UP/integrity/file$i.txt"; done
chown -R git:git "$UP/integrity"

gitlab-backup restore BACKUP="$BID2" force=yes | tee /root/integrity-restore.log

find "$UP/integrity" -type f | sort | xargs sha256sum > /root/integrity-after.sha256
diff -u /root/integrity-before.sha256 /root/integrity-after.sha256
echo "PASS: integrity sample matches after restore"
'

Collect evidence

docker exec -it gitlab-restore-test bash -lc '
tar --version
ls -l /root/restore*.log
'

This gives you: breakage reproduced, hotpatch verified, and minimum regression confidence (preexisting-data handling + repeat restore; plus optional nested-path check).

Edited Jul 27, 2026 by 🤖 GitLab Bot 🤖