Skip to content

Security merge-train is failing

Today after we merged security MRs into master and saw the merge-train get activated automatically as intended, we started seeing errors like this:

>>> Cloning gitlab-org/gitlab
client_global_hostkeys_private_confirm: server gave bad signature for RSA key 0
Downloading spec/fixtures/git-cheat-sheet.pdf (7.4 MB)
Error downloading object: spec/fixtures/git-cheat-sheet.pdf (35967a2): Smudge error: Error downloading spec/fixtures/git-cheat-sheet.pdf (35967a21e5d856eaea89d2e5dd55a5e3b5f4e1e4efe3b000ef6d60b31600f1d2): batch response: Fatal error: Server error: https://gitlab.com/gitlab-org/gitlab.git/info/lfs/objects/batch
Errors logged to /builds/gitlab-org/merge-train/source/.git/lfs/logs/20210628T223647.79877164.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: spec/fixtures/git-cheat-sheet.pdf: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Example: https://ops.gitlab.net/gitlab-org/merge-train/-/jobs/4214227

We tried to add the log paths to artifacts for investigation, which doesn't seem to be working, and reduced the number of clone attempts to save time/resources.

Until these are resolved, Security master will fall behind Canonical master and thus impact our MTTP, as well as waste resources trying and failing to clone the gitlab-rails repository (~1.5 GB) 3-5 times every hour.

Edited by Robert Speicher