Skip to content

Not all artifacts are migrated to object storage using rake task.

Summary

Report artifacts are not migrated to object storage.

Steps to reproduce

  1. Run a Gitlab instance with artifacts stored the local filesystem (i.e. without object storage).
  2. Create a CI job which stores a junit report as artifact and run it.
  3. Configure object storage in the GitLab settings.
  4. Migrate the artifacts using gitlab-rake gitlab:artifacts:migrate.

Example Project

Sorry :(

What is the current bug behavior?

Some artifacts aren't migrated to object storage. You can see it in PostgreSQL in the ci_job_artifacts table. Records with junit.xml.gz in the file column still have a 1 in the file_store column. Also the files are still located onto disk and not in the object storage.

SELECT * FROM ci_job_artifacts WHERE file = 'junit.xml.gz';

What is the expected correct behavior?

All artifacts are migrated to object storage. All records in the ci_job_artifacts table have a 2 in the file_store column.

The following SQL-query shouldn't return any results:

SELECT * FROM ci_job_artifacts WHERE "file_store" = '1';

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info

Using the official task-runner image: registry.gitlab.com/gitlab-org/build/cng/gitlab-task-runner-ee:v11.6.9

System information
System:
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.4.4p296
Gem Version:	2.7.6
Bundler Version:1.16.6
Rake Version:	12.3.2
Redis Version:	unknown
Git Version:	unknown
Sidekiq Version:5.2.3
Go Version:	go1.9.6 linux/amd64

GitLab information
Version:	11.6.9-ee
Revision:	d6fc7c74
Directory:	/srv/gitlab
DB Adapter:	postgresql
DB Version:	9.6.10
URL:		
HTTP Clone URL:	https:///some-group/some-project.git
SSH Clone URL:	git@:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	no

GitLab Shell
Version:	unknown
Repository storage paths:
- default: 	/var/opt/gitlab/repo
Hooks:		/home/git/gitlab-shell/hooks
Git:		/usr/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

Using the official task-runner image: registry.gitlab.com/gitlab-org/build/cng/gitlab-task-runner-ee:v11.6.9


WARNING: This version of GitLab depends on gitlab-shell 8.4.3, but you're running Unknown. Please update gitlab-shell.
Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 8.4.3 ? ... FAIL. Please update gitlab-shell to 8.4.3 from Unknown
Running /home/git/gitlab-shell/bin/check
gitlab-shell self-check failed
  Try fixing it:
  Make sure GitLab is running;
  Check the gitlab-shell configuration file:
  sudo -u git -H editor /home/git/gitlab-shell/config.yml
  Please fix the error above and rerun the checks.

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... no
  Try fixing it:
  sudo -u git -H RAILS_ENV=production bin/background_jobs start
  For more information see:
  doc/install/installation.md in section "Install Init Script"
  see log/sidekiq.log for possible errors
  Please fix the error above and rerun the checks.

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... no
Trying to fix error automatically. ...Failed
  Try fixing it:
  sudo -u git -H "/usr/bin/git" config --global core.autocrlf "input"
  For more information see:
  doc/install/installation.md in section "GitLab"
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... no
  Try fixing it:
  sudo chmod 700 /srv/gitlab/public/uploads
  For more information see:
  doc/install/installation.md in section "GitLab"
  Please fix the error above and rerun the checks.
Uploads directory tmp has correct permissions? ... no
  Try fixing it:
  sudo chown -R git /srv/gitlab/public/uploads
  sudo find /srv/gitlab/public/uploads -type f -exec chmod 0644 {} \;
  sudo find /srv/gitlab/public/uploads -type d -not -path /srv/gitlab/public/uploads -exec chmod 0700 {} \;
  For more information see:
  doc/install/installation.md in section "GitLab"
  Please fix the error above and rerun the checks.
Init script exists? ... no
  Try fixing it:
  Install the init script
  For more information see:
  doc/install/installation.md in section "Install Init Script"
  Please fix the error above and rerun the checks.
Init script up-to-date? ... can't check because of previous errors
Projects have namespace: ...

[all yes]

Redis version >= 2.8.0? ... no
  Try fixing it:
  Update your redis server to a version >= 2.8.0
  For more information see:
  gitlab-public-wiki/wiki/Trouble-Shooting-Guide in section sidekiq
  Please fix the error above and rerun the checks.
Ruby version >= 2.3.5 ? ... yes (2.4.4)
Git version >= 2.18.0 ? ... no
Your git bin path is "/usr/bin/git"
  Try fixing it:
  Update your git to a version >= 2.18.0 from Unknown
  Please fix the error above and rerun the checks.
Git user has default SSH configuration? ... yes
Active users: ... 11
Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled)

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

I did a manual migration using the Minio mc client binary and updated the records in the database.