Problem upgrading Gitlab 19.0.x to 19.1.0 due to changes with /opt/gitlab/embedded/service/gitlab-rails/.agents/skills

Summary

Whilst upgrading Gitlab to 19.1.0 from 19.0.1 an error is encountered:

Error: Transaction test error:
  file /opt/gitlab/embedded/service/gitlab-rails/.agents/skills from install of gitlab-ee-19.1.0-ee.0.el9.x86_64 conflicts with file from package gitlab-ee-19.0.2-ee.0.el9.x86_64

As above, it affects distributions based on Red Hat Enterprise Linux. On further analysis, we can see prior to the upgrade that the package content is:

/opt/gitlab/embedded/service/gitlab-rails/.agents/skills
/opt/gitlab/embedded/service/gitlab-rails/.agents/skills/gitlab-coding-principles
/opt/gitlab/embedded/service/gitlab-rails/.agents/skills/gitlab-coding-principles/SKILL.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/gitlab-coding-principles
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/gitlab-coding-principles/SKILL.md

After upgrade, we can see the package content is:

/opt/gitlab/embedded/service/gitlab-rails/.agents/skills
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/README.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/dependency-management-auto-remediation
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/dependency-management-auto-remediation/SKILL.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/gitlab-coding-principles
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/gitlab-coding-principles/SKILL.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/SKILL.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/assets
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/assets/graphql
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/assets/graphql/create-note.graphql
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/assets/graphql/epic-notes.graphql
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/epic-comments.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/epics.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/issue-api.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/issue-links.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/mr-review.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/multi-host.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/nested-groups.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/search.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/references/work-items.md
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/scripts
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/scripts/create-epic-note.sh
/opt/gitlab/embedded/service/gitlab-rails/.claude/skills/glab/scripts/epic-notes.sh

Further checking the /opt/gitlab/embedded/service/gitlab-rails/.agents directory - prior to upgrade .agents/skills is a directory with a directory called gitlab-coding-principles and a file SKILLS.md. After upgrade, we now see that the skills directory is a symlink to .claude/skills. Whilst the package links above do not show this, the status on disk does.

root@repo:/opt/gitlab/embedded/service/gitlab-rails/.agents# ls -lha
total 12K
drwxr-xr-x.  3 root root   38 Jun 19 09:30 .
drwxr-xr-x. 41 root root 8.0K Jun 19 09:30 ..
lrwxrwxrwx.  1 root root   17 Jun 17 12:14 skills -> ../.claude/skills
drwxr-xr-x.  3 root root   38 Jun 10 07:16 skills.old

It seems whoever packaged it, didn't take into account this change, which causes the upgrade to fail. A workaround is to rename skills to skills.old which is what I did (hence why it's shown in the results above), but it's also possible to delete it entirely so that the upgrade creates the symlink and doesn't end up with a conflict.

This issue was initially reported by other users on the Gitlab forum:

  1. https://forum.gitlab.com/t/gitlab-ce-19-0-0-19-1-0-upgrade-fails-with-agents-skills-file-conflict/134277/2
  2. https://forum.gitlab.com/t/error-in-upgrade-to-gitlab-19-1-0/134279
  3. https://forum.gitlab.com/t/upgrade-to-19-1-0-fails-with-file-conflict-on-opt-gitlab-embedded-service-gitlab-rails-agents-skills/134278

Steps to reproduce

  1. Install GitLab 19.0.0 on RHEL 8 (either CE or EE)
  2. Upgrade with yum/dnf to 19.1.0
  3. See the conflict error

Example Project

What is the current bug behavior?

Package upgrades (RPM) fail with file/directory conflicts and cannot continue.

What is the expected correct behavior?

File/directory changes do not impact upgrades and stop.

Relevant logs and/or screenshots

Output of checks

Possible fixes

Remove .agents and .claude from the distributed packages.

Workarounds

From https://forum.gitlab.com/t/gitlab-ce-19-0-0-19-1-0-upgrade-fails-with-agents-skills-file-conflict/134277

As a temporary workaround, renaming the directory allowed the upgrade transaction to succeed.

mv /opt/gitlab/embedded/service/gitlab-rails/.agents/skills > /opt/gitlab/embedded/service/gitlab-rails/.agents/skills.bak

From https://forum.gitlab.com/t/upgrade-to-19-1-0-fails-with-file-conflict-on-opt-gitlab-embedded-service-gitlab-rails-agents-skills/134278

Removing the conflicting path before upgrade resolves it:

rm -rf /opt/gitlab/embedded/service/gitlab-rails/.agents/skills
sudo dnf update --refresh ./gitlab-ee-19.1.0-ee.0.amazon2023.x86_64.rpm

Patch release information for backports

If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.

Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.

High-severity bug remediation

To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.

Edited by Michael Friedrich