FIPS Gitlab Runner not installing on FIPS enable RHEL 8 system
<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
<!-- Summarize the bug encountered concisely. -->
I am trying to install the GitLab Runner rpm on a Red Hat Enterprise Linux 8 machine that is FIPS enabled, the install consistently fails with and error of `Error unpacking rpm package gitlab-runner-fips-14.10.0-1.x86_64` This happened with version 14.9.1 as well.
### Steps to reproduce
<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
1. Install and configure RHEL 8 with FIPS enabled
2. Follow instructions on https://docs.gitlab.com/runner/install/linux-repository.html for installing on RHEL 8.
1. Step 3 of these instructions fail with the error `Error unpacking rpm package gitlab-runner-fips-14.10.0-1.x86_64`
### What is the current *bug* behavior?
Yum/DNF fail to install the gitlab-runner-fips package
### What is the expected *correct* behavior?
The gitlab-runner-fips package is installed and running
### Relevant logs and/or screenshots
```
[root@wci-svc-dev log]# dnf -v -d 10 --rpmverbosity debug install gitlab-runner-fips
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, kpatch, needs-restarting, playground, product-id, repoclosure, repodiff, repograph, repomanage, reposync, subscription-manager, uploadprofile
Updating Subscription Management repositories.
DNF version: 4.7.0
cachedir: /var/cache/dnf
User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.5; generic; Linux.x86_64)'
repo: using cache for: docker-ce-stable
docker-ce-stable: using metadata from Wed 23 Mar 2022 07:34:59 PM PDT.
Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) 97 kB/s | 4.5 kB 00:00
reviving: 'rhel-8-for-x86_64-appstream-rpms' can be revived - repomd matches.
rhel-8-for-x86_64-appstream-rpms: using metadata from Thu 21 Apr 2022 02:44:33 AM PDT.
Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) 88 kB/s | 4.1 kB 00:00
reviving: 'rhel-8-for-x86_64-baseos-rpms' can be revived - repomd matches.
rhel-8-for-x86_64-baseos-rpms: using metadata from Tue 19 Apr 2022 01:07:52 PM PDT.
EPEL 8 RPMs 57 kB/s | 2.3 kB 00:00
repo: using cache for: runner_gitlab-runner
runner_gitlab-runner: using metadata from Tue 19 Apr 2022 12:23:48 PM PDT.
repo: using cache for: runner_gitlab-runner-source
runner_gitlab-runner-source: using metadata from Mon 06 Apr 2020 04:02:30 PM PDT.
--> Starting dependency resolution
---> Package gitlab-runner-fips.x86_64 14.10.0-1 will be installed
--> Finished dependency resolution
Dependencies resolved.
==========================================================================================================================================================================================
Package Architecture Version Repository Size
==========================================================================================================================================================================================
Installing:
gitlab-runner-fips x86_64 14.10.0-1 runner_gitlab-runner 92 M
Transaction Summary
==========================================================================================================================================================================================
Install 1 Package
Total download size: 92 M
Installed size: 127 M
Is this ok [y/N]: y
Downloading Packages:
gitlab-runner-fips-14.10.0-1.x86_64.rpm 107 MB/s | 92 MB 00:00
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 107 MB/s | 92 MB 00:00
Using rpmkeys executable at /bin/rpmkeys to verify signatures
Running transaction check
Transaction check succeeded.
Running transaction test
fdio: 2949 reads, 96409253 total bytes in 0.017987 secs
fdio: 6 reads, 7632 total bytes in 0.000010 secs
Transaction test succeeded.
Running transaction
fdio: 2949 reads, 96409253 total bytes in 0.017439 secs
Preparing : 1/1
Installing : gitlab-runner-fips-14.10.0-1.x86_64 [============================================= ] 1/1ufdio: 1529 writes, 50084935 total bytes in 0.026225 secs
fdio: 1539 reads, 50085231 total bytes in 1.569995 secs
Installing : gitlab-runner-fips-14.10.0-1.x86_64 1/1
D: closed db index /var/lib/rpm/Packages
D: closed db index /var/lib/rpm/Enhancename
D: closed db index /var/lib/rpm/Supplementname
D: closed db index /var/lib/rpm/Suggestname
D: closed db index /var/lib/rpm/Recommendname
D: closed db index /var/lib/rpm/Transfiletriggername
D: closed db index /var/lib/rpm/Filetriggername
D: closed db index /var/lib/rpm/Sha1header
D: closed db index /var/lib/rpm/Sigmd5
D: closed db index /var/lib/rpm/Installtid
D: closed db index /var/lib/rpm/Dirnames
D: closed db index /var/lib/rpm/Triggername
D: closed db index /var/lib/rpm/Obsoletename
D: closed db index /var/lib/rpm/Conflictname
D: closed db index /var/lib/rpm/Providename
D: closed db index /var/lib/rpm/Requirename
D: closed db index /var/lib/rpm/Group
D: closed db index /var/lib/rpm/Basenames
D: closed db index /var/lib/rpm/Name
D: closed db environment /var/lib/rpm
D: opening db environment /var/lib/rpm cdb:0x401
D: opening db index /var/lib/rpm/Packages (none) mode=0x42
D: sanity checking 1 elements
D: opening db index /var/lib/rpm/Name (none) mode=0x42
D: Plugin: calling hook tsm_pre in selinux plugin
D: Plugin: calling hook tsm_pre in systemd_inhibit plugin
D: System shutdown blocked (fd 53)
D: running pre-transaction scripts
D: computing 6 file fingerprints
D: opening db index /var/lib/rpm/Basenames (none) mode=0x42
D: opening db index /var/lib/rpm/Group (none) mode=0x42
D: opening db index /var/lib/rpm/Requirename (none) mode=0x42
D: opening db index /var/lib/rpm/Providename (none) mode=0x42
D: opening db index /var/lib/rpm/Conflictname (none) mode=0x42
D: opening db index /var/lib/rpm/Obsoletename (none) mode=0x42
D: opening db index /var/lib/rpm/Triggername (none) mode=0x42
D: opening db index /var/lib/rpm/Dirnames (none) mode=0x42
D: opening db index /var/lib/rpm/Installtid (none) mode=0x42
D: opening db index /var/lib/rpm/Sigmd5 (none) mode=0x42
D: opening db index /var/lib/rpm/Sha1header (none) mode=0x42
D: opening db index /var/lib/rpm/Filetriggername (none) mode=0x42
D: opening db index /var/lib/rpm/Transfiletriggername (none) mode=0x42
D: opening db index /var/lib/rpm/Recommendname (none) mode=0x42
D: opening db index /var/lib/rpm/Suggestname (none) mode=0x42
D: opening db index /var/lib/rpm/Supplementname (none) mode=0x42
D: opening db index /var/lib/rpm/Enhancename (none) mode=0x42
D: computing file dispositions
D: 0x0000fd00 4096 11149874 29451912 /
D: ========== +++ gitlab-runner-fips-14.10.0-1 x86_64-linux 0x0
D: gitlab-runner-fips-14.10.0-1.x86_64: Header V4 RSA/SHA512 Signature, key ID 35dfa027: OK
D: gitlab-runner-fips-14.10.0-1.x86_64: Header SHA256 digest: OK
D: gitlab-runner-fips-14.10.0-1.x86_64: Header SHA1 digest: OK
D: install: gitlab-runner-fips-14.10.0-1.x86_64 has 6 files
D: Plugin: calling hook psm_pre in selinux plugin
D: ========== Directories not explicitly included in package:
D: 0 /usr/bin/
D: 1 /usr/lib/gitlab-runner/helper-images/
D: 2 /usr/share/gitlab-runner/
D: ==========
D: create 120777 1 ( 0, 0) 13 /usr/bin/gitlab-ci-multi-runner;62680dbe
D: Plugin: calling hook fsm_file_prepare in selinux plugin
D: create 100755 1 ( 0, 0)50084935 /usr/bin/gitlab-runner;62680dbe
Error unpacking rpm package gitlab-runner-fips-14.10.0-1.x86_64
fdio: 6 reads, 7632 total bytes in 0.000010 secs
Errors occurred during transaction.
Verifying : gitlab-runner-fips-14.10.0-1.x86_64 1/1
Completion plugin: Generating completion cache...
Installed products updated.
User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.5; generic; Linux.x86_64)'
User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.5; generic; Linux.x86_64)'
repo: using cache for: docker-ce-stable
docker-ce-stable: using metadata from Wed 23 Mar 2022 07:34:59 PM PDT.
reviving: 'rhel-8-for-x86_64-appstream-rpms' can be revived - repomd matches.
rhel-8-for-x86_64-appstream-rpms: using metadata from Thu 21 Apr 2022 02:44:33 AM PDT.
reviving: 'rhel-8-for-x86_64-baseos-rpms' can be revived - repomd matches.
rhel-8-for-x86_64-baseos-rpms: using metadata from Tue 19 Apr 2022 01:07:52 PM PDT.
repo: using cache for: runner_gitlab-runner
runner_gitlab-runner: using metadata from Tue 19 Apr 2022 12:23:48 PM PDT.
repo: using cache for: runner_gitlab-runner-source
runner_gitlab-runner-source: using metadata from Mon 06 Apr 2020 04:02:30 PM PDT.
Failed: gitlab-runner-fips-14.10.0-1.x86_64
Failed:
gitlab-runner-fips-14.10.0-1.x86_64
Error: Transaction failed
```
### Output of checks
<!-- If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com -->
This is happening on locally hosted machines that will be connected to an On Prem Omnibus install.
#### Results of GitLab environment info
<!-- Input any relevant GitLab environment information if needed. -->
```
System information
System: RedHatEnterpriseWorkstation 7.9
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.7.5p203
Gem Version: 3.1.4
Bundler Version:2.2.33
Rake Version: 13.0.6
Redis Version: 6.2.6
Sidekiq Version:6.4.0
Go Version: unknown
GitLab information
Version: 14.10.0-ee
Revision: ad109bc62af
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 12.7
URL: https://wci-git.llnl.gov
HTTP Clone URL: https://wci-git.llnl.gov/some-group/some-project.git
SSH Clone URL: git@wci-git.llnl.gov:some-group/some-project.git
Elasticsearch: no
Geo: no
Using LDAP: yes
Using Omniauth: yes
Omniauth Providers: openid_connect, esn
GitLab Shell
Version: 13.25.1
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
```
#### Results of GitLab application Check
```
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.25.1 ? ... OK (13.25.1)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
3/5 ... yes
4/7 ... yes
8/11 ... yes
8/13 ... yes
6/14 ... yes
3/17 ... yes
2/20 ... yes
8/24 ... yes
3/25 ... yes
2/33 ... yes
8/34 ... yes
8/35 ... yes
8/36 ... yes
2/46 ... yes
8/47 ... yes
2/48 ... yes
2/60 ... yes
2/65 ... yes
3/67 ... yes
3/81 ... yes
2/84 ... yes
69/85 ... yes
2/88 ... yes
6/89 ... yes
3/91 ... yes
18/93 ... yes
18/94 ... yes
18/95 ... yes
18/96 ... yes
21/97 ... yes
21/98 ... yes
21/99 ... yes
18/100 ... yes
21/101 ... yes
21/102 ... yes
17/103 ... yes
17/104 ... yes
21/105 ... yes
18/106 ... yes
18/107 ... yes
21/108 ... yes
21/109 ... yes
21/110 ... yes
21/111 ... yes
21/112 ... yes
21/113 ... yes
21/114 ... yes
21/115 ... yes
21/116 ... yes
21/117 ... yes
18/118 ... yes
3/119 ... yes
2/120 ... yes
460/121 ... yes
2/122 ... yes
23/123 ... yes
8/124 ... yes
30/125 ... yes
25/128 ... yes
6/129 ... yes
25/130 ... yes
30/131 ... yes
30/132 ... yes
30/133 ... yes
37/134 ... yes
40/135 ... yes
40/136 ... yes
159/137 ... yes
58/138 ... yes
40/139 ... yes
2/141 ... yes
295/142 ... yes
30/144 ... yes
30/145 ... yes
30/146 ... yes
64/148 ... yes
2/150 ... yes
30/151 ... yes
72/152 ... yes
71/154 ... yes
8/155 ... yes
77/157 ... yes
30/158 ... yes
117/159 ... yes
127/160 ... yes
2/161 ... yes
40/165 ... yes
195/168 ... yes
195/169 ... yes
146/171 ... yes
195/174 ... yes
195/175 ... yes
17/176 ... yes
3/177 ... yes
159/178 ... yes
30/179 ... yes
8/180 ... yes
159/181 ... yes
256/182 ... yes
51/183 ... yes
69/184 ... yes
184/185 ... yes
21/186 ... yes
21/187 ... yes
21/188 ... yes
30/189 ... yes
85/190 ... yes
195/191 ... yes
195/192 ... yes
194/193 ... yes
194/195 ... yes
195/196 ... yes
194/197 ... yes
146/199 ... yes
195/200 ... yes
194/201 ... yes
194/202 ... yes
195/203 ... yes
195/204 ... yes
195/205 ... yes
195/206 ... yes
194/207 ... yes
195/208 ... yes
195/209 ... yes
195/210 ... yes
194/211 ... yes
194/212 ... yes
195/213 ... yes
3/214 ... yes
195/215 ... yes
194/216 ... yes
195/217 ... yes
195/218 ... yes
195/219 ... yes
3/220 ... yes
195/221 ... yes
69/222 ... yes
195/223 ... yes
195/224 ... yes
194/225 ... yes
194/226 ... yes
195/227 ... yes
194/228 ... yes
195/229 ... yes
146/230 ... yes
195/231 ... yes
196/232 ... yes
146/233 ... yes
195/234 ... yes
146/235 ... yes
195/236 ... yes
196/237 ... yes
195/238 ... yes
3/239 ... yes
195/240 ... yes
194/241 ... yes
194/242 ... yes
111/244 ... yes
32/245 ... yes
32/246 ... yes
70/247 ... yes
2/249 ... yes
3/250 ... yes
266/251 ... yes
23/252 ... yes
25/253 ... yes
2/254 ... yes
206/256 ... yes
206/257 ... yes
25/258 ... yes
32/259 ... yes
42/260 ... yes
206/261 ... yes
212/262 ... yes
72/264 ... yes
295/266 ... yes
223/267 ... yes
221/268 ... yes
206/270 ... yes
206/271 ... yes
235/272 ... yes
229/273 ... yes
223/274 ... yes
21/275 ... yes
79/276 ... yes
223/277 ... yes
8/278 ... yes
58/279 ... yes
30/280 ... yes
266/282 ... yes
266/283 ... yes
271/284 ... yes
280/285 ... yes
25/286 ... yes
25/287 ... yes
200/289 ... yes
280/290 ... yes
25/291 ... yes
295/292 ... yes
295/293 ... yes
3/294 ... yes
295/296 ... yes
27/297 ... yes
296/298 ... yes
200/299 ... yes
8/300 ... yes
8/301 ... yes
298/302 ... yes
294/304 ... yes
309/305 ... yes
295/306 ... yes
207/307 ... yes
27/308 ... yes
306/309 ... yes
266/310 ... yes
184/311 ... yes
291/312 ... yes
308/313 ... yes
266/314 ... yes
8/315 ... yes
8/316 ... yes
8/318 ... yes
200/319 ... yes
266/320 ... yes
317/323 ... yes
30/324 ... yes
266/325 ... yes
291/326 ... yes
8/327 ... yes
6/328 ... yes
2/330 ... yes
6/331 ... yes
47/332 ... yes
334/335 ... yes
280/336 ... yes
25/337 ... yes
336/338 ... yes
336/339 ... yes
25/340 ... yes
8/342 ... yes
36/343 ... yes
25/344 ... yes
295/345 ... yes
23/346 ... yes
198/349 ... yes
200/350 ... yes
350/352 ... yes
26/353 ... yes
37/354 ... yes
23/357 ... yes
212/358 ... yes
71/359 ... yes
365/361 ... yes
365/362 ... yes
30/363 ... yes
367/364 ... yes
117/365 ... yes
71/366 ... yes
373/367 ... yes
373/368 ... yes
373/369 ... yes
309/371 ... yes
30/372 ... yes
373/373 ... yes
212/375 ... yes
365/376 ... yes
379/377 ... yes
2/378 ... yes
30/379 ... yes
2/380 ... yes
460/381 ... yes
388/383 ... yes
378/384 ... yes
373/387 ... yes
295/388 ... yes
379/389 ... yes
30/390 ... yes
295/391 ... yes
373/392 ... yes
428/393 ... yes
428/394 ... yes
2/395 ... yes
71/396 ... yes
212/397 ... yes
30/398 ... yes
431/399 ... yes
6/400 ... yes
373/411 ... yes
373/412 ... yes
23/414 ... yes
3/415 ... yes
6/416 ... yes
69/417 ... yes
443/418 ... yes
36/419 ... yes
6/420 ... yes
451/421 ... yes
212/422 ... yes
212/423 ... yes
30/424 ... yes
212/425 ... yes
378/426 ... yes
473/427 ... yes
2/428 ... yes
451/429 ... yes
72/430 ... yes
378/431 ... yes
451/432 ... yes
451/433 ... yes
231/434 ... yes
111/435 ... yes
499/436 ... yes
338/437 ... yes
25/438 ... yes
25/439 ... yes
503/440 ... yes
509/442 ... yes
499/443 ... yes
547/444 ... yes
306/445 ... yes
373/446 ... yes
30/447 ... yes
499/448 ... yes
499/449 ... yes
518/450 ... yes
379/451 ... yes
24/452 ... yes
212/453 ... yes
212/454 ... yes
367/455 ... yes
107/456 ... yes
212/457 ... yes
367/458 ... yes
367/459 ... yes
30/460 ... yes
295/461 ... yes
310/462 ... yes
460/464 ... yes
30/465 ... yes
212/466 ... yes
545/467 ... yes
547/468 ... yes
547/469 ... yes
547/470 ... yes
550/474 ... yes
36/475 ... yes
367/476 ... yes
473/477 ... yes
17/479 ... yes
566/480 ... yes
545/481 ... yes
545/482 ... yes
545/483 ... yes
545/484 ... yes
545/485 ... yes
545/486 ... yes
545/487 ... yes
545/488 ... yes
212/489 ... yes
23/490 ... yes
280/492 ... yes
549/493 ... yes
295/494 ... yes
280/495 ... yes
146/496 ... yes
238/499 ... yes
40/500 ... yes
550/501 ... yes
200/502 ... yes
30/503 ... yes
373/504 ... yes
30/505 ... yes
30/506 ... yes
72/507 ... yes
373/509 ... yes
540/510 ... yes
587/511 ... yes
236/512 ... yes
236/513 ... yes
25/515 ... yes
214/516 ... yes
459/517 ... yes
598/520 ... yes
602/521 ... yes
602/522 ... yes
602/523 ... yes
602/524 ... yes
602/525 ... yes
602/526 ... yes
602/527 ... yes
602/528 ... yes
601/530 ... yes
612/531 ... yes
614/532 ... yes
618/534 ... yes
614/535 ... yes
614/536 ... yes
623/537 ... yes
623/538 ... yes
627/539 ... yes
623/541 ... yes
614/542 ... yes
623/543 ... yes
614/544 ... yes
614/545 ... yes
23/546 ... yes
6/548 ... yes
645/549 ... yes
545/550 ... yes
652/551 ... yes
30/552 ... yes
25/553 ... yes
212/554 ... yes
25/555 ... yes
541/556 ... yes
640/557 ... yes
541/558 ... yes
460/559 ... yes
30/560 ... yes
540/561 ... yes
541/562 ... yes
539/563 ... yes
212/564 ... yes
290/565 ... yes
159/566 ... yes
306/567 ... yes
306/568 ... yes
539/569 ... yes
539/570 ... yes
460/571 ... yes
541/573 ... yes
541/574 ... yes
212/575 ... yes
541/576 ... yes
541/577 ... yes
30/578 ... yes
707/579 ... yes
541/580 ... yes
541/581 ... yes
541/582 ... yes
71/583 ... yes
541/584 ... yes
541/585 ... yes
707/586 ... yes
460/587 ... yes
541/588 ... yes
1164/589 ... yes
1166/590 ... yes
602/591 ... yes
539/592 ... yes
545/593 ... yes
460/594 ... yes
545/595 ... yes
707/599 ... yes
707/600 ... yes
379/601 ... yes
626/602 ... yes
379/603 ... yes
707/604 ... yes
698/605 ... yes
707/606 ... yes
707/607 ... yes
30/608 ... yes
460/609 ... yes
707/610 ... yes
25/611 ... yes
1203/612 ... yes
541/613 ... yes
254/614 ... yes
254/615 ... yes
Redis version >= 5.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.5)
Git user has default SSH configuration? ... yes
Active users: ... 178
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
```
### Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->
I am able to fully install the gitlab-runner-fips by disabling FIPS, restarting, running the install, enabling FIPS, and restarting the machine.
issue