FIPS - Maven Package Repo - 422 Unprocessable Entity Error
Summary
When pushing a maven package to the repo using mvn deploy
, a 422 Unprocessable Entity Error is received and the package is not added to the package registry.
It appears that the following conditions must be present for the issue to occur:
- The GitLab server is running on Kubernetes installed using GitLab Helm Chart
- The non-FIPS UBI containers are being used
- The Kubernetes nodes for the server cluster are in FIPS mode
The error did not occur with the Debian k8s containers or omnibus running on a FIPS VM.
Steps to reproduce
Full details of environment setup and test:
- Create an EC2 instance. AMI: ubuntu-pro-fips-server/images/hvm-ssd/ubuntu-focal-20.04-amd64-pro-fips-server-20220615-7bc828d1-c072-4d33-a989-fbad50380cfb ami-04ebe04990712e030.
- Instance Size: m5.xlarge 4vcpu, 16 Gib Mem Security Group. Open 22, 80, 443, 6443
- Update and configure EC2 instance
sudo apt update / upgrade
sudo hostnamectl set-hostname example.domain.com
sudo reboot now
- Set up DNS entries
- new DNS record (example.domain.com) pointing at ec2 instance
- set wildcard record (*.example.domain.com) pointing at ec2 instance
- Install k3s
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik" sh -s -
- Get kube config and move to the machine where you have kubectl and helm 3 installed
- Update host names and/or file permissions for kube config as needed.
- Install GitLab Using Helm Chart - Use UBI8 containers for webservice and workhorse
kubectl create namespace gitlab
helm repo add gitlab https://charts.gitlab.io/
helm repo update
helm upgrade --install gitlab gitlab/gitlab -n gitlab --set certmanager-issuer.email=email@address.com -f values.yaml
- Fix Error: The webservice pod will not start - Error from dependencies init container: fe_sendauth: error sending password authentication
- Get postgres password from secret
kubectl exec -it gitlab-toolbox -n gitlab -- /bin/bash
psql -d gitlabhq_production -h gitlab-postgresql -u gitlab
SET password_encryption = 'scram-sha-256';
ALTER USER "gitlab" with password 'DB_PASSWORD_HERE"
- Set up gitlab for test
- Log in as
root
with default password - Create a group
- Create a project inside the group - blank with README.MD
- Go to user settings and create a PAT
- Execute Test
- Create a test maven package
- Update pom.xml with test server URL and project ID
- Update settings.xml with PAT
mvn deploy
Configuration used
Values.yaml:
global:
hosts:
domain: example.domain.com
externalIP: 1.1.1.1
gitlab:
name: gitlab.example.domain.com
registry:
name: registry.example.domain.com
minio:
name: minio.example.domain.com
rails:
bootsnap:
enabled: false
shell:
# Configure the clone link in the UI to include the high-numbered NodePort
# value from below (`gitlab.gitlab-shell.service.nodePort`)
port: 32022
gitlab:
webservice:
image:
tag: v15.5.1-ubi8
minReplicas: 1
maxReplicas: 1
resources:
limits:
memory: 2G
requests:
cpu: 100m
memory: 900M
workhorse:
tag: v15.5.1-ubi8
resources:
limits:
memory: 100M
requests:
cpu: 10m
memory: 10M
kas:
minReplicas: 1
maxReplicas: 1
sidekiq:
minReplicas: 1
maxReplicas: 1
resources:
limits:
memory: 2G
requests:
cpu: 300m
memory: 1G
gitlab-shell:
minReplicas: 1
maxReplicas: 1
service:
type: NodePort
nodePort: 32022
registry:
hpa:
minReplicas: 1
maxReplicas: 1
Current behavior
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.205 s
[INFO] Finished at: 2022-10-29T20:44:19Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy)
on project maven-test: Failed to deploy artifacts:
Could not transfer artifact net.test.testing:maven-test:jar:1.8-20221029.204416-1
from/to maven-test (https://URL/api/v4/projects/2/packages/maven): transfer failed for
https://URL/api/v4/projects/2/packages/maven/net/testlab/testing/maven-test/1.8-SNAPSHOT/maven-test-1.8-20221029.204416-1.jar,
status: 422 Unprocessable Entity -> [Help 1]
Expected behavior
Package uploaded to maven repo successfully
Versions
- Chart: gitlab-6.5.1
- Platform:
- Self-hosted: k3s
- Kubernetes:
- Client: 1.25.3
- Server: 1.25.3
- Helm: v3.10.1
Implementing a fix
Edited by David Fernandez