ERROR: Uploading artifacts to coordinator... too large archive

Summary

(Summarize the bug encountered concisely) I got error "ERROR: Uploading artifacts to coordinator... too large archive " when try testing gitlab ci/cd with k8s cluster.

Steps to reproduce

  1. Create sample project with Spring boot template
  2. Create k8s cluster with rancher
  3. Add k8s cluster and install tiller, runner
  4. Create a test gitlab-ci.yaml with mvn
variables:
  # docker
  SERVICE_NAME: test-cicd
  SERVICE_VERSION: 0.0.1
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""
  DOCKER_HOST: tcp://localhost:2375
  #mvn
  MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
  MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository "
 
cache:
  key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG"
  paths:
    - .m2/repository
  
default:
    image: docker:18.09
    services:
      - name: docker:18.09-dind
        command: ["--insecure-registry=10.10.1.13:5002"]

stages:
  - verify
  - build
  
verify:
  stage: verify
  image: maven:3.6.2-ibmjava-8-alpine
  script:
  - mvn  $MAVEN_CLI_OPTS verify
  artifacts:
    reports:
      junit:
        - target/surefire-reports/TEST-*.xml
        - target/failsafe-reports/TEST-*.xml
  when: manual
  
package:
  stage: build
  image: maven:3.6.2-ibmjava-8-alpine
  script:
  - mvn  $MAVEN_CLI_OPTS package
  - ls -la
  artifacts:
    paths:
      - target/$SERVICE_NAME-$SERVICE_VERSION.jar
  1. Increase Maximum artifacts size (MB) to 4096 in Admin setting
  2. Add "client_max_body_size 0;" to nginx config in /var/opt/gitlab/nginx/conf/gitlab-http.conf Add nginx['client_max_body_size'] = '0' to gitlab.rb
  3. run gitlab-ctl reconfigure and restart
  4. Run pipeline (How one can reproduce the issue - this is very important)

Example Project

Spring boot template

What is the current bug behavior?

The stage build ended with error:

Creating cache build-master-6...
.m2/repository: found 2253 matching files          
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. 
Created cache
Uploading artifacts...
target/test-cicd-0.0.1.jar: found 1 matching files 
ERROR: Uploading artifacts to coordinator... too large archive  id=3398 responseStatus=413 Request Entity Too Large status=413 Request Entity Too Large token=yNLyY4xT
FATAL: too large                                   
ERROR: Job failed: command terminated with exit code 1

As I checked the jar file is around 16mb

What is the expected correct behavior?

Upload artifact successfully.

Relevant logs and/or screenshots

Screenshot_from_2019-10-07_10-23-18

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:env:info)




System information
System:		
Current User:	git
Using RVM:	no
Ruby Version:	2.6.3p62
Gem Version:	2.7.9
Bundler Version:1.17.3
Rake Version:	12.3.2
Redis Version:	3.2.12
Git Version:	2.22.0
Sidekiq Version:5.2.7
Go Version:	unknown

GitLab information
Version:	12.3.1
Revision:	3440d0f6100
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	9.6.14
URL:		http://gitlab.foxpay.vn
HTTP Clone URL:	http://gitlab.foxpay.vn/some-group/some-project.git
SSH Clone URL:	git@gitlab.foxpay.vn:some-group/some-project.git
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	10.0.0
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)




Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 10.0.0 ? ... OK (10.0.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK

gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ... 
5/1 ... yes
5/2 ... yes
5/3 ... yes
5/4 ... yes
6/5 ... yes
5/6 ... yes
6/7 ... yes
10/8 ... yes
2/10 ... yes
11/11 ... yes
2/12 ... yes
3/13 ... yes
6/14 ... yes
4/15 ... yes
11/16 ... yes
6/17 ... yes
3/18 ... yes
12/22 ... yes
12/23 ... yes
2/24 ... yes
3/25 ... yes
3/26 ... yes
3/27 ... yes
7/29 ... yes
13/30 ... yes
16/31 ... yes
20/32 ... yes
18/33 ... yes
20/34 ... yes
12/35 ... yes
16/36 ... yes
20/39 ... yes
11/41 ... yes
20/42 ... yes
12/43 ... yes
12/44 ... yes
11/45 ... yes
1/46 ... yes
23/48 ... yes
23/49 ... yes
23/51 ... yes
23/52 ... yes
23/54 ... yes
31/55 ... yes
31/56 ... yes
3/57 ... yes
3/58 ... yes
23/59 ... yes
23/60 ... yes
35/61 ... yes
32/63 ... yes
23/64 ... yes
7/65 ... yes
31/66 ... yes
31/67 ... yes
12/68 ... yes
23/69 ... yes
16/70 ... yes
21/71 ... yes
7/73 ... yes
16/74 ... yes
16/75 ... yes
16/76 ... yes
47/77 ... yes
23/80 ... yes
36/81 ... yes
21/85 ... yes
12/87 ... yes
12/88 ... yes
23/89 ... yes
12/92 ... yes
16/93 ... yes
16/94 ... yes
12/95 ... yes
49/97 ... yes
49/98 ... yes
49/99 ... yes
49/100 ... yes
49/101 ... yes
49/103 ... yes
12/104 ... yes
21/106 ... yes
21/107 ... yes
21/108 ... yes
21/109 ... yes
21/110 ... yes
21/111 ... yes
16/112 ... yes
49/114 ... yes
16/115 ... yes
1/116 ... yes
16/117 ... yes
12/118 ... yes
21/119 ... yes
12/120 ... yes
12/122 ... yes
21/123 ... yes
21/124 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.3)
Git version >= 2.22.0 ? ... yes (2.22.0)
Git user has default SSH configuration? ... yes
Active users: ... 39
Is authorized keys file accessible? ... yes

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

Try solution on similar issue like Admin setting and nginx configuration but nothing worked.