postreceive script breaks CI when using subgit due to missing GL_REPOSITORY var

Summary

The postreceive script assumes that the required repository id will be in the environment variable: GL_REPOSITORY. When commiting a change from subversion using subgit, this is not known. This script is regularly overwritten so customising this script doesn't fix this. I propose checking for a file to define this value, see below for code.

Steps to reproduce

  • Create a project with a CI script.
  • Configure subgit to mirror changes with a subversion repository
  • Commit change to subversion
  • note that the CI isn't run. the sidekiq log shows: 2018-02-27_20:00:59.00781 2018-02-27T20:00:59.007Z 3239 TID-osumwt2gc WARN: ArgumentError: Invalid GL Repository "" 2018-02-27_20:00:59.00781 2018-02-27T20:00:59.007Z 3239 TID-osumwt2gc WARN: /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gl_repository.rb:10:in 'parse'

What is the current bug behavior?

CI pipelines are not run on checkin

What is the expected correct behavior?

postreceive triggers the pipeline

Relevant logs and/or screenshots

Log snippit 

2018-02-27_10:48:37.16117 2018-02-27T10:48:37.160Z 3239 TID-osulogq74 PostReceive JID-b33134567ececc5949b
39260 INFO: fail: 0.001 sec
2018-02-27_10:48:37.16118 2018-02-27T10:48:37.160Z 3239 TID-osulogq74 WARN: {"context":"Job raised exception","job":{"class":"PostReceive","args":[null,"user-1","c14fd2fbb47400aa86c7e9511553116f12557276 a49458329926090b25f9cff084ebc051099bab5f refs/heads/master\n"],"retry":3,"queue":"post_receive","jid":"b33134567ececc5949b39260","created_at":1519728517.1566122,"enqueued_at":1519728517.1574302},"jobstr":"{\"class\":\"PostReceive\",\"args\":[null,\"user-1\",\"c14fd2fbb47400aa86c7e9511553116f12557276 a49458329926090b25f9cff084ebc051099bab5f refs/heads/master\\n\"],\"retry\":3,\"queue\":\"post_receive\",\"jid\":\"b33134567ececc5949b39260\",\"created_at\":1519728517.1566122,\"enqueued_at\":1519728517.1574302}"}
2018-02-27_10:48:37.16119 2018-02-27T10:48:37.160Z 3239 TID-osulogq74 WARN: ArgumentError: Invalid GL Repository ""
2018-02-27_10:48:37.16120 2018-02-27T10:48:37.160Z 3239 TID-osulogq74 WARN: /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gl_repository.rb:10:in `parse'
2018-02-27_10:48:37.16120 /opt/gitlab/embedded/service/gitlab-rails/app/workers/post_receive.rb:5:in `perform'
2018-02-27_10:48:37.16121 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
2018-02-27_10:48:37.16121 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
2018-02-27_10:48:37.16121 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
2018-02-27_10:48:37.16122 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
2018-02-27_10:48:37.16122 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-02-27_10:48:37.16122 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call'
2018-02-27_10:48:37.16122 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-02-27_10:48:37.16122 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:16:in `call'
2018-02-27_10:48:37.16123 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-02-27_10:48:37.16123 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/server/active_record.rb:15:in `call'
2018-02-27_10:48:37.16123 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-02-27_10:48:37.16124 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sentry-raven-2.5.3/lib/raven/integrations/sidekiq.rb:7:in `call'
2018-02-27_10:48:37.16124 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-02-27_10:48:37.16124 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:133:in `invoke'
2018-02-27_10:48:37.16124 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:169:in `block in process'
2018-02-27_10:48:37.16125 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:141:in `block (6 levels) in dispatch'
2018-02-27_10:48:37.16125 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:97:in `local'

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		
Current User:	git
Using RVM:	no
Ruby Version:	2.3.6p384
Gem Version:	2.6.13
Bundler Version:1.13.7
Rake Version:	12.3.0
Redis Version:	3.2.11
Git Version:	2.14.3
Sidekiq Version:5.0.5
Go Version:	unknown

GitLab information
Version:	10.5.2
Revision:	b951e0d
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
URL:		https://cqp-dev.phy.bris.ac.uk
HTTP Clone URL:	https://cqp-dev.phy.bris.ac.uk/some-group/some-project.git
SSH Clone URL:	git@cqp-dev.phy.bris.ac.uk:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers: github, google_oauth2


GitLab Shell
Version:	6.0.3
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab Shell ...

GitLab Shell version >= 6.0.3 ? ... OK (6.0.3)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
2/11 ... ok
2/12 ... ok
2/13 ... ok
6/14 ... ok
13/15 ... ok
5/16 ... repository is empty
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Reply by email is disabled in config/gitlab.yml
Checking LDAP ...


Server: ldapuob
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
DN: uid=drjrja,cn=users,dc=bris,dc=ac,dc=uk	 uid: drjrja
DN: uid=anmaa,cn=users,dc=bris,dc=ac,dc=uk	 uid: anmaa
DN: uid=merda,cn=users,dc=bris,dc=ac,dc=uk	 uid: merda
DN: uid=corwa,cn=users,dc=bris,dc=ac,dc=uk	 uid: corwa
DN: uid=hibwea,cn=users,dc=bris,dc=ac,dc=uk	 uid: hibwea
DN: uid=mdsja,cn=users,dc=bris,dc=ac,dc=uk	 uid: mdsja
DN: uid=ggmga,cn=users,dc=bris,dc=ac,dc=uk	 uid: ggmga
DN: uid=lwrea,cn=users,dc=bris,dc=ac,dc=uk	 uid: lwrea
DN: uid=opwja,cn=users,dc=bris,dc=ac,dc=uk	 uid: opwja
DN: uid=pydma,cn=users,dc=bris,dc=ac,dc=uk	 uid: pydma
DN: uid=cpmnra,cn=users,dc=bris,dc=ac,dc=uk	 uid: cpmnra
DN: uid=litfd,cn=users,dc=bris,dc=ac,dc=uk	 uid: litfd
DN: uid=chcmr,cn=users,dc=bris,dc=ac,dc=uk	 uid: chcmr
DN: uid=bidjs,cn=users,dc=bris,dc=ac,dc=uk	 uid: bidjs
DN: uid=csrmt,cn=users,dc=bris,dc=ac,dc=uk	 uid: csrmt
DN: uid=bomlg,cn=users,dc=bris,dc=ac,dc=uk	 uid: bomlg
DN: uid=burdl,cn=users,dc=bris,dc=ac,dc=uk	 uid: burdl
DN: uid=omncc,cn=users,dc=bris,dc=ac,dc=uk	 uid: omncc
DN: uid=phedta,cn=users,dc=bris,dc=ac,dc=uk	 uid: phedta
DN: uid=ecca,cn=users,dc=bris,dc=ac,dc=uk	 uid: ecca
DN: uid=ggjob,cn=users,dc=bris,dc=ac,dc=uk	 uid: ggjob
DN: uid=lijdcb,cn=users,dc=bris,dc=ac,dc=uk	 uid: lijdcb
DN: uid=enjfb,cn=users,dc=bris,dc=ac,dc=uk	 uid: enjfb
DN: uid=ctggbk,cn=users,dc=bris,dc=ac,dc=uk	 uid: ctggbk
DN: uid=mawhhb,cn=users,dc=bris,dc=ac,dc=uk	 uid: mawhhb
DN: uid=sompb,cn=users,dc=bris,dc=ac,dc=uk	 uid: sompb
DN: uid=phpjb,cn=users,dc=bris,dc=ac,dc=uk	 uid: phpjb
DN: uid=ogrjb,cn=users,dc=bris,dc=ac,dc=uk	 uid: ogrjb
DN: uid=edsbb,cn=users,dc=bris,dc=ac,dc=uk	 uid: edsbb
DN: uid=lvarsb,cn=users,dc=bris,dc=ac,dc=uk	 uid: lvarsb
DN: uid=cppb,cn=users,dc=bris,dc=ac,dc=uk	 uid: cppb
DN: uid=eemhb,cn=users,dc=bris,dc=ac,dc=uk	 uid: eemhb
DN: uid=rumgb,cn=users,dc=bris,dc=ac,dc=uk	 uid: rumgb
DN: uid=ggkab,cn=users,dc=bris,dc=ac,dc=uk	 uid: ggkab
DN: uid=papmb,cn=users,dc=bris,dc=ac,dc=uk	 uid: papmb
DN: uid=drgb,cn=users,dc=bris,dc=ac,dc=uk	 uid: drgb
DN: uid=bojeb,cn=users,dc=bris,dc=ac,dc=uk	 uid: bojeb
DN: uid=phmvb,cn=users,dc=bris,dc=ac,dc=uk	 uid: phmvb
DN: uid=emeb,cn=users,dc=bris,dc=ac,dc=uk	 uid: emeb
DN: uid=ccab,cn=users,dc=bris,dc=ac,dc=uk	 uid: ccab
DN: uid=cedib,cn=users,dc=bris,dc=ac,dc=uk	 uid: cedib
DN: uid=eesrb,cn=users,dc=bris,dc=ac,dc=uk	 uid: eesrb
DN: uid=mawgcb,cn=users,dc=bris,dc=ac,dc=uk	 uid: mawgcb
DN: uid=fejkb,cn=users,dc=bris,dc=ac,dc=uk	 uid: fejkb
DN: uid=pajwbb,cn=users,dc=bris,dc=ac,dc=uk	 uid: pajwbb
DN: uid=dreb,cn=users,dc=bris,dc=ac,dc=uk	 uid: dreb
DN: uid=corgb,cn=users,dc=bris,dc=ac,dc=uk	 uid: corgb
DN: uid=ecab,cn=users,dc=bris,dc=ac,dc=uk	 uid: ecab
DN: uid=edpmb,cn=users,dc=bris,dc=ac,dc=uk	 uid: edpmb
DN: uid=spdrb,cn=users,dc=bris,dc=ac,dc=uk	 uid: spdrb
DN: uid=pakwb,cn=users,dc=bris,dc=ac,dc=uk	 uid: pakwb
DN: uid=sorxb,cn=users,dc=bris,dc=ac,dc=uk	 uid: sorxb
DN: uid=aepcb,cn=users,dc=bris,dc=ac,dc=uk	 uid: aepcb
DN: uid=ecsb,cn=users,dc=bris,dc=ac,dc=uk	 uid: ecsb
DN: uid=clrgab,cn=users,dc=bris,dc=ac,dc=uk	 uid: clrgab
DN: uid=pydac,cn=users,dc=bris,dc=ac,dc=uk	 uid: pydac
DN: uid=chjaxs,cn=users,dc=bris,dc=ac,dc=uk	 uid: chjaxs
DN: uid=potfc,cn=users,dc=bris,dc=ac,dc=uk	 uid: potfc
DN: uid=phdc,cn=users,dc=bris,dc=ac,dc=uk	 uid: phdc
DN: uid=lwdnc,cn=users,dc=bris,dc=ac,dc=uk	 uid: lwdnc
DN: uid=megcc,cn=users,dc=bris,dc=ac,dc=uk	 uid: megcc
DN: uid=hicgac,cn=users,dc=bris,dc=ac,dc=uk	 uid: hicgac
DN: uid=enrrc,cn=users,dc=bris,dc=ac,dc=uk	 uid: enrrc
DN: uid=epjrtc,cn=users,dc=bris,dc=ac,dc=uk	 uid: epjrtc
DN: uid=maejc,cn=users,dc=bris,dc=ac,dc=uk	 uid: maejc
DN: uid=cingc,cn=users,dc=bris,dc=ac,dc=uk	 uid: cingc
DN: uid=mdapc,cn=users,dc=bris,dc=ac,dc=uk	 uid: mdapc
DN: uid=runjc,cn=users,dc=bris,dc=ac,dc=uk	 uid: runjc
DN: uid=emmdc,cn=users,dc=bris,dc=ac,dc=uk	 uid: emmdc
DN: uid=eeeld,cn=users,dc=bris,dc=ac,dc=uk	 uid: eeeld
DN: uid=rterd,cn=users,dc=bris,dc=ac,dc=uk	 uid: rterd
DN: uid=eerjd,cn=users,dc=bris,dc=ac,dc=uk	 uid: eerjd
DN: uid=adsrd,cn=users,dc=bris,dc=ac,dc=uk	 uid: adsrd
DN: uid=lwgcd,cn=users,dc=bris,dc=ac,dc=uk	 uid: lwgcd
DN: uid=cejpd,cn=users,dc=bris,dc=ac,dc=uk	 uid: cejpd
DN: uid=phjccd,cn=users,dc=bris,dc=ac,dc=uk	 uid: phjccd
DN: uid=birmd,cn=users,dc=bris,dc=ac,dc=uk	 uid: birmd
DN: uid=gldld,cn=users,dc=bris,dc=ac,dc=uk	 uid: gldld
DN: uid=ctrnd,cn=users,dc=bris,dc=ac,dc=uk	 uid: ctrnd
DN: uid=egged,cn=users,dc=bris,dc=ac,dc=uk	 uid: egged
DN: uid=uskjd,cn=users,dc=bris,dc=ac,dc=uk	 uid: uskjd
DN: uid=hiwd,cn=users,dc=bris,dc=ac,dc=uk	 uid: hiwd
DN: uid=papjd,cn=users,dc=bris,dc=ac,dc=uk	 uid: papjd
DN: uid=cdpmd,cn=users,dc=bris,dc=ac,dc=uk	 uid: cdpmd
DN: uid=opdle,cn=users,dc=bris,dc=ac,dc=uk	 uid: opdle
DN: uid=ecdae,cn=users,dc=bris,dc=ac,dc=uk	 uid: ecdae
DN: uid=meege,cn=users,dc=bris,dc=ac,dc=uk	 uid: meege
DN: uid=pacje,cn=users,dc=bris,dc=ac,dc=uk	 uid: pacje
DN: uid=pydam,cn=users,dc=bris,dc=ac,dc=uk	 uid: pydam
DN: uid=cdame,cn=users,dc=bris,dc=ac,dc=uk	 uid: cdame
DN: uid=phjee,cn=users,dc=bris,dc=ac,dc=uk	 uid: phjee
DN: uid=madve,cn=users,dc=bris,dc=ac,dc=uk	 uid: madve
DN: uid=phre,cn=users,dc=bris,dc=ac,dc=uk	 uid: phre
DN: uid=omjwe,cn=users,dc=bris,dc=ac,dc=uk	 uid: omjwe
DN: uid=saerf,cn=users,dc=bris,dc=ac,dc=uk	 uid: saerf
DN: uid=omadf,cn=users,dc=bris,dc=ac,dc=uk	 uid: omadf
DN: uid=socsf,cn=users,dc=bris,dc=ac,dc=uk	 uid: socsf
DN: uid=adngf,cn=users,dc=bris,dc=ac,dc=uk	 uid: adngf
DN: uid=sokdpf,cn=users,dc=bris,dc=ac,dc=uk	 uid: sokdpf
DN: uid=fherbf,cn=users,dc=bris,dc=ac,dc=uk	 uid: fherbf


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
2/11 ... yes
2/12 ... yes
2/13 ... yes
6/14 ... yes
13/15 ... yes
5/16 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.3.6)
Git version >= 2.9.5 ? ... yes (2.14.3)
Git user has default SSH configuration? ... yes
Active users: ... 9


Checking GitLab ... Finished

Possible fixes

I think the postreceive hook could be changed to the following which would support subgit without affecting any other systems. The project id can be stored in a file in the root of the git repository with content such as: project-11

#!/opt/gitlab/embedded/bin/ruby --disable-gems
# Fix the PATH so that gitlab-shell can find git-upload-pack and friends.
ENV['PATH'] = '/opt/gitlab/bin:/opt/gitlab/embedded/bin:' + ENV['PATH']

#!/usr/bin/env ruby

# This file was placed here by GitLab. It makes sure that your pushed commits
# will be processed properly.

refs = $stdin.read
key_id  = ENV.delete('GL_ID')
if ENV.has_key?('GL_REPOSITORY')
  gl_repository = ENV.delete('GL_REPOSITORY')

elsif File.exists?(Dir.pwd + '/GL_REPOSITORY')
  file = File.open(Dir.pwd + '/GL_REPOSITORY', 'r')
  gl_repository = file.read.chomp
  file.close
else
  STDERR.puts("GL_REPOSITORY is not set")
  exit 2
end

repo_path = Dir.pwd

require_relative '../lib/gitlab_custom_hook'
require_relative '../lib/gitlab_post_receive'

if GitlabPostReceive.new(gl_repository, repo_path, key_id, refs).exec &&
    GitlabCustomHook.new(repo_path, key_id).post_receive(refs)
  exit 0
else
  exit 1
end