Merge request button spins indefinitely with a failing custom git hook
Summary
If there is a git hook that hits an exit code of != 0, the merge request button spinner spins indefinitely. The merge is not committed. Refreshing the page returns the button to its "waiting to be pressed" state.
On the machine, the hooks do run to completion. They are not appearing to hang.
Steps to reproduce
- Create a merge request from one branch to another (diff does not matter, testing with an empty commit).
- Create a
custom_hooks/pre-receive
file in the repository on the gitlab server (with the content following this list) - Click the "Accept Merge Request" button on the MR page.
pre-receive content:
#!/usr/bin/env bash
echo "Testing failing merge request"
exit 1
or
#!/usr/bin/env ruby
STDERR.puts "Testing failing merge request"
exit(1)
Expected behavior
The merge request should fail and the spinner should be replaced with the text "Testing failing merge request"
Actual behavior
The merge request spinner continues to spin indefinitely.
Results of GitLab environment info
Comes back green. Truncated output below:
GitLab Shell version >= 4.1.1 ? ... OK (4.1.1)
...
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.3)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.10.2)
Active users: 79
Gitlab EE version: 8.16.4
# lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.2.1511 (Core)
Release: 7.2.1511
Codename: Core
Possible fixes
We only noticed this issue occuring AFTER upgrading from 8.15.3 to 8.16.3. We have since tried to upgrade to 8.16.4 to confirm the most recent release had the same issue.
We did notice some very weird behavior with the embedded ruby on the box.
- Save the above example of the pre-receive hook into a file on the gitlab server.
- Open irb (
/opt/gitlab/embedded/bin/irb
) - Execute the following
`./pre-receive`
=> "test\n"
puts $?.inspect
You would expect to see #<Process::Status: pid 9467 exit 1>
. However, what you do see is: #<Process::Status: pid 9544 exit 0>
. As you can see, the exit code is wrong. Trying this locally on my machine, which the same revision of ruby (2.3.3p222 REV 56859), we see the expect behavior:
`./pre-receive`
=> "test\n"
puts $?.inspect
#<Process::Status: pid 4106 exit 1>
=> nil
Another oddity in the above scenario using the embedded ruby on the gitlab server:
irb(main):001:0> p $?.inspect
"#<Process::Status: pid 10642 exit 0>"
=> "#<Process::Status: pid 10642 exit 0>"
irb(main):002:0> p $?.inspect
"#<Process::Status: pid 10649 exit 0>"
=> "#<Process::Status: pid 10649 exit 0>"
irb(main):003:0> p $?.inspect
"#<Process::Status: pid 10656 exit 0>"
=> "#<Process::Status: pid 10656 exit 0>"
irb(main):004:0> p $?.inspect
"#<Process::Status: pid 10664 exit 0>"
=> "#<Process::Status: pid 10664 exit 0>"
irb(main):005:0> p $?.inspect
"#<Process::Status: pid 10671 exit 0>"
=> "#<Process::Status: pid 10671 exit 0>"
Compared to locally:
irb(main):008:0* puts $?.inspect
#<Process::Status: pid 7611 exit 1>
=> nil
irb(main):009:0> puts $?.inspect
#<Process::Status: pid 7611 exit 1>
=> nil
irb(main):010:0> puts $?.inspect
#<Process::Status: pid 7611 exit 1>
=> nil
irb(main):011:0> puts $?.inspect
#<Process::Status: pid 7611 exit 1>
=> nil
irb(main):012:0> puts $?.inspect
#<Process::Status: pid 7611 exit 1>
=> nil
As you can see, the PID rotates with the gitlab irb console, but not locally. You can confirm that the IRB console is static in both locations by running:
p $$
a few times and seeing that the PID for the ongoing process doesn't change.
I don't know how related this is to whats going on with the above merge request issue, but it definitely seemed strange and worth mentioning.
For what its worth, this issue ONLY occurs with the embedded ruby/irb on the gitlab server. It does not happen with the pre-installed ruby (from the OS).