Web terminal connection failure - mux handler not found

Summary

Trying to use web terminals results in a connection failure.

Steps to reproduce

  • Setup HAproxy to handle SSL (websocket config based on this page)

    # relevant frontend/backend
    frontend vip-https                                               
      bind virtual_ip:443 ssl no-sslv3 crt /etc/ssl/private/ssl.pem
      mode http                                                      
      rspadd Strict-Transport-Security:\ max-age=15768000            
      acl is_request_for_git hdr(Host) -i gitlab
      acl is_websocket hdr(Upgrade) -i WebSocket                     
      use_backend git-ws if is_request_for_git is_websocket          
      use_backend git-server if is_request_for_git                   
    
    backend git-ws                                                   
      mode http                                                      
      timeout tunnel 3600s                                           
      stats enable                                                   
      reqadd X-Forwarded-Connection:\ Upgrade                        
      reqadd X-Forwarded-Upgrade:\ websocket                         
      server runner runner-ip:8093 check
  • Setup gitlab v11.8.1 using omnibus leveraging HAproxy for SSL

    # important part of the gitlab config
    external_url "https://git-dev"
    nginx['listen_port'] = 80
    nginx['listen_https'] = false
    nginx['proxy_set_headers'] = {"X-Forwarded-Proto"=>"https", "X-Forwarded-Ssl"=>"no"}
  • Setup gitlab runner v11.8.0 on Ubuntu 18.04 using the docker executor (config based on this doc)

    # runner config
    concurrent = 1                                
    check_interval = 0                            
                                            
    [session_server]                              
      listen_address = "runner-ip:8093"         
      session_timeout = 1800                      
                                            
    [[runners]]                                   
      name = "test web terminals"                 
      url = "https://git-dev/"
      token = "token"              
      executor = "docker"                         
      [runners.docker]                            
        tls_verify = false                        
        image = "ruby:2.5.3"                      
        privileged = false                        
        disable_entrypoint_overwrite = false      
        oom_kill_disable = false                  
        disable_cache = false                     
        volumes = ["/cache"]                      
        shm_size = 0                              
      [runners.cache]                             
        [runners.cache.s3]                        
        [runners.cache.gcs]                       
  • Setup a project that uses Gitlab CI

Actual behavior

  • Push a commit to trigger a Gitlab CI job

  • Go to the running job's page

  • Click the Debug button on the job's page to access the web terminal and see this:

    • image
  • And this in Chrome's developer tools JS console:

    WebSocket connection to 'wss://mygitlab/mygroup/myproject/-/jobs/myjobid/terminal.ws' failed:
    Error during WebSocket handshake: Unexpected response code: 404
  • In /var/log/syslog on the gitlab runner:

    gitlab-runner[7397]: #033[31;1mERROR: Mux handler not found #033[0;m  #033[31;1mbuilds#033[0;m=0 #033[31;1muri#033[0;m=/mygroup/myproject/-/jobs/myjobid/terminal.ws
  • Be sad 😞

Expected behavior

I should be able to access the web terminal. Please?

Edited by 🤖 GitLab Bot 🤖