Verifying runner failed status=404 Not Found when using runner 9.0.0 with CE 8.16.7

Summary

If your central instance runs Gitlab CE 8.16.7 and your runners use gitlab-ci-multi-runner 9.0.0 (because you upgrade them nightly with apt-get) they will stop picking up builds. gitlab-runner verify reports failure to verify

Downgrading to an older runner version fixes the problem.

Background: Central server is maintained by someone else, so I can't upgrade it myself, but I maintain my own runners. To ensure I get any urgent security updates, I install software updates with apt-get upgrade every night.

Steps to reproduce

  1. Server running Gitlab CE 8.16.7
  2. Runner running previous version of gitlab-ci-multi-runner, registered with the server and running builds.
  3. Runner upgrades to gitlab-ci-multi-runner 9.0.0

Actual behavior

  1. Runner stops picking up builds.
  2. Server displays green traffic light for runner, does not warn about stuck builds.
  3. Investigating on runner reveals:
mtandy@gitlab-builder-mtandy:~$ sudo gitlab-runner status
gitlab-runner: Service is running!
mtandy@gitlab-builder-mtandy:~$ sudo gitlab-runner verify
Running in system-mode.                            
                                                   
ERROR: Verifying runner... failed                   runner=4e77b075 status=404 Not Found
ERROR: Verifying runner... failed                   runner=707a0bc6 status=404 Not Found
  1. Can fix problem by downgrading like so:
sudo apt-get install gitlab-ci-multi-runner=1.9.5
sudo apt-mark hold gitlab-ci-multi-runner

Expected behavior

Either (a) new runner versions should maintain backwards compatibility or (b) something in the wiki or documentation should provide a recommended config to automatically install non-breaking updates without installing breaking updates.

Environment description

Runners on Ubuntu 16.04 LTS. Runners configured in config.toml like:

[[runners]]
  name = <snip>
  url = <snip>
  token = <snip>
  executor = "docker"
  limit = 1
  [runners.docker]
    tls_verify = false
    image = "ubuntu:16.04"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    dns = [<snip>]
  [runners.cache]
  [runners.kubernetes]
    host = ""
    cert_file = ""
    key_file = ""
    ca_file = ""
    image = ""
    namespace = ""
    privileged = false
    cpus = ""
    memory = ""
    service_cpus = ""
    service_memory = ""

Used GitLab Runner version

$ sudo gitlab-runner --version
Version:      9.0.0
Git revision: 08a9e6f
Git branch:   9-0-stable
GO version:   go1.7.5
Built:        Wed, 22 Mar 2017 16:29:52 +0000
OS/Arch:      linux/amd64