Related to https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1987
cc @rspeicher @stanhu @brodock