Environment errors when environment URL is not resolvable
Summary
The problem is two-sided for both creating an environment and stopping the environment when the URL set is not resolvable.
- When creating an environment with an
url
set to a hostname that is not resolved, the link to the URL does not appear in the environment page - When an environment was successfully created (URL resolvable at the moment of creation) and then trying to stop the environment, a 500 error appears
Steps to reproduce
To reproduce problem 1 above:
- Create a new project
- Add a simple .gitlab-ci.yml file:
stages:
- start
- stop
start:dev:
stage: start
environment:
name: demonstrates-environment-dns-issue
url: https://this-does-not-resolve.com
on_stop: stop:dev
script:
- echo starting
stop:dev:
stage: stop
environment:
name: demonstrates-environment-dns-issue
action: stop
when: manual
script:
- echo stopping
- Go to environment page => the environment link is not shown
To reproduce problem 2 above:
- Create a new project
- Add a .gitlab-ci.yml file as above, but replace
https://this-does-not-resolve.com
with a DNS that you control. - Start the job, create the environment
- Remove the A record for the URL you've set up in step 2
- Try to stop the environment => 500, with the logs pasted below
Example Project
For the problem 1 above, see cirimie/repro-environment-urls> environment 936581
I've reproduced problem 2 on the latest (12.2) version locally with the logs pasted below.
What is the current bug behavior?
- Environment link is not visible when environment URL is not resolvable
- Started environment cannot be stopped when environment URL was resolvable when environment was created but is not resolvable when stopping
What is the expected correct behavior?
Either, this should be properly documented (as a limitation, or why this happens) or:
- Environment link should be visible when environment URL was not resolvable
- Started environment can be stopped even if URL is not resolvable anymore
Relevant logs and/or screenshots
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/root/repro-environment-addressable-urls/environments/1/stop" for 192.168.56.101 at 2019-08-28 18:36:18 +0100
Processing by Projects::EnvironmentsController#stop as HTML
Parameters: {"authenticity_token"=>"[FILTERED]", "namespace_id"=>"root", "project_id"=>"repro-environment-addressable-urls", "id"=>"1"}
Completed 500 Internal Server Error in 331ms (ActiveRecord: 27.5ms | Elasticsearch: 0.0ms)
StateMachines::InvalidTransition (Cannot transition state via :stop from :available (Reason(s): External url is blocked: Host cannot be resolved or invalid)):
app/models/environment.rb:148:in `stop_with_action!'
lib/gitlab/metrics/instrumentation.rb:161:in `block in stop_with_action!'
lib/gitlab/metrics/method_call.rb:36:in `measure'
lib/gitlab/metrics/instrumentation.rb:161:in `stop_with_action!'
app/controllers/projects/environments_controller.rb:93:in `stop'
ee/lib/gitlab/ip_address_state.rb:10:in `with'
ee/app/controllers/ee/application_controller.rb:28:in `set_current_ip_address'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:445:in `set_session_storage'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:439:in `set_locale'
lib/gitlab/middleware/rails_queue_duration.rb:27:in `call'
lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
lib/gitlab/metrics/transaction.rb:57:in `run'
lib/gitlab/metrics/rack_middleware.rb:17:in `call'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:16:in `call'
ee/lib/gitlab/jira/middleware.rb:17:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:42:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:26:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'
Possible fixes
Line 36 in the environment model has an addressable_url
check, which runs when the state machine tries to switch states and it will fail if the DNS can't be resolved, producing the 500 above