Skip to content

Ensure we advertise screenshots taken when a QA failure happen outside of an RSpec example

Rémy Coutable requested to merge qa-197 into master

What does this MR do?

This happens when the failure happens outside of an RSpec example, in that case we do save the screenshot manually in Session#perform with screenshot_and_save_page, we just don't advertise the screenshots. Also, by setting Capybara::Screenshot.autosave_on_failure = false after we take the screenshot, we can ensure that RSpec won't try to take a screenshot afterward (since we already did manually) and show us the WARN: Screenshot could not be saved. page.current_path is empty. message.

Are there points in the code the reviewer needs to double check?

With an exception while a session is active but outside of an RSpec::Core::ExampleGroup

› CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/api/users_spec.rb:10
Run options: include {:core=>true, :locations=>{"./qa/specs/features/api/users_spec.rb"=>[10]}}

Randomized with seed 27622

API users
  when authenticated
  Image screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-23-30.747.png
  HTML screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-23-30.747.html
    get list of users (FAILED - 1)

Failures:

  1) API users when authenticated get list of users
     Failure/Error: let(:request) { Runtime::API::Request.new(@api_client, '/users') }
     
     RuntimeError:
       Exception while a session is active but outside of an RSpec::Core::ExampleGroup
     # ./qa/runtime/api/client.rb:33:in `block in create_personal_access_token'
     # ./qa/runtime/browser.rb:105:in `perform'
     # ./qa/runtime/browser.rb:27:in `visit'
     # ./qa/runtime/browser.rb:31:in `visit'
     # ./qa/runtime/api/client.rb:31:in `create_personal_access_token'
     # ./qa/runtime/api/client.rb:24:in `get_personal_access_token'
     # ./qa/runtime/api/client.rb:15:in `personal_access_token'
     # ./qa/runtime/api/request.rb:8:in `initialize'
     # ./qa/specs/features/api/users_spec.rb:8:in `new'

The manually taken screenshots are advertised.

With an exception while no session is active inside of an RSpec::Core::ExampleGroup

› CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/api/users_spec.rb:10
Run options: include {:core=>true, :locations=>{"./qa/specs/features/api/users_spec.rb"=>[10]}}

Randomized with seed 46755

API users
  when authenticated
WARN: Screenshot could not be saved. `page.current_path` is empty.
    get list of users (FAILED - 1)

Failures:

  1) API users when authenticated get list of users
     Failure/Error: raise 'Exception while no session is active inside of an RSpec::Core::ExampleGroup'
     
     RuntimeError:
       Exception while no session is active inside of an RSpec::Core::ExampleGroup
     # ./qa/specs/features/api/users_spec.rb:12:in `block (3 levels) in <module:QA>'

In this case the warning is valid since there's no active session.

With an exception while a session is active inside of an RSpec::Core::ExampleGroup

› CHROME_HEADLESS=0 bin/qa Test::Instance http://localhost:3000 -- qa/specs/features/login/standard_spec.rb 
Run options: include {:core=>true}

Randomized with seed 4702

standard user login
  user logs in using credentials (FAILED - 1)
  HTML screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-28-55.599.html
  Image screenshot: file:///Users/remy/Code/GitLab/gdk/gitlab/qa/tmp/screenshot_2018-06-13-09-28-55.599.png

Failures:

  1) standard user login user logs in using credentials
     Failure/Error: raise 'Exception while a session is active inside of an RSpec::Core::ExampleGroup'
     
     RuntimeError:
       Exception while a session is active inside of an RSpec::Core::ExampleGroup
     # ./qa/specs/features/login/standard_spec.rb:6:in `block (2 levels) in <module:QA>'

The automatically (by capybara-screenshot/rspec) taken screenshots are advertised as before.

Does this MR meet the acceptance criteria?

What are the relevant issue numbers?

Fixes gitlab-qa#197 (closed).

Edited by Rémy Coutable

Merge request reports