Screenshot diffing in GitLab - Meta

Screenshot diffing(visual regression testing) in GitLab meta issue,

Fork gitlab-ce

  • Create a personal gitlab-ce fork to test with
    • Might not be necessary

Run GitLab rspec tests through the GitLab Selenium Server/proxy

Fingerprint each test

  • Fingerprint each test by their nested describe/it name so we can easily compare master to the feature branch

spec/support/capybara.rb

RSpec.configure do |config|
  config.before(:example, :js) do |example|
    # POST "/session/#{Capybara.current_session.driver.browser.capabilities['webdriver.remote.sessionid']}/gitlab-meta" with name of spec
    #req.body = {
    #  description: example.metadata[:full_description],
    #  location: example.metadata[:location],
    #}.to_json
  end
end

This would add a log entry in selenium-logs.json

{
  "gitlab": [ 
    {
      "id": "a10d9a192e-1520453590211",
      "type": null,
      "path": "/session/c69eb6fd-8df2-4d9c-a704-dea48fa921e3/gitlab-meta"
      "info": {
        "specName": "Create notes on issues mentioning issue on a private project"
      }
    },
  ],
  ...
}

Screenshot diffing

Flesh out GitLab Selenium proxy/server

After we have a PoC for all of the above points, we need to come back to the GitLab Selenium proxy and flesh out all of the aspects and make sure things are properly tested.

  • Add more screenshot trigger criteria, #4
  • Run against someone else's existing project Selenium tests - QA sanity test, #2
  • Ensure everything has test coverage

CI views

Auto DevOps

  • During the Auto DevOps test stage (where Selenium tests would run)
    • Startup GitLab Selenium Proxy (and if necessary backing Selenium server/chromedriver)
    • Define SELENIUM_REMOTE_URL: http://localhost:4545/wd/hub (points at GitLab Selenium Proxy)
    • Define GITLAB_TARGET_SELENIUM_REMOTE_URL: http://localhost:4444/wd/hub (points at backing Selenium server)
  • It would make things a lot more slick if we had Docker networks connected between services, gitlab-runner!1041 (closed)
Edited by Eric Eastwood