Skip to content

WIP: Using the GitLab Selenium Proxy for screenshots of rspec tests

Eric Eastwood requested to merge gitlab-selenium-proxy-visual-regression-testing into master

What does this MR do?

See gitlab-selenium-server#5, https://gitlab.com/gitlab-org/gitlab-ee/issues/6278

Are there points in the code the reviewer needs to double check?

Debugging notes

Net::ReadTimeout because Chrome didn't start properly

I fixed the following error by bumping selenium/standalone-chrome:3.4.0-francium to selenium/standalone-chrome:3.11.0-californium, gitlab-selenium-server!7 (merged)

I was able to get more info than Net::ReadTimeout by logging out all of the HTTP requests we try to send to the Selenium server. Just add require 'httplog' to spec/support/capybara.rb

D, [2018-04-06T06:44:18.417251 #71] DEBUG -- : [httplog] Sending: POST http://gitlab-selenium-server:4545/wd/hub/session
D, [2018-04-06T06:44:18.417709 #71] DEBUG -- : [httplog] Data: {"desiredCapabilities":{"browserName":"chrome","version":"","platform":"ANY","javascriptEnabled":true,"cssSelectorsEnabled":true,"takesScreenshot":true,"nativeEvents":false,"rotatable":false,"loggingPrefs":{"browser":"ALL","client":"ALL","driver":"ALL","server":"ALL"},"chromeOptions":{"args":["window-size=1240,1400","no-sandbox","headless","disable-gpu","disable-dev-shm-usage"]}},"capabilities":{"firstMatch":[{"browserName":"chrome"}]}}
D, [2018-04-06T06:45:18.479737 #71] DEBUG -- : [httplog] Status: 500
D, [2018-04-06T06:45:18.479887 #71] DEBUG -- : [httplog] Benchmark: 60.061879 seconds
D, [2018-04-06T06:45:18.480107 #71] DEBUG -- : [httplog] Response:
{"state":"unknown error","sessionId":null,"hCode":95776705,"value":{"additionalInformation":"\nDriver info: driver.version: ChromeDriver","localizedMessage":"unknown error: Chrome failed to start: exited normally\n  (Driver info: chromedriver=2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8),platform=Linux 4.13.5-coreos-r2 x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 60.02 seconds\nBuild info: version: '3.4.0', revision: 'unknown', time: 'unknown'\nSystem info: host: '32a03e8d55b7', ip: '172.17.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '4.13.5-coreos-r2', java.version: '1.8.0_131'\nDriver info: driver.version: ChromeDriver","systemInformation":"System info: host: '32a03e8d55b7', ip: '172.17.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '4.13.5-coreos-r2', java.version: '1.8.0_131'","supportUrl":null,"cause":null,"stackTrace":[{"fileName":"NativeConstructorAccessorImpl.java","nativeMethod":true,"methodName":"newInstance0","className":"sun.reflect.NativeConstructorAccessorImpl","hCode":-917798116,"lineNumber":-2,"class":"java.lang.StackTraceElement"},{"fileName":"NativeConstructorAccessorImpl.java","nativeMethod":false,"methodName":"newInstance","className":"sun.reflect.NativeConstructorAccessorImpl","hCode":-1864225098,"lineNumber":62,"class":"java.lang.StackTraceElement"},{"fileName":"DelegatingConstructorAccessorImpl.java","nativeMethod":false,"methodName":"newInstance","className":"sun.reflect.DelegatingConstructorAccessorImpl","hCode":-2122307259,"lineNumber":45,"class":"java.lang.StackTraceElement"},{"fileName":"Constructor.java","nativeMethod":false,"methodName":"newInstance","className":"java.lang.reflect.Constructor","hCode":-1319859919,"lineNumber":423,"class":"java.lang.StackTraceElement"},{"fileName":"ErrorHandler.java","nativeMethod":false,"methodName":"createThrowable","className":"org.openqa.selenium.remote.ErrorHandler","hCode":514255497,"lineNumber":215,"class":"java.lang.StackTraceElement"},{"fileName":"ErrorHandler.java","nativeMethod":false,"methodName":"throwIfResponseFailed","className":"org.openqa.selenium.remote.ErrorHandler","hCode":1091067094,"lineNumber":167,"class":"java.lang.StackTraceElement"},{"fileName":"JsonWireProtocolResponse.java","nativeMethod":false,"methodName":"lambda$new$0","className":"org.openqa.selenium.remote.JsonWireProtocolResponse","hCode":-1663764462,"lineNumber":53,"class":"java.lang.StackTraceElement"},{"fileName":"JsonWireProtocolResponse.java","nativeMethod":false,"methodName":"lambda$getResponseFunction$2","className":"org.openqa.selenium.remote.JsonWireProtocolResponse","hCode":985207625,"lineNumber":91,"class":"java.lang.StackTraceElement"},{"fileName":"ProtocolHandshake.java","nativeMethod":false,"methodName":"lambda$createSession$22","className":"org.openqa.selenium.remote.ProtocolHandshake","hCode":933529798,"lineNumber":365,"class":"java.lang.StackTraceElement"},{"fileName":"ReferencePipeline.java","nativeMethod":false,"methodName":"accept","className":"java.util.stream.ReferencePipeline$3$1","hCode":-1838936501,"lineNumber":193,"class":"java.lang.StackTraceElement"},{"fileName":"Spliterators.java","nativeMethod":false,"methodName":"tryAdvance","className":"java.util.Spliterators$ArraySpliterator","hCode":243767033,"lineNumber":958,"class":"java.lang.StackTraceElement"},{"fileName":"ReferencePipeline.java","nativeMethod":false,"methodName":"forEachWithCancel","className":"java.util.stream.ReferencePipeline","hCode":-1532128058,"lineNumber":126,"class":"java.lang.StackTraceElement"},{"fileName":"AbstractPipeline.java","nativeMethod":false,"methodName":"copyIntoWithCancel","className":"java.util.stream.AbstractPipeline","hCode":-725863351,"lineNumber":498,"class":"java.lang.StackTraceElement"},{"fileName":"AbstractPipeline.java","nativeMethod":false,"methodName":"copyInto","className":"java.util.stream.AbstractPipeline","hCode":1635055484,"lineNumber":485,"class":"java.lang.StackTraceElement"},{"fileName":"AbstractPipeline.java","nativeMethod":false,"methodName":"wrapAndCopyInto","className":"java.util.stream.AbstractPipeline","hCode":-1370696581,"lineNumber":471,"class":"java.lang.StackTraceElement"},{"fileName":"FindOps.java","nativeMethod":false,"methodName":"evaluateSequential","className":"java.util.stream.FindOps$FindOp","hCode":380471590,"lineNumber":152,"class":"java.lang.StackTraceElement"},{"fileName":"AbstractPipeline.java","nativeMethod":false,"methodName":"evaluate","className":"java.util.stream.AbstractPipeline","hCode":-597548379,"lineNumber":234,"class":"java.lang.StackTraceElement"},{"fileName":"ReferencePipeline.java","nativeMethod":false,"methodName":"findFirst","className":"java.util.stream.ReferencePipeline","hCode":-605640091,"lineNumber":464,"class":"java.lang.StackTraceElement"},{"fileName":"ProtocolHandshake.java","nativeMethod":false,"methodName":"createSession","className":"org.openqa.selenium.remote.ProtocolHandshake","hCode":-960127588,"lineNumber":368,"class":"java.lang.StackTraceElement"},{"fileName":"ProtocolHandshake.java","nativeMethod":false,"methodName":"createSession","className":"org.openqa.selenium.remote.ProtocolHandshake","hCode":-960127797,"lineNumber":159,"class":"java.lang.StackTraceElement"},{"fileName":"HttpCommandExecutor.java","nativeMethod":false,"methodName":"execute","className":"org.openqa.selenium.remote.HttpCommandExecutor","hCode":-1852790187,"lineNumber":142,"class":"java.lang.StackTraceElement"},{"fileName":"DriverCommandExecutor.java","nativeMethod":false,"methodName":"execute","className":"org.openqa.selenium.remote.service.DriverCommandExecutor","hCode":-633333582,"lineNumber":82,"class":"java.lang.StackTraceElement"},{"fileName":"RemoteWebDriver.java","nativeMethod":false,"methodName":"execute","className":"org.openqa.selenium.remote.RemoteWebDriver","hCode":1767885124,"lineNumber":637,"class":"java.lang.StackTraceElement"},{"fileName":"RemoteWebDriver.java","nativeMethod":false,"methodName":"startSession","className":"org.openqa.selenium.remote.RemoteWebDriver","hCode":-1182648224,"lineNumber":250,"class":"java.lang.StackTraceElement"},{"fileName":"RemoteWebDriver.java","nativeMethod":false,"methodName":"startSession","className":"org.openqa.selenium.remote.RemoteWebDriver","hCode":-1182648238,"lineNumber":236,"class":"java.lang.StackTraceElement"},{"fileName":"RemoteWebDriver.java","nativeMethod":false,"methodName":"<init>","className":"org.openqa.selenium.remote.RemoteWebDriver","hCode":2001602317,"lineNumber":137,"class":"java.lang.StackTraceElement"},{"fileName":"ChromeDriver.java","nativeMethod":false,"methodName":"<init>","className":"org.openqa.selenium.chrome.ChromeDriver","hCode":988972012,"lineNumber":184,"class":"java.lang.StackTraceElement"},{"fileName":"ChromeDriver.java","nativeMethod":false,"methodName":"<init>","className":"org.openqa.selenium.chrome.ChromeDriver","hCode":988971976,"lineNumber":148,"class":"java.lang.StackTraceElement"},{"fileName":"NativeConstructorAccessorImpl.java","nativeMethod":true,"methodName":"newInstance0","className":"sun.reflect.NativeConstructorAccessorImpl","hCode":-917798116,"lineNumber":-2,"class":"java.lang.StackTraceElement"},{"fileName":"NativeConstructorAccessorImpl.java","nativeMethod":false,"methodName":"newInstance","className":"sun.reflect.NativeConstructorAccessorImpl","hCode":-1864225098,"lineNumber":62,"class":"java.lang.StackTraceElement"},{"fileName":"DelegatingConstructorAccessorImpl.java","nativeMethod":false,"methodName":"newInstance","className":"sun.reflect.DelegatingConstructorAccessorImpl","hCode":-2122307259,"lineNumber":45,"class":"java.lang.StackTraceElement"},{"fileName":"Constructor.java","nativeMethod":false,"methodName":"newInstance","className":"java.lang.reflect.Constructor","hCode":-1319859919,"lineNumber":423,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultDriverProvider.java","nativeMethod":false,"methodName":"callConstructor","className":"org.openqa.selenium.remote.server.DefaultDriverProvider","hCode":-2096353669,"lineNumber":102,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultDriverProvider.java","nativeMethod":false,"methodName":"newInstance","className":"org.openqa.selenium.remote.server.DefaultDriverProvider","hCode":1201680334,"lineNumber":96,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultDriverFactory.java","nativeMethod":false,"methodName":"newInstance","className":"org.openqa.selenium.remote.server.DefaultDriverFactory","hCode":-1365021416,"lineNumber":60,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultSession.java","nativeMethod":false,"methodName":"call","className":"org.openqa.selenium.remote.server.DefaultSession$BrowserCreator","hCode":2056168471,"lineNumber":222,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultSession.java","nativeMethod":false,"methodName":"call","className":"org.openqa.selenium.remote.server.DefaultSession$BrowserCreator","hCode":2056168458,"lineNumber":209,"class":"java.lang.StackTraceElement"},{"fileName":"FutureTask.java","nativeMethod":false,"methodName":"run","className":"java.util.concurrent.FutureTask","hCode":-820313554,"lineNumber":266,"class":"java.lang.StackTraceElement"},{"fileName":"DefaultSession.java","nativeMethod":false,"methodName":"run","className":"org.openqa.selenium.remote.server.DefaultSession$1","hCode":-255143467,"lineNumber":176,"class":"java.lang.StackTraceElement"},{"fileName":"ThreadPoolExecutor.java","nativeMethod":false,"methodName":"runWorker","className":"java.util.concurrent.ThreadPoolExecutor","hCode":-1285150177,"lineNumber":1149,"class":"java.lang.StackTraceElement"},{"fileName":"ThreadPoolExecutor.java","nativeMethod":false,"methodName":"run","className":"java.util.concurrent.ThreadPoolExecutor$Worker","hCode":-242190638,"lineNumber":624,"class":"java.lang.StackTraceElement"},{"fileName":"Thread.java","nativeMethod":false,"methodName":"run","className":"java.lang.Thread","hCode":-82946317,"lineNumber":748,"class":"java.lang.StackTraceElement"}],"suppressed":[],"message":"unknown error: Chrome failed to start: exited normally\n  (Driver info: chromedriver=2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8),platform=Linux 4.13.5-coreos-r2 x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 60.02 seconds\nBuild info: version: '3.4.0', revision: 'unknown', time: 'unknown'\nSystem info: host: '32a03e8d55b7', ip: '172.17.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '4.13.5-coreos-r2', java.version: '1.8.0_131'\nDriver info: driver.version: ChromeDriver","hCode":1789738219,"class":"org.openqa.selenium.WebDriverException","buildInformation":{"buildRevision":"unknown","buildTime":"unknown","releaseLabel":"3.4.0","hCode":276619488,"class":"org.openqa.selenium.internal.BuildInfo"},"screen":null},"class":"org.openqa.selenium.remote.Response","status":13}

Unable to access the localhost GitLab app from within the Docker image

We are unable to access http://127.0.0.1:53903/namespace1/project1/issues/1 when we are inside the GitLab Selenium proxy Docker image because it refers to a different localhost.

Related to

Failing to connect to GitLab Selenium proxy, Connection refused

Ex. https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/71731180

Running into Failed to connect to localhost port 4545: Connection refused when trying to connect to the GitLab Selenium proxy after starting it up in the existing GitLab CE pipelines.

$ nohup npm run gitlab-selenium-server &
$ curl http://localhost:4545/ || true
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 4545: Connection refused

It works fine in another project 🤷, MadLittleMods/playing-with-selenium-on-gitlab-ci!1 (merged)

The problem ended up being because gitlab-selenium-server was installed as a npm devDependency and we use yarn install --production which only installs dependency. I figured this out after piping the output to a file (via mkdir -p selenium/ && nohup npm run gitlab-selenium-server 2>&1 | tee selenium/gitlab-selenium-server-command-output.txt &) and adding some sleep 1 to let things play out, https://gitlab-org.gitlab.io/-/gitlab-ce/-/jobs/72705428/artifacts/selenium/gitlab-selenium-server-command-output.txt. I fixed this by moving gitlab-selenium-server to dependencies.

I also added a command to wait for the server to come up and be available, while ! curl --output /dev/null --silent --head --fail http://localhost:4545/ ; do printf '.' && sleep 1 ; doneA

Does this MR meet the acceptance criteria?

What are the relevant issue numbers?

Edited by Eric Eastwood

Merge request reports