Skip to content
GitLab Next
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • T team-tasks
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Issues 421
    • Issues 421
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Insights
    • Issue
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Create a new issue
  • Issue Boards
Collapse sidebar
  • GitLab.org
  • Quality Department
  • team-tasks
  • Issues
  • #45
Closed
Open
Created Sep 15, 2018 by Mek Stittri@meksOwner

Crossbrowser Testing - Overall Plan

Overview

We need to close out our test coverage gap for cross-browser and mobile browsers.

We do not want to run all of our tests in a 3rd party service. The majority of our test will be handled via our internal grid. We aim to have ~20% of our test automation bandwidth be tested on all browsers and on real devices if mobile browsers

We already have browserstack (maintained by the FE team) But this is a small account which also does not support SAML configuration.

Requirements

  • Infrastructure

    • Handle all the flavor of browsers with many screen sizes
    • Handle mobile browser layouts
    • Handle the majority of mobile browsers that run on real devices.
    • Handle SSO auth for engineering
  • Test coverage

    • At first we will run our existing smoke tests.
    • We will need to determine what are the browsers that we will support.
      • This can be a mix of industry numbers e.g. via http://gs.statcounter.com/browser-market-share
      • This also should be data driven taken from CS and also any in app analytics that we log.
    • We will add additional E2E visual validation tests in addition to the functional smoke tests.

Comparison

BrowserStack SauceLabs

From a quick comparison it appears that the Saucelabs Enterprise plan seems to have more features.

Plan

Phase 1 - Initial gitlab-org&609

  • Setup Saucelabs infrastructure with an enterprise plan that supports SSO. Saucelabs also supports mobile tests our of the box for the desktop plan.
  • Framework support, we will need to ensure that the test runner can kick off test in Saucelabs. There has to be arguments that can specify run environments.
    • Flag to specify which env
    • Tag to enable cross-browser testing suite
  • Coverage, determine first MVC of optimal browser coverage

Phase 2 - MVC of basic test running

  • Identify cross-browser smoke test that needs to be run
    • Coordinate with CS and FE team as necessary
  • Configure cross-browser test run for the current release process on staging
    • Fix tests as necessary, page object customization maybe needed.

Phase 3 - implementation at scale across engineering

  • Integrate with Review Apps CI test pipeline
  • Configure SAML SSO for engineering login.
  • Address test coverage for dev stage, add more tests for cross-browser suite.

Phase 4 - Mobile coverage

  • Page objects for mobile
  • Functional tests for mobile
Edited Mar 27, 2021 by Mek Stittri
Assignee
Assign to
Time tracking