Skip to content
GitLab
Next
    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    Projects Groups Topics Snippets
  • Register
  • Sign in
  • GitLab GitLab
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
    • Locked files
  • Issues 55.3k
    • Issues 55.3k
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1.6k
    • Merge requests 1.6k
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Terraform modules
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • GitLabGitLab
  • Issues
  • #340298
Closed
Open
Issue created Sep 07, 2021 by Matthias Käppler@mkaeppler2️⃣Maintainer

Test main branch against Ruby 3 in a scheduled pipeline

Recently we were able to achieve our first green(-ish) build for our Ruby 3 MR 🎉 : the only two job failures came from the upstream, and all downstream pipelines and QA were passing: https://gitlab.slack.com/archives/CGN8BUCKC/p1650463827523419, https://gitlab.com/gitlab-org/gitlab/-/pipelines/520562059.

The next step is to run Ruby 3 pipeline on a regular basis, ideally for every MR.

Plan

To support the deployment of Ruby 3 to GitLab.com, we eventually need to have a green build on the master branch. We think that building every MR against Ruby 3 on top of Ruby 2 is too costly (see estimation below), so instead we will implement a staged approach to testing:

  • Milestone 1: Developers and CI run on Ruby 2 by default, by every 2 hours we run a Ruby 3 pipeline against master. This pipeline is just informative and must not block deployments or disrupt other workflows. Errors should only be communicated to developers working on the Ruby 3 migration.
  • Milestone 2: If we find that these Ruby 3 builds are mostly stable, and we understand the kinds of regressions that occur, we can devise a workflow for developers (separate issue) for how to tackle such failures and extend the audience of these builds to every contributor. At this point it is expected that if a changeset breaks these builds, the commit authors tackle these (with of our help, if necessary).
  • Milestone 3: If we find that this workflow works well enough, and enough contributors have been exposed to Ruby 3 related builds and failures (i.e. confidence is established in keeping these green), we will swap the 2-hour schedule with Ruby 2. At this point, every MR will build against Ruby 3, and Ruby 2 is only tested against every 2 hours, and before deployments. We should consider to have developers swap over their default Ruby to Ruby 3 at this point.
  • Milestone 4: This would coincide with running Ruby 3 in production; we remove the 2-hourly Ruby 2 build and Ruby 3 becomes the new default.

Rough approximation of additional CI costs

Additional cost of running Ruby 2 jobs on 2-hourly scheduled pipelines:

  • We have 94 RSpec jobs
  • RSpec jobs average duration is approximately 25 minutes
  • Cost per hour of runner VM: $0.095
  • Total additional hours per pipeline: 25 * 94 / 60 = 39.1
  • Total additional cost per pipeline: 39.1 * $0.095 = $3.7
  • We have 12 2-hourly scheduled pipelines per day
  • Total additional cost per day: $3.7 * 12 = $44.4
  • Running these duplicated tests for a year (e.g. until 16.x is released) would result in an additional: $44.4 * 365 = $16,206.
Edited May 19, 2022 by Yannis Roussos
Assignee
Assign to
Time tracking