Skip to content

Remove product_stage from metrics 10: plan 1/3

Michał Wielich requested to merge michold-rm-metrics-stage-10-plan into master

What does this MR do and why?

Related to #440305 (closed)

We are removing product_stage and product_section from our metrics. As we already pushed changes that make sure that we're not using these attributes in our codebase - both in Gitlab and in other applications [version app, metrics dictionary] - we should be good to remove them from the actual metrics.

This MR is one in a series of auto-generated changes. The changes have been handled like this because of the mass of files that we need to change [we have over 2900 metrics with product_stage]. The script used to generate branches looks as follows:

script 
defs = {}
Gitlab::Usage::MetricDefinition.all.each do |definition|
  stage = definition.attributes[:product_stage].to_s
  defs[stage] ||= []
  defs[stage] << definition.path
end


original_limit = 200.freeze
mr_files = []
mr_file_index = -1
used_limit = 0

defs.sort_by{ |k,v| v.count }.reverse.each do |original_stage, paths|
  paths_count = paths.count
  chunks_count = 1
  chunk_index = 1
  if (original_limit - used_limit) < paths_count
    mr_file_index += 1
    used_limit = 0
  end

  if paths_count > original_limit
    chunks_count = paths_count / original_limit + 1
    limit = paths_count.fdiv(chunks_count).ceil
  else
    limit = original_limit
  end

  stage = original_stage
  stage = "#{stage} #{chunk_index}/#{chunks_count}" if chunks_count > 1
  paths.each do |path|
    if used_limit > limit
      mr_file_index += 1
      chunk_index += 1
      stage = "#{original_stage} #{chunk_index}/#{chunks_count}"
      used_limit = 0
    end

    mr_files[mr_file_index] ||= { stages: [], files: []}
    mr_files[mr_file_index][:stages] << stage unless mr_files[mr_file_index][:stages].include?(stage)
    mr_files[mr_file_index][:files] << path
    used_limit += 1
  end
end

# mr_files.map {|f| f.merge(files: f[:files].count) }

branch_prefix = "michold-rm-metrics-stage"

links = mr_files.each_with_index.map do |mr, index|
  `git checkout master`
  files = mr[:files]
  stage = mr[:stages].first

  branch_name="#{branch_prefix}-#{index + 1}-#{stage.split(" ").first}"
  `git checkout -b "#{branch_name}"`
  files.each do |path|
    `sed -i '' '/^product_stage/d; /^product_section/d' "#{path}"`
  end
  `git add --all`
  subject = "Remove product_stage from metrics #{index + 1}: #{stage}"
  `git commit --no-verify -m "#{subject}" -m "#{subject}"`
  output = Open3.popen3("git push --no-verify -f") { |stdin, stdout, stderr, wait_thr| stderr.read }
  link = output.split("\n")[2].sub('remote:', '').strip
  puts link
  link
end

links.each do |link|
  `open #{link}`
end

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Everything should work as previously: as long as the tests are succeeding, we should be safe to go.

Merge request reports