Skip to content

Draft: Capture remaining cross-join/cross-modification exceptions

What does this MR do and why?

Use scripts from !69037 (closed)

mkdir call_sites
cd call_sites
../scripts/api/download_pipeline_artifact.rb -p 278964 -a log/ci-cross-db-modification.log --pipeline_id <pipeline_id>

Another script

#!/usr/bin/env ruby

require 'json'
require 'yaml'
require 'set'

data = ARGF.read

data = JSON.parse(data)

cross_joins = Set.new()
cross_modifications = Set.new()
unmatched = Set.new()
data['test_suites'].each do |suite|
  next unless suite['failed_count'] > 0

  suite['test_cases'].each do |kase|
    next unless kase['status'] == 'failed'

    output = kase['system_output']
    if output.include?('CrossJoinAcrossUnsupportedTablesError') || output.include?('Unsupported cross-join')
      cross_joins << kase['file']
    elsif output.include?('PreventCrossDatabaseModification') || output.include?('Cross-database data modification')
      cross_modifications << kase['file']
    else
      unmatched << kase['file']
      #puts "#{suite['name']} #{kase['file']}"
    end
  end
end

cross_joins = cross_joins.to_a.sort
cross_modifications = cross_modifications.to_a.sort

File.open('joins-failed.yml', 'w') { |f| f.write(YAML.dump(cross_joins)[4...-1]) }
File.open('modifications-failed.yml', 'w') { |f| f.write(YAML.dump(cross_modifications)[4...-1]) }

puts "There were #{unmatched.count} unmatched"
puts unmatched.to_a.join("\n")
curl "https://gitlab.com/api/v4/projects/278964/pipelines/389986454/test_report" > test_report.json
./parse-test-report-for-cross-joins.rb test_report.json
cat joins-failed.yml
cat modifications-failed.yml

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Dylan Griffith

Merge request reports