Draft: Capture remaining cross-join/cross-modification exceptions
What does this MR do and why?
Use scripts from !69037 (closed)
mkdir call_sites
cd call_sites
../scripts/api/download_pipeline_artifact.rb -p 278964 -a log/ci-cross-db-modification.log --pipeline_id <pipeline_id>
Another script
#!/usr/bin/env ruby
require 'json'
require 'yaml'
require 'set'
data = ARGF.read
data = JSON.parse(data)
cross_joins = Set.new()
cross_modifications = Set.new()
unmatched = Set.new()
data['test_suites'].each do |suite|
next unless suite['failed_count'] > 0
suite['test_cases'].each do |kase|
next unless kase['status'] == 'failed'
output = kase['system_output']
if output.include?('CrossJoinAcrossUnsupportedTablesError') || output.include?('Unsupported cross-join')
cross_joins << kase['file']
elsif output.include?('PreventCrossDatabaseModification') || output.include?('Cross-database data modification')
cross_modifications << kase['file']
else
unmatched << kase['file']
#puts "#{suite['name']} #{kase['file']}"
end
end
end
cross_joins = cross_joins.to_a.sort
cross_modifications = cross_modifications.to_a.sort
File.open('joins-failed.yml', 'w') { |f| f.write(YAML.dump(cross_joins)[4...-1]) }
File.open('modifications-failed.yml', 'w') { |f| f.write(YAML.dump(cross_modifications)[4...-1]) }
puts "There were #{unmatched.count} unmatched"
puts unmatched.to_a.join("\n")
curl "https://gitlab.com/api/v4/projects/278964/pipelines/389986454/test_report" > test_report.json
./parse-test-report-for-cross-joins.rb test_report.json
cat joins-failed.yml
cat modifications-failed.yml
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by Dylan Griffith