Skip to content

Improve rake invalid feature flag signature diversity ratio

Pranshu Sharmarequested to merge
prsharma-rake_invalid_feature_flag into master

What does this MR do and why?

Improves log normalization by adding support for additional dynamic values that helps reduce the failure signature diversity score for rake_invalid_feature_flag.

Changes

  1. Better warnings handling - system now recognizes patterns for warning fragments, normalizes them consistently
  2. Enhanced testing Expected impact & dry-runs
  • Before (Master branch) - Diversity ratio: 49.2%

Download the CSV file from https://app.snowflake.com/ys68254/gitlab/w33qYqpFqfDA

fca --csv <path to csv>
./bin/analyze_signatures results.csv --category rake_invalid_feature_flag
Click to view logs 
 ./bin/analyze_signatures results.csv --category rake_invalid_feature_flag
====================================================================================================
CI FAILURE SIGNATURE ANALYSIS
====================================================================================================
Data source: results.csv
Category filter: cng
Total records: 230
DETAILED ANALYSIS FOR: cng
----------------------------------------------------------------------------------------------------
Total failures: 230
Unique signatures: 124
Diversity ratio: 49.2%

  • After (prsharma-rake_invalid_feature_flag branch) - Diversity score: 37.8%

    With the new normalization rules applied, the diversity score for rake_invalid_feature_flag is significantly reduced from 49.2% to 37.8%.

Click to view logs 
➜    triage-ops git:(prsharma-rake_invalid_feature_flag) ✗ ./bin/analyze_signatures results.csv --category rake_invalid_feature_flag
====================================================================================================
CI FAILURE SIGNATURE ANALYSIS
====================================================================================================
Data source: results.csv
Category filter: rake_invalid_feature_flag
Total records: 230
DETAILED ANALYSIS FOR: rake_invalid_feature_flag
----------------------------------------------------------------------------------------------------
Total failures: 230
Unique signatures: 87
Diversity ratio: 37.8%

Related Issue

gitlab-org/quality/analytics/team#297

Edited by Pranshu Sharma

Merge request reports