Remove failure category/signature logic and Snowflake dual-write

Lohit Peesapatirequested to merge
lp-remove-failure-analysis-snowflake-dual-write into master

What

Removes the CiFailureAnalysis module, its InternalEvents::Client Snowflake push, and the PushFailureCategoryForJobs reactive processor from triage-ops.

Why

This logic is now fully superseded by quality-observer, which owns failure categorization, signature generation, and ClickHouse export directly from its own webhook receiver (Webhooks::FailureAnalysisJob → ClickHouse::Schemas::FailureAnalysisMetrics). Observer's implementation is a superset of what lived here (timeout post-processing, more normalization rules, configurable context windows, larger pattern set) and doesn't depend on triage-ops at all.

Since ClickHouse/Grafana are now the source of truth for CI failure analytics, the Snowflake dual-write path in triage-ops is dead weight.

Removed

  • lib/internal_events/client.rb + spec — Snowflake/Snowplow push client (single caller, deleted below)
  • triage/processor/push_failure_category_for_jobs.rb + spec — reactive processor, and its registration in triage/triage/handler.rb / spec/triage/handler_spec.rb
  • lib/ci_failure_analysis/ (entire module: categorization, signature generation, normalization, patterns) + specs
  • bin/failure_category_analyzer, bin/analyze_signatures — local CLI tools, superseded by Observer's own versions
  • doc/failure_categories.md — superseded by Observer's doc/failure_categories.md + doc/ci_failure_analysis/
  • failure-categories-reference-check CI job (.gitlab/ci/test.yml) and matching lefthook pre-commit hook — Observer has an equivalent rails failure_categories:check task/CI job
  • Stale .rubocop_todo.yml exclusion for the deleted spec

Not touched: triage/triage/pipeline_failure/ (master-broken incident system) — confirmed via code-graph query it has zero dependency on ci_failure_analysis, fully independent.

Verification

  • bundle exec rspec: 5076 examples, 0 new failures (8 pre-existing unrelated failures in spec/processor/runway_migration_test_spec.rb, confirmed present on master too)
  • bundle exec rubocop: 648 files, no offenses
  • Grep sweep confirms zero remaining references to CiFailureAnalysis, InternalEvents, PushFailureCategoryForJobs, failure_categories_reference

Related to gitlab-org/quality/analytics/team#352

Merge request reports