Implement notifications for GitHub importer errors

Why are we doing this work

During the GitHub import incident with Figure.ai, we discovered that the system lacks monitoring for third-party API changes and new errors. The root cause analysis identified that we need to proactively check the GitHub Importer Kibana dashboard for new errors, rather than waiting for customer reports.

Click to expand

The incident revealed that:

  • NoMethodError on new_pos was caused by GitHub changing their PR metadata format (Oct 2025)
  • We had no mechanism to detect this API change automatically
  • The issue went undetected until a critical customer escalation
  • Multiple import failures occurred before we identified the root cause

By implementing proactive error notifications, we can:

  1. Detect new errors in the GitHub importer early
  2. Identify third-party API changes before they impact customers
  3. Reduce time-to-detection for import failures
  4. Enable faster response to critical issues

Relevant links

Non-functional requirements

  • Documentation: Add runbook for monitoring GitHub importer errors
  • Feature flag: n/a
  • Performance: Notifications should not impact performance or be too noisy to be actionable or cause them to be ignored
  • Testing:

Implementation plan

  1. Identify notification mechanism (Slack, email, PagerDuty, etc.)
  2. Define error thresholds for what constitutes a "new error"
  3. Implement error detection logic for GitHub importer logs
  4. Set up alerting for new exception types and API changes
  5. Create escalation procedures for critical errors
  6. Add documentation and runbooks

Verification steps

  • Verify notifications are sent when new errors occur
  • Verify notifications include sufficient context
  • Verify escalation procedures are followed
  • Test with synthetic errors
  • Verify no false positives from expected errors
Edited by Thiago Figueiró