Feedback issue: Manage Secret Detection False Positives with AI (GA)

Welcome to the GA for Manage Secret Detection False Positives with AI!

The purpose of this feedback issue is to collect your experiences with the AI-powered false positive detection feature for secret detection vulnerabilities. Our goal is to understand how this feature is helping (or hindering) your security workflows, identify bugs and improvement areas, and prioritize enhancements based on real usage.

What is the Secret Detection False Positive Detection Feature?

The Secret Detection False Positive Detection feature is a GitLab Duo-powered capability that helps security teams identify and manage false positives in secret detection findings. It analyzes detected secrets and provides intelligent recommendations on which ones might be false positives (test credentials, example values, dummy tokens), enabling faster triage and more efficient vulnerability management.

Current GA capabilities (19.1)

Current GA capabilities

✅ What the feature CAN do:

False Positive Analysis

  • Analyze secret detection findings for potential false positives
  • Provide AI-powered recommendations on false positive likelihood
  • Display false positive information on vulnerability details
  • Show false positive badges in the vulnerability report
  • Identify test credentials, example values, and dummy tokens

Reporting & Visibility

  • Export false positive information in vulnerability report exports
  • View false positive metrics and trends
  • Monitor false positive detection workflow

⚠️ GA Limitations:

  • Limited to Ultimate tier with Duo add-on subscription
  • False positive detection runs on the default branch only
  • Feature is disabled by default and must be explicitly enabled

Feedback we're especially interested in

  1. Accuracy: Does the AI correctly identify false positives in secret detection?
  2. Usefulness: Does the feature save you time in secret triage?
  3. User Experience: How intuitive is the interface for managing false positives?
  4. Integration: How well does this fit into your existing security workflows?
  5. Missing capabilities: What false positive management tasks can't you accomplish?
  6. Performance: Are there any performance issues with the feature?
  7. Recommendations Quality: Are the AI recommendations helpful and trustworthy?
  8. Secret Types: Which secret types does the feature work well with? Which need improvement?

How to give feedback

  1. Check existing feedback: Review threads below to see if your issue is already reported. Add a 👍 or comment to show support.
  2. Start a new thread: Use a descriptive title like "False positive detection misses AWS keys" or "UI is confusing for dismissing multiple secrets"
  3. Include context:
    • What you were trying to do
    • The response or behavior you received
    • What you expected vs. what happened
    • URLs or screenshots (sanitized as needed)
    • Vulnerability IDs or project information
  4. Rate the response: On a scale of 1-5, how useful was it?
Example feedback format
  • Title: AI incorrectly identifies test token as false positive
  • Context: Analyzed secret detection findings in my project
  • What happened: Feature marked a test API token as likely false positive
  • Expected: Should correctly identify actual secrets vs. false positives
  • Usefulness: 2/5 - Had to manually review and override the recommendation
  • Screenshots: [If applicable]

What you can expect from us

  1. We will read all feedback during the GA period
  2. We will prioritize fixes for GA based on feedback patterns
  3. We will create issues for reproducible problems
  4. We may reach out for clarification on complex security issues

Known GA Issues

🛡️ 🤖 🔍 Thank you for helping us make the Secret Detection False Positive Detection feature an indispensable part of your security workflow! Your feedback during this GA period is crucial for delivering a GA release that truly transforms vulnerability management through AI-powered automation.

Edited by Nate Rosandich