feat: [experimental] AI supported false positive analysis for secret detection

What does this merge request do and why?

DEMO (Log in with GitLab Unfiltered to view): https://youtu.be/K1M00G6fkik

Prompt tuning pending in a follow-up. This is still experimental

description generated by duo not mad if you tl;dr

This merge request adds a new AI-powered security feature for detecting false positives in secret scanning results. The system introduces several specialized AI agents that work together to analyze code repositories when secrets (like passwords or API keys) are detected, determine whether these findings are real security threats or harmless test data, and automatically report the results back to GitLab.

The feature includes five new AI agents:

• One agent validates secret vulnerabilities
• Another agent reads source files
• One agent extracts specific code lines
• One agent generates comprehensive security reports
• A final agent posts analysis results to GitLab

Each agent uses Claude AI models and has detailed instructions for analyzing code context, evaluating whether detected secrets are genuine risks or false alarms (like test data or placeholder values), and providing remediation guidance when real secrets are found. The system aims to reduce manual security review work by automatically distinguishing between actual leaked credentials that need immediate attention and harmless code patterns that secret detection tools often flag incorrectly.- Refs: This merge request adds a new AI-powered security feature for detecting false positives in secret scanning results. The system introduces several specialized AI agents that work together to analyze code repositories when secrets (like passwords or API keys) are detected, determine whether these findings are real security threats or harmless test data, and automatically report the results back to GitLab.

The feature includes five new AI agents: one validates secret vulnerabilities, another reads source files, one extracts specific code lines, one generates comprehensive security reports, and a final agent posts analysis results to GitLab. Each agent uses Claude AI models and has detailed instructions for analyzing code context, evaluating whether detected secrets are genuine risks or false alarms (like test data or placeholder values), and providing remediation guidance when real secrets are found.

The system aims to reduce manual security review work by automatically distinguishing between actual leaked credentials that need immediate attention and harmless code patterns that secret detection tools often flag incorrectly. https://gitlab.com/gitlab-org/gitlab/-/issues/577058

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

0. Check out the branch for this MR
1. Have runners setup (prepare yourself for potential pain)
2. Run the following to kick off the workflow:

curl -X POST 'http://0.0.0.0:3000/api/v4/ai/duo_workflows/workflows' \
  --header 'Authorization: Bearer $YOUR_TOKEN' \
  --header 'Content-Type: application/json' \
  --data "{
    \"project_id\": \"$PROJECT_ID\",
    \"agent_privileges\": [1, 2, 3, 4, 5],
    \"goal\": \"$GOAL\",
    \"start_workflow\": true,
    \"workflow_definition\": \"secrets_fp_detection/experimental\",
    \"environment\": \"web\",
    \"source_branch\": \"master\"
}"

3. Observe CI jobs output for workflow reply.

Merge request checklist

• Tests added for new functionality. If not, please raise an issue to follow up.
• Documentation added/updated, if needed.
• If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.