Draft: MR widget AI possible FP flag

What does this MR do and why?

This is draft prototype for SAST FP implemetation in merge request widget

Note that backend code here is "throw-away" just to give understanding how it could work

References

Screenshots or screen recordings

How to set up and validate locally

1. Ensure that you have gitlab runner set up in your GDK https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/runner.md
2. Copy the project to your GDK (replace gdk.test with your GDK domain and gitlab-duo/demo-sast with desired project path in your GDK)

cd /tmp # just to be nice
git clone git@gitlab.com:compliance-group-testing-and-demos/demos/test-sast.git
cd test-sast
git push --all ssh://git@gdk.test:2222/gitlab-duo/demo-sast.git -omerge_request.create
git checkout xtra-branch
git push ssh://git@gdk.test:2222/gitlab-duo/demo-sast.git HEAD -omerge_request.create

3. Verify that your project gitlab-duo/demo-sast was created and has 1 merge request
4. Wait for all pipelines to finish (you can verify this with Build > Pipelines
5. Run rails console
6. Enable feature flag by running

Feature.enable(:ai_experiment_sast_fp_detection)

8. Seed security finding flags (fp detection) by running script

srand(Time.now.to_i)

DUMMY_DESCRIPTIONS = [
  "This confidence score is based on automated analysis of the security finding.\nThe algorithm considers multiple factors including code patterns, context analysis,\nand historical data from similar vulnerabilities to determine accuracy likelihood.",
  
  "Confidence assessment derived from static code analysis engine.\nFactors include pattern matching accuracy, false positive rates,\ncode complexity analysis, and vulnerability signature strength.\nHigher scores indicate greater certainty in the finding's validity.",
  
  "Machine learning model evaluation of security finding reliability.\nThe score incorporates semantic analysis of the vulnerable code,\ncomparison with known vulnerability databases,\nand statistical analysis of similar patterns across codebases.\nThis metric helps prioritize remediation efforts.",
  
  "Automated confidence calculation based on multiple detection heuristics.\nIncludes analysis of code flow patterns, data taint analysis,\nand cross-reference validation with security knowledge bases.\nThe scoring algorithm has been trained on thousands of verified vulnerabilities\nto provide accurate confidence estimates.",
  
  "Statistical confidence derived from vulnerability detection algorithms.\nConsiders code context, execution paths, input validation patterns,\nand known exploit techniques to assess finding accuracy.\nThis score helps development teams focus on high-priority security issues\nwhile reducing time spent on potential false positives.",
  
  "Dynamic confidence assessment using hybrid analysis techniques.\nCombines static code analysis with behavioral pattern recognition,\nleveraging both rule-based detection and machine learning models.\nThe confidence metric reflects the probability of a true positive finding\nbased on comprehensive code evaluation and threat modeling.",
  
  "Confidence score generated through multi-layered security analysis.\nIncorporates vulnerability severity, exploitability assessment,\ncode quality metrics, and environmental context factors.\nThis comprehensive scoring helps security teams prioritize\nremediation activities based on both likelihood and impact potential.",
  
  "Algorithmic confidence evaluation of detected security vulnerability.\nUtilizes advanced pattern recognition and semantic code analysis\nto determine the reliability of the security finding.\nThe scoring model accounts for code complexity and potential attack vectors."
].freeze

findings = Security::Finding.all
total_count = findings.count

puts "Total findings: #{total_count}"
puts "Will update ALL findings"

processed = 0

findings.find_each(batch_size: 100) do |finding|
  confidence_score = (rand * 0.7 + 0.3).round(2)
  description = DUMMY_DESCRIPTIONS.sample
  current_data = finding.finding_data.dup
  
  current_data["latest_flag"] = { 
    "confidence_score" => confidence_score,
    "description" => description
  }
  
  finding.update_column(:finding_data, current_data)
  
  processed += 1
  puts "Updated finding #{finding.id} with confidence score: #{confidence_score}" if processed % 100 == 0
end

puts "Completed! Updated #{processed} of #{total_count} findings."

9. Verify functionality
   • Open merge request in cloned repo
   • See security report

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.