Explain this Vulnerability: Prompt / Response export tool for analysis

Why are we doing this work

This issue formally tracks the development of a tool that exports prompt/responses for continued analysis. The thread at #412538 (comment 1409485379) provides context.

The tool should use https://gitlab.com/gitlab-org/security-products/tests/webgoat.net as it's base project. This project contains 165 intentional SAST (via Semgrep) vulnerabilities

Sheet location

https://docs.google.com/spreadsheets/d/1KMN3_8g6SIwG9J-pS7S7kIzixdOnYVpoMp2qTLJy1Bw/edit#gid=1179892551

Sheet columns:

Automated

Export a CSV for the vulnerabilities in this project: https://gitlab.com/gitlab-org/govern/threat-insights-demos/personal-test-projects/webgoat.net

Capture

1. Id
2. Severity
3. Identifiers
4. Prompt
5. Language
6. code_bison
7. chat_bison
   • result of "safetyAttributes"=>{"blocked"=>**true**} to aid in Responsible AI moderation refinement
   • API response time
8. text_bison
9. anthropic
10. openai_completion
11. openai_chat

Results will be imported into the sheet at https://docs.google.com/spreadsheets/d/1KMN3_8g6SIwG9J-pS7S7kIzixdOnYVpoMp2qTLJy1Bw/edit#gid=689565258

Relevant links

Non-functional requirements

• Documentation:
• Feature flag:
• Performance:
• Testing:

Implementation plan

Verification steps