Add mutation to report ai catalog item
What does this MR do and why?
Issue - #578591 (closed)
This change adds a new reporting feature for AI catalog items in GitLab. Users can now report problematic AI catalog items (like agents or flow or third party flow) by selecting from predefined reasons such as security threats, spam, excessive resource usage, or other issues. When someone reports an item, they can provide additional details up to 1000 characters, and the system automatically sends an email notification to administrators. The feature includes proper permission checks to ensure only authorized users can report items, and it works with different types of AI catalog items. This helps maintain quality and safety in the AI catalog by allowing the community to flag inappropriate or harmful content.
References
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
- Turn on FF
global_ai_catalog - Add an abuse notification email by sending a PUT request to
api/v4/application/settings:
{
"abuse_notification_email": "test@example.com"
}
- Create agent/flow by executing below mutation.
mutation createAgent {
aiCatalogAgentCreate(
input: {
projectId: "gid://gitlab/Project/1000000",
name: "My name",
description: "My description",
release: true,
public: false,
systemPrompt: "My system prompt",
userPrompt: "My user prompt"
}
) {
errors
item {
id
name
description
latestVersion{
id
released
}
}
}
}
- Now report agent created from above mutation.
mutation report_ai_catalog_item {
aiCatalogItemReport(input: {
id: "gid://gitlab/Ai::Catalog::Item/<ITEM_ID_FROM_ABOVE_RESPONSE>",
reason: IMMEDIATE_SECURITY_THREAT
body: "Contains dangerous code, exploits, or harmful actions."
}
) {
clientMutationId
errors
}
}
SuccessFull Response:
{
"data": {
"aiCatalogItemReport": {
"clientMutationId": null
}
},
"correlationId": "01K9P710375W8WX8DK2H9C5P3Y"
}
If you don't have permission to report (e.g., an anonymous user), the failure response will be:
{
"errors": [
{
"graphQLErrors": [
{
"message": "The resource that you are attempting to access does not exist or you don't have permission to perform this action",
"locations": [
{
"line": 2,
"column": 3
}
],
"path": [
"aiCatalogItemReport"
]
}
],
"clientErrors": [],
"networkError": null,
"message": "The resource that you are attempting to access does not exist or you don't have permission to perform this action",
"stack": "Error\n at new ApolloError2 (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-UEUDGWSA.js?v=dc230448:3251:28)\n at http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-UEUDGWSA.js?v=dc230448:4764:21\n at Object.next (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1449:25)\n at notifySubscription (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1290:18)\n at onNotify (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1328:3)\n at SubscriptionObserver2.next (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1383:9)\n at http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1834:23\n at Array.forEach (<anonymous>)\n at iterateObserversSafely (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1833:23)\n at Object.next (http://gdk.test:3038/assets/vite/@fs/Users/jaydippansuriya/Documents/projects/gdk/gitlab/tmp/cache/vite/deps/chunk-VFJF4DZ3.js?v=dc230448:1961:15)"
}
]
}
- Open
http://gdk.test:3000/admin/sidekiqyou will see one job related to sending mail (mailerAi::CatalogItemAbuseReportMailer) - Once the above job executed, you will be able to see the sent mail at
<YOUR-GDK-URL>/rails/letter_opener - When Other reason is selected and black body is passed it should though an error like below.
mutation report_ai_catalog_item {
aiCatalogItemReport(input: {
id: "gid://gitlab/Ai::Catalog::Item/225",
reason: OTHER
body: ""
}
) {
clientMutationId
errors
}
}
Error Response:
{
"data": {
"aiCatalogItemReport": {
"clientMutationId": null,
"errors": [
"Additional details are required when reason is OTHER"
]
}
},
"correlationId": "edf53414-aad9-4c32-9e38-93ff047e0f58"
}
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #578591 (closed)