feat: Update DCR prompt to include custom instructions
What does this merge request do and why?
- This MR updates the prompt for duo code review to pass custom instructions
-
{{custom_instructions_section}}
is the only change in the new prompt. Otherwise, it's identical to1.0.0
-
- As per the versioning guide, I've created new versions
2.0.0
given we're adding a new parameter with no default value - Rails counterpart (currently in draft mode and requires bunch of updates): Update DCR prompt to include custom instructions (gitlab-org/gitlab!192570 - merged)
- Relates to Design Duo Code Review prompt that accepts cust... (gitlab-org/gitlab#545136 - closed), Retrieve Duo Code Review custom instructions fr... (gitlab-org/gitlab#545339 - closed)
Testing on this prompt change (LangSmith)
Test Case 1 (simple; one-file test)
- Description: API controller endpoint without authentication - tests if custom instructions for API security are properly applied
- LangSmith link: https://smith.langchain.com/public/932bdb6c-f6ed-47b9-988e-b75ab7d426ab/r
-
Results:
-
✅ Custom instruction successfully applied - LLM identified missing authentication for API endpoint -
✅ Appropriate priority assignment - authentication/authorization issues marked as priority 3, rate limiting as priority 2 -
✅ Context-aware analysis - LLM used full file content to notice existingauthenticate!
at class level andfind_user
helper -
✅ Both standard and custom review criteria applied - standard security checks (authorization, error handling) plus custom instructions (authentication, rate limiting)
-
Test Case 2 (multi-file test)
- Description: API controller and service object - tests if custom instructions are only applied to files matching the glob pattern and not to others
- LangSmith link: https://smith.langchain.com/public/87196265-be3e-4794-b0bb-a7a312bfb113/r
-
Results:
-
✅ Selective application worked perfectly - Custom instructions were ONLY applied to the API controller, NOT to the service object -
✅ API Controller (app/controllers/api/v4/search_controller.rb
):- Received custom instruction comments about missing authentication:
This API endpoint is missing authentication and authorization checks. According to the custom review instructions, all API endpoints should have proper authentication using the 'authenticate!' method.
- Received custom instruction comments about rate limiting
- Both marked as priority 3 as per custom instructions
- Received custom instruction comments about missing authentication:
-
✅ Service Object (app/services/user_search_service.rb
):- Did NOT receive any custom instruction comments
- Only received standard code review comments (SQL injection vulnerability, pagination)
- SQL injection correctly identified as priority 3 (standard security issue)
-
✅ Glob pattern matching confirmed working - The patternapp/controllers/api/**/*.rb
correctly excluded the service file atapp/services/user_search_service.rb
-
✅ Standard review still applied to all files - Both files received appropriate standard security and performance feedback
-
How to set up and validate locally
- Go to http://0.0.0.0:5052/docs#/prompts/invoke_v1_prompts__prompt_id__post and run the following tests (make sure
2.0.0
prompt is being used by looking at the logs fromgdk tail gitlab-ai-gateway
): - Test 1: Empty custom instructions
{
"inputs": {
"mr_title": "Fix SQL injection vulnerability in user search endpoint",
"mr_description": "This MR addresses a critical security vulnerability in the user search functionality where raw user input was being directly interpolated into SQL queries. The fix implements proper parameterized queries using prepared statements.\n\n## Changes:\n- Replace string interpolation with parameterized queries in UserRepository\n- Add input validation for search terms\n- Update related unit tests\n\n## Testing:\n- Tested with various SQL injection payloads\n- All existing tests pass\n- Added new security-focused test cases",
"full_file_intro": "",
"custom_instructions_section": "",
"diff_lines": "<file_diff filename=\"src/repositories/UserRepository.java\">\n<line type=\"context\" old_line=\"45\" new_line=\"45\"> public List<User> searchUsers(String searchTerm) {</line>\n<line type=\"deleted\" old_line=\"46\" new_line=\"\"> String query = \"SELECT * FROM users WHERE name LIKE '%\" + searchTerm + \"%' OR email LIKE '%\" + searchTerm + \"%'\";</line>\n<line type=\"added\" old_line=\"\" new_line=\"46\"> String query = \"SELECT * FROM users WHERE name LIKE ? OR email LIKE ?\";</line>\n<line type=\"context\" old_line=\"47\" new_line=\"47\"> </line>\n<line type=\"deleted\" old_line=\"48\" new_line=\"\"> return jdbcTemplate.query(query, new UserRowMapper());</line>\n<line type=\"added\" old_line=\"\" new_line=\"48\"> String searchPattern = \"%\" + searchTerm + \"%\";</line>\n<line type=\"added\" old_line=\"\" new_line=\"49\"> return jdbcTemplate.query(query, new UserRowMapper(), searchPattern, searchPattern);</line>\n<line type=\"context\" old_line=\"49\" new_line=\"50\"> }</line>\n<chunk_header>@@ -78,6 +79,12 @@</chunk_header>\n<line type=\"context\" old_line=\"78\" new_line=\"79\"> public User findById(Long id) {</line>\n<line type=\"deleted\" old_line=\"79\" new_line=\"\"> String query = \"SELECT * FROM users WHERE id = \" + id;</line>\n<line type=\"added\" old_line=\"\" new_line=\"80\"> String query = \"SELECT * FROM users WHERE id = ?\";</line>\n<line type=\"context\" old_line=\"80\" new_line=\"81\"> try {</line>\n<line type=\"deleted\" old_line=\"81\" new_line=\"\"> return jdbcTemplate.queryForObject(query, new UserRowMapper());</line>\n<line type=\"added\" old_line=\"\" new_line=\"82\"> return jdbcTemplate.queryForObject(query, new UserRowMapper(), id);</line>\n<line type=\"context\" old_line=\"82\" new_line=\"83\"> } catch (EmptyResultDataAccessException e) {</line>\n<line type=\"context\" old_line=\"83\" new_line=\"84\"> return null;</line>\n<line type=\"context\" old_line=\"84\" new_line=\"85\"> }</line>\n</file_diff>\n<file_diff filename=\"src/controllers/UserController.java\">\n<line type=\"context\" old_line=\"32\" new_line=\"32\"> @GetMapping(\"/search\")</line>\n<line type=\"context\" old_line=\"33\" new_line=\"33\"> public ResponseEntity<List<User>> searchUsers(@RequestParam String term) {</line>\n<line type=\"added\" old_line=\"\" new_line=\"34\"> // Validate input length</line>\n<line type=\"added\" old_line=\"\" new_line=\"35\"> if (term.length() > 100) {</line>\n<line type=\"added\" old_line=\"\" new_line=\"36\"> return ResponseEntity.badRequest().build();</line>\n<line type=\"added\" old_line=\"\" new_line=\"37\"> }</line>\n<line type=\"context\" old_line=\"34\" new_line=\"38\"> List<User> users = userRepository.searchUsers(term);</line>\n<line type=\"context\" old_line=\"35\" new_line=\"39\"> return ResponseEntity.ok(users);</line>\n<line type=\"context\" old_line=\"36\" new_line=\"40\"> }</line>\n</file_diff>\n<file_diff filename=\"src/test/UserRepositoryTest.java\">\n<chunk_header>@@ -67,4 +67,18 @@</chunk_header>\n<line type=\"context\" old_line=\"67\" new_line=\"67\"> assertEquals(\"test@example.com\", user.getEmail());</line>\n<line type=\"context\" old_line=\"68\" new_line=\"68\"> }</line>\n<line type=\"added\" old_line=\"\" new_line=\"69\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"70\"> @Test</line>\n<line type=\"added\" old_line=\"\" new_line=\"71\"> public void testSearchUsersWithSQLInjectionAttempt() {</line>\n<line type=\"added\" old_line=\"\" new_line=\"72\"> // Test that SQL injection attempts are properly escaped</line>\n<line type=\"added\" old_line=\"\" new_line=\"73\"> String maliciousInput = \"'; DROP TABLE users; --\";</line>\n<line type=\"added\" old_line=\"\" new_line=\"74\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"75\"> List<User> results = userRepository.searchUsers(maliciousInput);</line>\n<line type=\"added\" old_line=\"\" new_line=\"76\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"77\"> // Should return empty list, not cause SQL error</line>\n<line type=\"added\" old_line=\"\" new_line=\"78\"> assertTrue(results.isEmpty());</line>\n<line type=\"added\" old_line=\"\" new_line=\"79\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"80\"> // Verify table still exists</line>\n<line type=\"added\" old_line=\"\" new_line=\"81\"> assertDoesNotThrow(() -> userRepository.findById(1L));</line>\n<line type=\"added\" old_line=\"\" new_line=\"82\"> }</line>\n<line type=\"context\" old_line=\"69\" new_line=\"83\">}</line>\n</file_diff>",
"full_content_section": ""
},
"prompt_version": "2.0.0",
"stream": false
}
- Test 2: Non-empty custom instructions
{
"inputs": {
"mr_title": "Implement Redis caching for product catalog API",
"mr_description": "This MR adds Redis caching to improve performance of our product catalog API endpoints. The implementation includes:\n\n## Changes:\n- Add Redis cache layer for product queries\n- Implement cache invalidation on product updates\n- Add cache TTL configuration\n- Update Docker compose to include Redis service\n\n## Performance improvements:\n- Product list endpoint: ~200ms → ~15ms\n- Product detail endpoint: ~150ms → ~10ms\n\n## Configuration:\n- Default TTL: 1 hour\n- Cache key pattern: `product:{id}` and `products:list:{page}:{size}`",
"full_file_intro": "",
"custom_instructions_section": "<custom_instructions>\nPlease pay special attention to:\n1. Cache invalidation logic - ensure all cache entries are properly invalidated when products are updated\n2. Error handling - Redis connection failures should not break the application\n3. Memory usage - verify that cache keys have appropriate TTLs\n4. Thread safety - ensure concurrent cache operations are handled correctly\n5. Configuration - check that all Redis settings are properly externalized\n\nNote: Our production Redis cluster has a 2GB memory limit, so please flag any potential memory issues.\n</custom_instructions>",
"diff_lines": "<file_diff filename=\"src/main/java/com/example/service/ProductService.java\">\n<line type=\"context\" old_line=\"15\" new_line=\"15\">@Service</line>\n<line type=\"context\" old_line=\"16\" new_line=\"16\">public class ProductService {</line>\n<line type=\"context\" old_line=\"17\" new_line=\"17\"> </line>\n<line type=\"context\" old_line=\"18\" new_line=\"18\"> private final ProductRepository productRepository;</line>\n<line type=\"added\" old_line=\"\" new_line=\"19\"> private final RedisTemplate<String, Object> redisTemplate;</line>\n<line type=\"added\" old_line=\"\" new_line=\"20\"> private static final String PRODUCT_KEY_PREFIX = \"product:\";</line>\n<line type=\"added\" old_line=\"\" new_line=\"21\"> private static final String PRODUCT_LIST_KEY_PREFIX = \"products:list:\";</line>\n<line type=\"context\" old_line=\"19\" new_line=\"22\"> </line>\n<chunk_header>@@ -25,8 +28,22 @@</chunk_header>\n<line type=\"context\" old_line=\"25\" new_line=\"28\"> public Product getProductById(Long id) {</line>\n<line type=\"added\" old_line=\"\" new_line=\"29\"> String cacheKey = PRODUCT_KEY_PREFIX + id;</line>\n<line type=\"added\" old_line=\"\" new_line=\"30\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"31\"> // Check cache first</line>\n<line type=\"added\" old_line=\"\" new_line=\"32\"> Product cachedProduct = (Product) redisTemplate.opsForValue().get(cacheKey);</line>\n<line type=\"added\" old_line=\"\" new_line=\"33\"> if (cachedProduct != null) {</line>\n<line type=\"added\" old_line=\"\" new_line=\"34\"> return cachedProduct;</line>\n<line type=\"added\" old_line=\"\" new_line=\"35\"> }</line>\n<line type=\"added\" old_line=\"\" new_line=\"36\"> </line>\n<line type=\"context\" old_line=\"26\" new_line=\"37\"> Product product = productRepository.findById(id)</line>\n<line type=\"context\" old_line=\"27\" new_line=\"38\"> .orElseThrow(() -> new ProductNotFoundException(\"Product not found: \" + id));</line>\n<line type=\"added\" old_line=\"\" new_line=\"39\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"40\"> // Cache the result</line>\n<line type=\"added\" old_line=\"\" new_line=\"41\"> redisTemplate.opsForValue().set(cacheKey, product, Duration.ofHours(1));</line>\n<line type=\"added\" old_line=\"\" new_line=\"42\"> </line>\n<line type=\"context\" old_line=\"28\" new_line=\"43\"> return product;</line>\n<line type=\"context\" old_line=\"29\" new_line=\"44\"> }</line>\n<chunk_header>@@ -35,6 +52,15 @@</chunk_header>\n<line type=\"context\" old_line=\"35\" new_line=\"52\"> public Product updateProduct(Long id, ProductUpdateDto updateDto) {</line>\n<line type=\"context\" old_line=\"36\" new_line=\"53\"> Product product = getProductById(id);</line>\n<line type=\"context\" old_line=\"37\" new_line=\"54\"> product.setName(updateDto.getName());</line>\n<line type=\"context\" old_line=\"38\" new_line=\"55\"> product.setPrice(updateDto.getPrice());</line>\n<line type=\"context\" old_line=\"39\" new_line=\"56\"> product.setUpdatedAt(LocalDateTime.now());</line>\n<line type=\"context\" old_line=\"40\" new_line=\"57\"> </line>\n<line type=\"deleted\" old_line=\"41\" new_line=\"\"> return productRepository.save(product);</line>\n<line type=\"added\" old_line=\"\" new_line=\"58\"> Product savedProduct = productRepository.save(product);</line>\n<line type=\"added\" old_line=\"\" new_line=\"59\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"60\"> // Invalidate cache</line>\n<line type=\"added\" old_line=\"\" new_line=\"61\"> redisTemplate.delete(PRODUCT_KEY_PREFIX + id);</line>\n<line type=\"added\" old_line=\"\" new_line=\"62\"> // Clear all list caches</line>\n<line type=\"added\" old_line=\"\" new_line=\"63\"> Set<String> keys = redisTemplate.keys(PRODUCT_LIST_KEY_PREFIX + \"*\");</line>\n<line type=\"added\" old_line=\"\" new_line=\"64\"> redisTemplate.delete(keys);</line>\n<line type=\"added\" old_line=\"\" new_line=\"65\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"66\"> return savedProduct;</line>\n<line type=\"context\" old_line=\"42\" new_line=\"67\"> }</line>\n</file_diff>\n<file_diff filename=\"src/main/java/com/example/config/RedisConfig.java\">\n<line type=\"added\" old_line=\"\" new_line=\"1\">package com.example.config;</line>\n<line type=\"added\" old_line=\"\" new_line=\"2\"></line>\n<line type=\"added\" old_line=\"\" new_line=\"3\">import org.springframework.beans.factory.annotation.Value;</line>\n<line type=\"added\" old_line=\"\" new_line=\"4\">import org.springframework.context.annotation.Bean;</line>\n<line type=\"added\" old_line=\"\" new_line=\"5\">import org.springframework.context.annotation.Configuration;</line>\n<line type=\"added\" old_line=\"\" new_line=\"6\">import org.springframework.data.redis.connection.RedisConnectionFactory;</line>\n<line type=\"added\" old_line=\"\" new_line=\"7\">import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;</line>\n<line type=\"added\" old_line=\"\" new_line=\"8\">import org.springframework.data.redis.core.RedisTemplate;</line>\n<line type=\"added\" old_line=\"\" new_line=\"9\"></line>\n<line type=\"added\" old_line=\"\" new_line=\"10\">@Configuration</line>\n<line type=\"added\" old_line=\"\" new_line=\"11\">public class RedisConfig {</line>\n<line type=\"added\" old_line=\"\" new_line=\"12\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"13\"> @Value(\"${spring.redis.host}\")</line>\n<line type=\"added\" old_line=\"\" new_line=\"14\"> private String redisHost;</line>\n<line type=\"added\" old_line=\"\" new_line=\"15\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"16\"> @Value(\"${spring.redis.port}\")</line>\n<line type=\"added\" old_line=\"\" new_line=\"17\"> private int redisPort;</line>\n<line type=\"added\" old_line=\"\" new_line=\"18\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"19\"> @Bean</line>\n<line type=\"added\" old_line=\"\" new_line=\"20\"> public RedisConnectionFactory redisConnectionFactory() {</line>\n<line type=\"added\" old_line=\"\" new_line=\"21\"> return new LettuceConnectionFactory(redisHost, redisPort);</line>\n<line type=\"added\" old_line=\"\" new_line=\"22\"> }</line>\n<line type=\"added\" old_line=\"\" new_line=\"23\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"24\"> @Bean</line>\n<line type=\"added\" old_line=\"\" new_line=\"25\"> public RedisTemplate<String, Object> redisTemplate() {</line>\n<line type=\"added\" old_line=\"\" new_line=\"26\"> RedisTemplate<String, Object> template = new RedisTemplate<>();</line>\n<line type=\"added\" old_line=\"\" new_line=\"27\"> template.setConnectionFactory(redisConnectionFactory());</line>\n<line type=\"added\" old_line=\"\" new_line=\"28\"> return template;</line>\n<line type=\"added\" old_line=\"\" new_line=\"29\"> }</line>\n<line type=\"added\" old_line=\"\" new_line=\"30\">}</line>\n</file_diff>\n<file_diff filename=\"src/main/resources/application.yml\">\n<line type=\"context\" old_line=\"15\" new_line=\"15\">spring:</line>\n<line type=\"context\" old_line=\"16\" new_line=\"16\"> datasource:</line>\n<line type=\"context\" old_line=\"17\" new_line=\"17\"> url: ${DATABASE_URL:jdbc:postgresql://localhost:5432/products}</line>\n<line type=\"context\" old_line=\"18\" new_line=\"18\"> username: ${DATABASE_USER:postgres}</line>\n<line type=\"context\" old_line=\"19\" new_line=\"19\"> password: ${DATABASE_PASSWORD:postgres}</line>\n<line type=\"added\" old_line=\"\" new_line=\"20\"> redis:</line>\n<line type=\"added\" old_line=\"\" new_line=\"21\"> host: ${REDIS_HOST:localhost}</line>\n<line type=\"added\" old_line=\"\" new_line=\"22\"> port: ${REDIS_PORT:6379}</line>\n<line type=\"added\" old_line=\"\" new_line=\"23\"> timeout: 2000ms</line>\n<line type=\"added\" old_line=\"\" new_line=\"24\"> lettuce:</line>\n<line type=\"added\" old_line=\"\" new_line=\"25\"> pool:</line>\n<line type=\"added\" old_line=\"\" new_line=\"26\"> max-active: 8</line>\n<line type=\"added\" old_line=\"\" new_line=\"27\"> max-idle: 8</line>\n<line type=\"added\" old_line=\"\" new_line=\"28\"> min-idle: 0</line>\n</file_diff>\n<file_diff filename=\"docker-compose.yml\">\n<line type=\"context\" old_line=\"10\" new_line=\"10\">services:</line>\n<line type=\"context\" old_line=\"11\" new_line=\"11\"> db:</line>\n<line type=\"context\" old_line=\"12\" new_line=\"12\"> image: postgres:15</line>\n<line type=\"context\" old_line=\"13\" new_line=\"13\"> environment:</line>\n<line type=\"context\" old_line=\"14\" new_line=\"14\"> POSTGRES_DB: products</line>\n<line type=\"context\" old_line=\"15\" new_line=\"15\"> POSTGRES_USER: postgres</line>\n<line type=\"context\" old_line=\"16\" new_line=\"16\"> POSTGRES_PASSWORD: postgres</line>\n<line type=\"context\" old_line=\"17\" new_line=\"17\"> ports:</line>\n<line type=\"context\" old_line=\"18\" new_line=\"18\"> - \"5432:5432\"</line>\n<line type=\"added\" old_line=\"\" new_line=\"19\"> </line>\n<line type=\"added\" old_line=\"\" new_line=\"20\"> redis:</line>\n<line type=\"added\" old_line=\"\" new_line=\"21\"> image: redis:7-alpine</line>\n<line type=\"added\" old_line=\"\" new_line=\"22\"> ports:</line>\n<line type=\"added\" old_line=\"\" new_line=\"23\"> - \"6379:6379\"</line>\n<line type=\"added\" old_line=\"\" new_line=\"24\"> command: redis-server --maxmemory 2gb --maxmemory-policy allkeys-lru</line>\n</file_diff>",
"full_content_section": ""
},
"prompt_version": "2.0.0",
"stream": false
}
Screenshots:
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Kinshuk Singh