Claude 3.7 Sonnet Code Review Summary rollout plan
Overview
We should update the Code Review Summary feature to use the latest available models from Anthropic (Sonnet 3.7 as of this writing).
Related to #521034 (closed).
| Resource | Links |
|---|---|
| Model | https://www.anthropic.com/news/claude-3-7-sonnet |
| Epic or Issue | #521034 (closed) |
| Feature Flag Rollout Issue | #522987 (closed) |
| Status updates |
Rollout success criteria
Add a list of success criteria here
Dashboard References
This can be the acceptance rate or latency dashboards filtered to the new model. Add as many dashboards as is relevant.
Legal notes
Add legal notes here
Known issue list
List of issues identified throughout the evaluation, implementation, and rollout of the model.
Rollout
Timeline
Optional: add a short description here of the expected timeline.
| Date | Audience | Status |
|---|---|---|
| 2025-03-05 | Code Review team members and other stakeholeders | |
| 2025-03-06 | All GitLab team members | |
| 2025-03-07 | 100% of all users |
Feedback from GitLab team members
Add link to the internal feedback issue.
Persevere / Continue Criteria
- Latency remains within observed p50/90/95 ranges below
- Error rate remains within observed range below, or improves
- Nothing was raised as a blocker
Observed latency from May 17 to Aug 21
- p50: X
- p90: X
- p95: X
Observed error rate from July 4 to Aug 21
- X %
Pivot / Pause / Rollback Criteria
- Requests are not using the new model as expected
- Opposite of what's defined in the previous section
Mitigation and Rollback Plan
We will use a Feature Flag to control the rollout. If there are any concerns (see above), we will disable the feature flag, especially for external users, to investigate any potential issues.
Release Announcement
Add details here about where to make announcements when the model is ready for rollout to external users.