Investigate 500 errors in close succession when serving code-gecko ( google model)
We have seen fairly consistent stream of 500 errors ( ~5 hour) and need to investigate this. It is occasionnaly trigger our Error SLO.
Note that this issue is just about the 500
responses, not the 429
responses - this is being tracked in https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/24033.
UPD #1
Google returns a 500 error due to the prefix/suffix passed.
Examples - https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/24038#note_1449919345
Created a support case since this error happens on the Google side - https://console.cloud.google.com/support/cases/detail/v2/45677477?project=unreview-poc-390200e5
Edited by Alexander Chueshev