Skip to content

Serialize json.extra.raw_response to avoid field explosion in logs

Igor requested to merge json-extra-raw-response into master

What does this MR do?

We're working on reducing the number of fields being logged as part of https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10173.

Previous MRs in this series:

In this case we're indexing a lot of fields under json.extra.raw_response. By serializing that value, we'll save on 134 fields which account for ~25% of the current schema.

The consequence is that we won't be able to perform searches on individual values. We shouldn't need that though. The main use case is to find a specific error and then look up the full response that occurred.

Screenshots

Here's a flamegraph:

fields.svg

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Mayra Cabrera

Merge request reports

Loading