Investigation into DB issues related to ci_build_trace_chunks

Based on the conversation in the standup today, I thought it would be helpful to create an issue to capture the investigation work being done by the various folks looking into this and work towards a root cause and mitigation.

Related to this incident gitlab-com/gl-infra/production#4486 (closed)

Below is a list of folks supporting this initiative:

DRIs	Role
@darbyfrey, @sgoldstein	Escalation point to pull in Development people when needed
@marin	Leading technical RCA investigation
@cmiskell, @msmiley and @andrewn	SRE's supporting RCA investigation
@mbobin, @allison.browne, @furkanayhan, @drew	Development engineers ready to jump in if needed

This issue will be updated as more information is available.