Copy job trace to external system for search indexing
Problem to solve
Following the changes in Gitlab CE merge 16702, build traces no longer appear on the filesystem in a location that is convenient to index with an external search agent, such as the Splunk log forwarder. We would like to propose a global setting that would allow us to either send the trace log as an HTTP POST (which would be compatible with Splunk HEC), or to pass the trace file to an external command when complete.
Further details
Users may want to search build trace files to determine when a given warning or error first appeared in the output from a build.
Users may want an automated action when a given pattern appears in the output from a build.
Proposal
Gitlab should provide a global setting for either an HTTP URL to which a build trace will be sent using HTTP POST, or a system command which should be run and given the build trace on standard input (or both of these things) when the file is complete.
What does success look like, and how can we measure that?
The trace file available via the web API (GET /projects/:id/jobs/:job_id/trace) should match the data given in the HTTP POST, or standard input to the command.