Proposal to improve build logs
Problem
- Service logging is now supported, but comes with the caveat that it can completely break the masking of secrets, token and sensitive URL parameters.
- No timestamp support
- We hoped for a quick win with sections (
FF_SCRIPT_SECTIONS
), but this is broken (scripts easily break the format, rendering it useless)
- We hoped for a quick win with sections (
- No semantic markup of logs
- We only support ANSI colour codes, and use them even to mark up print from the executor. This falls behind the competition.
Similar to service logging, the introduction of timestamps with the current state of logging would also break masking. This is due to all log streams (Runner/executor messages, job execution and service logs) all being merged prior to masking. A similar problem can exist for stdout/stderr streams, which is also merged early.
Proposal
- Redirect all logging through the
BuildLogger
(at the moment, bothBuildLogger
and direct access toTrace
is used throughout code-base) -
BuildLogger
should be updated to support:- multiple streams: executor, workload and services
-
stdout
/stderr
writers per stream - handle masking (migrating away from masking at the
Trace
level) per stream - split data by lines, prefixing with stream information: timestamp, stdout/stderr, stream id (as per Semantic markup for job logs (gitlab#368058))
- move away from ANSI colours codes and use semantic markup (as per Semantic markup for job logs (gitlab#368058))
This should be done in several phases:
-
Refactor BuildLogger, separate stdout/stderr, executor/workload/service streams but have them merged at the
Trace
level.Trace
will still handle masking. (POC: !3983 (merged)) -
Move masking functionality to BuildLogger, per stream. This might require us to improve the masking algorithms, selectively enable masking only where it makes sense for some streams.
-
Add stream information per log line. Prior to this, users hopefully wouldn't notice a change in how logs perform/are rendered. This can be behind a flag to begin with.
Due to long log lines being split, the frontend will likely need to parse and re-merge these. The log format should make this easy to detect.
-
Add semantic markup to output (Semantic markup for job logs (gitlab#368058)) and stop solely relying on ANSI colour codes.