With the introduction of FF_TIMESTAMPS to runner, logs of a any runner contain extra metadata data in each log line that we can pass when we transform the log into json or html format.
Proposal
Modify our parser libraries to consider a possible line prefix which contains the timestamps.
lib/gitlab/ci/ansi2json.rb: Add the timestamp to the json data to contain a date.
lib/gitlab/ci/ansi2html.rb: Ignore the timestamp (but parse correctly) the log lines.
Once the json data has been extended we can proceed to display this data in our frontend #455582 (closed) when present.
Additional Note
FF_SCRIPT_SECTIONS also changes the output of logs, so we should make sure the we maintain compatibility with this flag as well.
In 16.11, we've added support for FF_TIMESTAMP, but the existing UI doesn't work too well with this. We cannot make logs the default until this issue is resolved. I think simply "skipping" the timestamp would be enough (for html and json).
@mrincon@pedropombeiro Do you know how much work would be here to have this done on the backend? We probably don't have time to make FF_TIMESTAMP the default, but I do wonder whether we'd be able to implement something that can skip over the timestamp format for 17.0.
Personally, for the ruby side, I wouldn't even extract them. Just skip over.
Wen should probably interpret the + append flag, which essentially means that this line needs to be joined to the last, and that the last log lines \n was not a literal (it didn't exist in the original log, but was added because the line was either too long or \r was present and we wanted to yield that line so that progress meters update in a timely manner).