API: Allow to configure `TraceForceSendInterval` in a similar way as `X-GitLab-Trace-Update-Interval`
Currently Runner does:
- Try to send trace in a predefined intervals configurable by
X-GitLab-Trace-Update-Interval
- If no trace is generated for longer period a additional request is generated
/api/:version/jobs/:id
withstatus: running
to PING GitLab - We can configure trace sending interval via
X-GitLab-Trace-Update-Interval
- We lack ability to configure
TraceForceSendInterval
Proposal
- Add
X-GitLab-Trace-Ping-Interval
to Rails that is a value higher thanX-GitLab-Trace-Update-Interval
- Likely a default should be 5 mins (it is enough)
that would configure
TraceForceSendInterval
- Extend Runner to parse header in a similar way as
X-GitLab-Trace-Update-Interval
and apply this asTraceForceSendInterval
- Update the
common.MaxUpdateInterval
to be higher than3 * time.Minute
, likely to be15 * time.Minute
(for future proofing) - Define the
common.MaxTraceForceSendInterval
to be a value of at least30 * time.Minute
(for future proofing)
Changes
- Extend Rails to send a new header in a similar way to
X-GitLab-Trace-Ping-Interval
- Extend Runner to receive
X-GitLab-Trace-Ping-Interval
and apply forTraceForceSendInterval
in a similar way as forUpdateInterval
Related
- #324371 (comment 536524306): discussion on how this is implemented today
- !56743 (merged): MR reducing frequency of updates
- #324372 (comment 535047226): investigation on frequency of PING
Outcome
We expect a significant reduction of requests that perform PING (status: running
) as described in #324372 (comment 535047226). Based on interval, we might be looking at:
- from: running: 65% (39,117,530 hits, filter used:
json.meta.caller_id.keyword: "/api/:version/jobs/:id" and json.params.key.keyword: "state" and json.params.value.keyword: "running"
) - to: running: 3,911,7530 (reducing amount of requests for this state by 90% for 7 day interval) for a value of that is for 5 minutes if all runners would be updated
- effective: I would assume that we would reduce this number by around 50%, since not all runners would be updated. However, since we control majority of builds processed this will have a significant effect very early when this change is made.
Edited by Kamil Trzciński