Restrict access to jobs logs when CI_DEBUG_TRACE=true is set via Runner
In GitLab 13.7 and later, we restricted the ability to see job logs to users with developer or higher permissions when debug logging is enabled(CI_DEBUG_TRACE=true) in YAML defined or GitLab UI defined variables.
However, users without developer or higher permissions are still able to access job logs if the variable is set to True directly in Runner or anywhere outside of GitLab and the ci config files.
On the original issue @grzesiek said:
Probably the most simple fix here is to check if
CI_DEBUG_TRACE
is set on the GitLab side, this might be the quick fix, but in order to fix that properly I think that the runner should send this information the it is a detailed trace.
@tmaczukin said:
In next step we can create another job metadata field like
trace_debug boolean
(not sure where to put it, sinceci_builds
is already too big...) that would be set totrue
on detection ofCI_DEBUG_TRACE
and the permission would act depending on this field.And as the third step, we could extend the Runner API for the trace update or trace patch, to send information about
trace_debug=(true|false)
depending on what's detected on the Runner side (since at the end the decision about enabling debug trace is happening in Runner). This could then cover all possible cases whereCI_DEBUG_TRACE
can be set
Proposal
Hide debug logs from developers or above even if CI_DEBUG_TRACE=true
is set only on the runner side by passing back that information and storing it alongside the metadata.
Ensure that the security vulnerability described here is addressed: #375180 (closed)
Technical
Change the debug_mode
function to check a new persisted column stored for each build that is populated via the runner API from the state of the variables in runner.
In this issue we will implement changing the permission checks to look at the new build related column rather than the variables set in gitlab rails. We can do this via the security developer process.
We must ensure that when the CI_DEBUG_TRACE is set to "1", "t", "T", "true", "TRUE", "True"
the logs stay hidden. More details around that vulnerability here: #375180 (closed). We can fall back on the parsing the variables from gitlab if data is not received from the runner(they are not on the correct version) but change debug_mode?
to consider "1", "t", "T", "true", "TRUE", "True"as
true`.
Implementation notes
Trace patch endpoint changes
We need to make a change to how debug_trace
is saved when it is sent to the append trace endpoint.
Runner will send the debug_trace
value as a boolean on the first append or whenever <code data-sourcepos="36:89-36:102">CI_DEBUG_TRACE</code> or <code data-sourcepos="36:109-36:125">CI_DEBUG_SERVICES</code> changes to <code data-sourcepos="36:140-36:143">true</code>, and currently we only save on first append . We should update the append trace endpoint to also save if debug_trace
is not null.
Build model changes
Then in app/models/ci/build.rb
we can change the debug_mode?
check to look at the value on both the gitlab and runner side. This will ensure backwards compatibility with older versions of runner.
def debug_mode?
# perform the check on both sides in case the runner version is old
[
"1", "t", "T", "true", "TRUE", "True"
].include?(variables['CI_DEBUG_TRACE']&.value) || trace_metadata.debug_trace
end
We check debug_mode
before allowing access to the logs.
Project Application controller changes
We currently have authorize_read_build_trace! in the Projects::ApplicationController
where we set a custom error message that needs to be updated to reflect that the variables value is respected on the runner side and not just what is set in the yaml file or on the gitlab side. I've also made a small tech debt issue to move this code #387676 (closed).
Documentation
We need to update the documentation here: https://docs.gitlab.com/ee/ci/variables/#restrict-access-to-debug-logging