Deprecate FF_USE_DIRECT_DOWNLOAD in favour of a direct_download setting
What does this MR do?
Adds a direct_download setting to [runners.artifact] and an ARTIFACT_DIRECT_DOWNLOAD CI/CD variable, and deprecates FF_USE_DIRECT_DOWNLOAD for removal in %19.9 (#40200).
- Precedence, as agreed with vtak:
direct_download, thenFF_USE_DIRECT_DOWNLOADin[runners.feature_flags], thenARTIFACT_DIRECT_DOWNLOAD, thenFF_USE_DIRECT_DOWNLOADas a job variable, thentrue.config.tomlwins over job variables, as it does for feature flags since !5782 (merged). - The runner passes the result to the helper as
--direct-download=<bool>, on both the shell and Concrete paths. The helper has accepted this flag since 13.1, and it still readsFF_USE_DIRECT_DOWNLOADfrom its environment until the flag is removed. - The resolved value is also exported as
FF_USE_DIRECT_DOWNLOAD, so a helper that only reads the flag, such as an older one on the Concrete path, still gets it. - A job-log warning appears in every job where the flag resolves to
false. Whendirect_downloadorARTIFACT_DIRECT_DOWNLOADoverrides the flag, the warning says the flag is ignored.
Why was this MR needed?
Setting the flag to false is the only runner-side way to keep artifact downloads proxied through GitLab, and people rely on it, including on GitLab Dedicated. That makes it configuration rather than a temporary toggle, so it needs a setting before the flag can be removed (#39682 (closed)).
What's the best way to test this MR?
Unit tests cover the precedence order, the warnings and the exported flag value, both execution paths, and the helper parsing --direct-download=false over FF_USE_DIRECT_DOWNLOAD=true in its environment.
To try it by hand, use an instance that allows direct downloads. Set direct_download = false in [runners.artifact] and run a job that downloads artifacts. The helper shouldn't log Direct download from.
Caveats:
- The warning also appears in jobs that don't download artifacts.
- The job log header lists
FF_USE_DIRECT_DOWNLOAD:falsewhenever downloads resolve tofalse, because it shows the resolved value.
What are the relevant issue numbers?
Closes #40198 (closed). Part of #39682 (closed).