Gitlab offers way to provide CI/CD variables that jobs can consume.
What happens if a "required"/"needed" variable is not available to the job? My thought is it should fail/pause until added.
The workaround today is to do a script variable check, i.e. If (!(Test-Path variable:global:foo)) {Throw "Variable foo is missing..."}. This seems very ugly.
Proposal
Add a required_vars entry for job definition, ensuring a job doesn't run without missing a variable dependency.
This specifically affected Production/non-test environments as test environments had dynamically generated variables where as Prod environments uses variables generally entered into Gitlab CI/CD settings page. #tookdownprod
This item has been automatically determined to have 4 or fewer upvotes and is, for now, being marked as awaiting further demand. If you feel this is incorrect, please comment on the issue and I'll be happy to take a look.
+1 Would really like to see this -- it's one of the main features available in Jenkins pipelines that makes me hesitant to move all flows over to gitlab. The biggest problem is if I start a job without the variables, and it's late in the pipeline, the job itself fails and I have to start from the beginning.
Going to add a +1 here. The use-case for me would be for a manual release job, where I know I want my devops team to enter some variables in before running the job. Would be great if we could simply require variables, even BETTER would be being able to define enums that they could pick values from a dropdown. That's probably outside of this scope, though.
Why don't these issues get traction? Is GitLab's philosophy really simply "WHY AREN'T YOU DOING CD!?"
There are at least two other issues that make this kind of a show-stopper for anyone who wants to utilize manual jobs but requires some sort of fine grain control. One of which is really bad, since you can't even wrap the issue with a different UI because the actual REST endpoint doesn't even support additional parameters.
#37267 (closed) < this one is even worse. Really? How is this not a priority?
There are many folks who are trying to migrate to GitLab for all their deployment pipeline needs and just aren't ready for 100%, hands-off continuous deployment because they're either bound by company policies or just haven't graduated to that level of excellence in their pipelines yet. The entire environments feature of GitLab is kind of negated and ruined by simple lack of functionality in these areas.
For now, I have to stick with Jenkins, TeamCity, or other solutions to offer my users the user-friendly abilities afforded by a usable UI for doing manual deployments. I love GitLab, but this isn't a good way to get customers who aren't ready for pure code-driven CD yet (which I am sure there are a LOT of out there).
Thank you @farvour for providing your passionate input for this feature! Really appreciated. I recommend reaching out to your sales rep or your technical account manager. They can help advocate on your behalf to move the needle on issues which are critical to your organisation.
Thanks @stevekanter . GitHub Workflows already has this capability, and it's disappointing it does not yet exist with GitLab. It's kind of a show-stopper for a lot of our manual jobs.
Just happened to stumble upon this issue. This indeed would also solve our problem with manually triggered release jobs that would require various inputs from whoever is triggering that job.
@dhershkovitch a Premium customer just pinged us about this issue and if any timeline has been determined for this as they're looking to utilize this feature, any thoughts as to when this may get released?
Extending "variables:" concept to allow more fine grained control over what variables are and how they shall behave in term of
type, value optional or mandatory with and without defaults 'defaul' value is far better than introducing a very scoped new paramater specific for jobs:
In azure devops they also started to introduce parameter's (for tenmplates) on top of variables and that started to feel like never well integrated and always leads to copy paste from variabels to parameters and vice versa.
The magic would be to make old and new be compatible either in same variables section or at least support only old and or only new schema per block, so people could migrate variables by variables block.
Why not extend variables like this;
simple/current schema
variables: var_string: "my string" var_num: 12 var _bool: true var_composite: "notyet: supported type"
advanced new schema
..functional downwards comaptible.
variables: - name: "var_string" value: "my string" # required: false # is optional vecause defaulted to false always - name: "var_string" value: "my string" # type: string # type: string is default so is optional to be set in this case - name: "var_num" value: 12 # type: number # type: number is default if valie is set and can be detected as number - name: "var_bool" value: true # type: bool # type: bool is default if value is set and can be detected as number
- name: "simple_date" type: date value: 2001-12-14T21:59:43.10-05:00 - name: "my_time_date" value: 2002-12-14 - name: "var_composite" type: yaml value: my: composite: value_string: "test" value_num: 3 value_bool: true - name: "var_composite" value: type: json # type: bool is default if value is set and can be detected as number value: {"my": {"composite": { "value_string": "test","value_num": 3, "value_bool": true } - name: "var_composite" type: file # type: bool is default if value is set and can be detected as number value: "path/to/my/file.bin"
default variable value
- name: "var_string" default: "my string" # type: string # type: string is default if valie is set and can be detected as number - name: "var_number" default: 10 # type: number # type: number is default if valie is set and can be detected as number - name: "var_bool" default: false # type: number # type: bool is default if valie is set and can be detected as number
required value (optional, but if value is not set implicit is set to true)
- name: "var_bool" type: bool # required: true # type: bool is optional - because set to true automatically if "value:" field is not specified
secret variable value ( optional flag to mask variables at console, logs and UI while typing)
Below exmaple requires to enter value interactively (e.g manual rule job execution) or its passed downstream by another job by template, trigger or (external) api call by passing in from somwhere (e.g. vault)
- name: "var_secret_string" secret: true # false by default # required: true # type: bool is optional - because set to true automatically if "value:" field is not specified - name: "var_secret_string" secret: true # modifier is false by default type: vault # source value from hashciorp vault value: "vault:uri"
Some rules for new schema
.. which will the usage keep easry to undertand, debugable, strict and maintainable
In context of downstream/inheritance or variables along "include template"-> pipeline -> jobs
Overloading the specification of type, required and secret is not allowed
Overwriting default values is not allowed
Overwriting "value:" value donwstream is allowed
If a variables is defined as "required: true" (aka neither value nor default is set ) in a template, the value shall be set upstream for a variable (re)defined ther as well with same name, typ and required:true (optional secret:true). Only value is allowed to bet set upstream
In contextof resolution of variables.
The usage of variables being required: true without default value will at runtime lead
Running the pipelne via the UI shall autoamtically render a job variable field for all required variables and mark them to be set as mandatory
Running the pipelne via the UI shall offer dropwdown select when typing variable names for defined variables and use a type specific input field editor (string. number, boolean, date, yaml comsposite)
Sry, but i missed where the solution for this issue is provided.
#383375 (closed) does not solve the problem for standard pipelines, juts for pipeline components.
If i create a pipeline where i consume variables, e.g as secrets, and those secrets are not set via ci variables the pipeline shall somehow validate and warn that i am using vars which are not existing.
Also at runtime directly after click the pipeline/jobs shall check if those envs vars are present if set as mandatory arguments, nothing which will be created at runtime.
I understand that the feature is also needed for components, but i don't want to create extra a component just to allow me to use this specification feature
I can't find docs how to use the feature in a regular pipeline or on jog level.
Btw in azure devops they have as well such a feature called "parameters" which is an enhancement to variables to allow exactly such behavior.
However it would be even better to just add a flag for a regular var, that its expected to be present as env var mandatory ad pipeline start.