gitlab-runner issueshttps://gitlab.com/gitlab-org/gitlab-runner/-/issues2024-03-06T01:03:51Zhttps://gitlab.com/gitlab-org/gitlab-runner/-/issues/29223GitLab Runner Fleeting plugin for Azure Virtual Machines - (GA)2024-03-06T01:03:51ZDarren Eastmandeastman@gitlab.comGitLab Runner Fleeting plugin for Azure Virtual Machines - (GA)## Overview
The GitLab Runner Azure VMs Fleeting Plugin is now generally available.
### Supported OS & compute architectures
- Linux on x86-64
### Example configuration)
```
concurrent = 10
[[runners]]
name = "instance autoscaler...## Overview
The GitLab Runner Azure VMs Fleeting Plugin is now generally available.
### Supported OS & compute architectures
- Linux on x86-64
### Example configuration)
```
concurrent = 10
[[runners]]
name = "instance autoscaler example"
url = "https://gitlab.com"
token = "<token>"
shell = "sh"
executor = "instance"
# Autoscaler config
[runners.autoscaler]
plugin = "fleeting-plugin-azure"
capacity_per_instance = 1
max_use_count = 1
max_instances = 10
```
### Disclaimer
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->17.0https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29222GitLab Runner Fleeting plugin for AWS EC2 instances (GA)2024-03-06T01:03:51ZDarren Eastmandeastman@gitlab.comGitLab Runner Fleeting plugin for AWS EC2 instances (GA)## Release notes
The GitLab Runner AWS EC2 Fleeting Plugin is now generally available.
## Overview
This issue is the capstone issue that delivers an AWS EC2 plug-in for autoscaling Runners on AWS virtual machines.
## Supported opera...## Release notes
The GitLab Runner AWS EC2 Fleeting Plugin is now generally available.
## Overview
This issue is the capstone issue that delivers an AWS EC2 plug-in for autoscaling Runners on AWS virtual machines.
## Supported operating systems & compute architectures
- Linux OS on x86-64
## Example Configuration
```
concurrent = 10
[[runners]]
name = "instance autoscaler example"
url = "https://gitlab.com"
token = "<token>"
shell = "sh"
executor = "instance"
# Autoscaler config
[runners.autoscaler]
plugin = "fleeting-plugin-aws"
capacity_per_instance = 1
max_use_count = 1
max_instances = 10
[runners.autoscaler.plugin_config] # plugin specific configuration (see plugin documentation)
name = "my-linux-asg" # AWS Autoscaling Group name
profile = "default" # optional, default is 'default'
config_file = "/home/user/.aws/config" # optional, default is '~/.aws/config'
credentials_file = "/home/user/.aws/credentials" # optional, default is '~/.aws/credentials'
[runners.autoscaler.connector_config]
username = "ec2-user"
use_external_addr = true
[[runners.autoscaler.policy]]
idle_count = 5
idle_time = "20m0s"
```
### Disclaimer
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->17.0https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29221GitLab Runner Fleeting plugin for GCP Compute Engine - GA2024-03-06T01:02:29ZDarren Eastmandeastman@gitlab.comGitLab Runner Fleeting plugin for GCP Compute Engine - GA## Overview
The GitLab Runner GCP Fleeting Plugin is now generally available.
### Supported OS & compute architectures
- Linux on x86-64
### Example configuration)
```
concurrent = 10
[[runners]]
name = "instance autoscaler examp...## Overview
The GitLab Runner GCP Fleeting Plugin is now generally available.
### Supported OS & compute architectures
- Linux on x86-64
### Example configuration)
```
concurrent = 10
[[runners]]
name = "instance autoscaler example"
url = "https://gitlab.com"
token = "<token>"
shell = "sh"
executor = "instance"
# Autoscaler config
[runners.autoscaler]
plugin = "fleeting-plugin-gcp"
capacity_per_instance = 1
max_use_count = 1
max_instances = 10
```
## Disclaimer
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->17.0https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29063Sign build artifacts in Runner with Cosign2024-02-13T20:28:17ZSam WhiteSign build artifacts in Runner with Cosign## Description
<!--
Include problem, use cases, benefits, and/or goals
-->
## Proposal
1. For jobs where `RUNNER_GENERATE_ARTIFACTS_METADATA = "true"`, all artifacts that are produced by the GitLab Runner will be signed automatically ...## Description
<!--
Include problem, use cases, benefits, and/or goals
-->
## Proposal
1. For jobs where `RUNNER_GENERATE_ARTIFACTS_METADATA = "true"`, all artifacts that are produced by the GitLab Runner will be signed automatically by default. A `.sig` file will be generated and stored as a build artifact containing the signature.
1. The format of the `.sig` file should match https://github.com/sigstore/cosign/blob/main/specs/SIGNATURE_SPEC.md.
1. Additionally, we will begin adding the required envelope to the generated attestation per https://github.com/in-toto/attestation/tree/v0.1.0/spec#envelope
1. The JSON that we currently generate as our attestation file will be base64 encoded and will become the value of the `payload` attribute.
1. The `payloadType` attribute is static and should always be set to `application/vnd.in-toto+json`
1. The `signatures` attribute will contain the signature of the `payload`
The resulting attestation file should match this format:
```json
{
"payloadType": "application/vnd.in-toto+json",
"payload": "<Base64(SERIALIZED_BODY)>",
"signatures": [{
"keyid": "<KEYID>",
"sig": "<Base64(SIGNATURE)>"
}]
}
```
Currently the latest proposal to facilitate this is to use [Fulcio](https://github.com/sigstore/fulcio) to manage the key that is used for signing.
### Note:
- The implementation plan is to limit the use of the OIDC token to only the artifact signing step (Runner/Cosign/Fulcio). This approach ensures that the inserted secret used to sign the artifact is used only once.
## Disclaimer
<!--
Please paste a link of the related issues or/and merge requests
-->
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->Backloghttps://gitlab.com/gitlab-org/gitlab-runner/-/issues/4643Support multi-threaded download/upload cache and artifacts2023-10-25T21:05:29ZBlack-Holegitlab@bugs.ccSupport multi-threaded download/upload cache and artifacts## Description
<!--
Include problem, use cases, benefits, and/or goals
-->
Currently we use minio as a storage object and everything works fine. But because the project itself is nodejs. Causes the size of the cache to exceed 800M.
I ...## Description
<!--
Include problem, use cases, benefits, and/or goals
-->
Currently we use minio as a storage object and everything works fine. But because the project itself is nodejs. Causes the size of the cache to exceed 800M.
I looked at the source code and found that when downloading, I used a single-threaded download.
https://gitlab.com/gitlab-org/gitlab-runner/blob/master/commands/helpers/cache_extractor.go#L92-104
https://gitlab.com/gitlab-org/gitlab-runner/blob/master/commands/helpers/cache_extractor.go#L70
Whether it is s3 or mimio, or other storage methods, theoretically support multi-threaded download. The current single-threaded download will cause the time to become very long when the cache is downloaded.
Of course, if you use multi-threading when uploading, it would be better (if you consider support)
## Discussions to date on the feasibility of implementing this proposal (revised 2022-01-11)
- This proposal would certainly improve performance, but there's technical reasons for why it cannot be done at the moment. This was recently discussed, but we haven't decided on a way forward yet.
- Multipart uploads don't work particularly well for pre-signed requests, which is what we have now, because GitLab-Runner pre-generates the upload URL and provides it to the job. The reason for this is because we don't want to reveal the authentication token to the job because it would expose the entire bucket to it.
- Multipart uploads require more communication with the cloud storage provider than just this simple pre-signed request and requires the authentication token.
### The solution we want for this is something that works across all executors, but becomes quite **complex**:
- We could proxy all requests to a service that performs the authentication, without revealing the authentication token to the job.
- We could allow jobs to communicate with Runner and pre-signed each multipart upload chunk URL. But GitLab Jobs don't currently communicate to Runner like this, so it would be a very large change.
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->Backloghttps://gitlab.com/gitlab-org/gitlab-runner/-/issues/1057Allow user to specify root folder for artifacts2024-01-19T19:16:08ZJürgen SteinblockAllow user to specify root folder for artifacts## Release notes
{placeholder for release notes}
## Problem
I have a build step
```
deploy:
stage: deploy
script: ...
artifacts:
paths:
- ci/deploy/
```
In gitlab I can browse and download the artifacts. However all f...## Release notes
{placeholder for release notes}
## Problem
I have a build step
```
deploy:
stage: deploy
script: ...
artifacts:
paths:
- ci/deploy/
```
In gitlab I can browse and download the artifacts. However all files are stored in a folder `ci/deploy`
Lets say my directory structure looks like this
```
ci/deploy/packages/package1.zip
ci/deploy/packages/package2.zip
ci/deploy/docs/toc.txt
ci/deploy/docs/file.txt
```
What I really want is an archive containing
```
packages/package1.zip
packages/package2.zip
docs/toc.txt
docs/file.txt
```
## Proposal
Specify a `root` directory. Paths/excludes will then match relative to that root.
```
# File on disk is located in <build_dir>/root/dir/file.txt
artifacts:
root: root
paths:
- dir/file.txt
```
## Disclaimer
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->Backlog