Kubernetes Windows-builds fail with Helm-Charts versions 0.30.0 and 0.31.0
Summary
Using Kubernetes (AKS) to build on Windows nodes works with the Kubernetes Runner Helm-Chart version 0.29.0 (gitlab-runner 13.12.0), but fails with 0.30.0 and 0.31.0 (gitlab-runner 14.0.0)
Steps to reproduce
I've installed the Helm-chart with the following section in it's values.yml:
config: |
[[runners]]
shell = "pwsh"
executor = "kubernetes"
[runners.feature_flags]
FF_USE_POWERSHELL_PATH_RESOLVER = true
[runners.kubernetes]
helper_image = "gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809"
poll_timeout = 3600
[runners.kubernetes.node_selector]
"kubernetes.io/arch" = "amd64"
"kubernetes.io/os" = "windows"
"node.kubernetes.io/windows-build" = "10.0.17763"
[runners.kubernetes.node_tolerations]
"kubernetes.azure.com/scalesetpriority=spot" = "NoSchedule" Note that I had to set the helper_image and shell explicitly, even with kubernetes.io/os="windows". Thus the windows-detection-system did not work for me.
Then I did :
helm3 install --namespace gitlab gitlab-runner-fpcprojects-windows -f values_runner_fpcprojects_windows.yaml gitlab/gitlab-runner --kubeconfig ~/.kube/config --version 0.30.0It works with --version 0.29.0. Probably because that version does not use the init-logs container which fails.
.gitlab-ci.yml
stages:
- build
windows build:
image: fppkg.azurecr.io/fpc-322-windows-buildimage
stage: build
tags:
- windows
script:
- fppkg build
- fppkg archive
artifacts:
paths:
- fpcdab-*.source.zip
expire_in: 1 month
linux build:
image: quay.io/loesje/fpc-322-linux-buildimage
stage: build
tags:
- linux
script:
- fppkg build
- fppkg archive
artifacts:
paths:
- fpcdab-*.source.zip
expire_in: 1 monthActual behavior
The build fails, because some 'init-logs' container doesn't start. This is because it tries to execute a linux-script. This is the message it shows:
Error: failed to start container "init-logs": Error response from daemon: container init-logs encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2)And this is a snippet from kubectl describe...
Init Containers:
init-logs:
Container ID: docker://10acbf2edf0ff4bb1b5d1677c4384bfd33a9307d8aa62f50bb52d4fccdca767e
Image: gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809
Image ID: docker-pullable://gitlab/gitlab-runner-helper@sha256:037a4999e4c6fa904a4cc13a7fa938e7a4cc2df0872fe4dec93ed0f867ebc559
Port: <none>
Host Port: <none>
Command:
sh
-c
touch /logs-28515712-1524304017/output.log && (chmod 777 /logs-28515712-1524304017/output.log || exit 0)
State: Terminated
Reason: ContainerCannotRun
Message: container 10acbf2edf0ff4bb1b5d1677c4384bfd33a9307d8aa62f50bb52d4fccdca767e encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2)
Exit Code: 127
Started: Mon, 23 Aug 2021 01:12:42 +0200
Finished: Mon, 23 Aug 2021 01:12:42 +0200
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/builds from repo (rw)
/logs-28515712-1524304017 from logs (rw)
/scripts-28515712-1524304017 from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4dhfs (ro)Note the command:
sh -c touch /logs-28515712-1524304017/output.log && (chmod 777 /logs-28515712-1524304017/output.log || exit 0)which obviously will fail in Windows.
Expected behavior
Do a full build
Relevant logs and/or screenshots
job log
Running with gitlab-runner 14.0.0 (3b6f852e)
on aks_cnoc_jvdsluis_windows gfbyL6sY
feature flags: FF_USE_POWERSHELL_PATH_RESOLVER:true
Resolving secrets 00:00
Preparing the "kubernetes" executor 00:00
Using Kubernetes namespace: gitlab
Using Kubernetes executor with image fppkg.azurecr.io/fpc-322-windows-buildimage ...
Using attach strategy to execute scripts...
Preparing environment 00:12
Waiting for pod gitlab/runner-gfbyl6sy-project-28515712-concurrent-04rlrj to be running, status is Pending
Waiting for pod gitlab/runner-gfbyl6sy-project-28515712-concurrent-04rlrj to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-logs]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab/runner-gfbyl6sy-project-28515712-concurrent-04rlrj to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-logs]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab/runner-gfbyl6sy-project-28515712-concurrent-04rlrj to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-logs]"
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
ERROR: Job failed (system failure): prepare environment: waiting for pod running: pod status is failed. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more informationKubernetes description of pod
Name: runner-gfbyl6sy-project-28515712-concurrent-0rlfs6
Namespace: gitlab
Priority: 0
Node: aksmspool00000b/10.240.0.35
Start Time: Mon, 23 Aug 2021 01:12:40 +0200
Labels: pod=runner-gfbyl6sy-project-28515712-concurrent-0
Annotations: <none>
Status: Failed
IP: 10.240.0.50
IPs:
IP: 10.240.0.50
Init Containers:
init-logs:
Container ID: docker://10acbf2edf0ff4bb1b5d1677c4384bfd33a9307d8aa62f50bb52d4fccdca767e
Image: gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809
Image ID: docker-pullable://gitlab/gitlab-runner-helper@sha256:037a4999e4c6fa904a4cc13a7fa938e7a4cc2df0872fe4dec93ed0f867ebc559
Port: <none>
Host Port: <none>
Command:
sh
-c
touch /logs-28515712-1524304017/output.log && (chmod 777 /logs-28515712-1524304017/output.log || exit 0)
State: Terminated
Reason: ContainerCannotRun
Message: container 10acbf2edf0ff4bb1b5d1677c4384bfd33a9307d8aa62f50bb52d4fccdca767e encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2)
Exit Code: 127
Started: Mon, 23 Aug 2021 01:12:42 +0200
Finished: Mon, 23 Aug 2021 01:12:42 +0200
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/builds from repo (rw)
/logs-28515712-1524304017 from logs (rw)
/scripts-28515712-1524304017 from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4dhfs (ro)
Containers:
build:
Container ID:
Image: fppkg.azurecr.io/fpc-322-windows-buildimage
Image ID:
Port: <none>
Host Port: <none>
Command:
pwsh
-NoProfile
-NoLogo
-InputFormat
text
-OutputFormat
text
-NonInteractive
-ExecutionPolicy
Bypass
-Command
-
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION: false
FF_NETWORK_PER_BUILD: false
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: false
FF_USE_DIRECT_DOWNLOAD: true
FF_SKIP_NOOP_BUILD_STAGES: true
FF_USE_FASTZIP: false
FF_GITLAB_REGISTRY_HELPER_IMAGE: true
FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR: false
FF_ENABLE_BASH_EXIT_CODE_CHECK: false
FF_USE_WINDOWS_LEGACY_PROCESS_STRATEGY: true
FF_SKIP_DOCKER_MACHINE_PROVISION_ON_CREATION_FAILURE: false
FF_USE_NEW_BASH_EVAL_STRATEGY: false
FF_USE_POWERSHELL_PATH_RESOLVER: false
CI_JOB_IMAGE: fppkg.azurecr.io/fpc-322-windows-buildimage
CI_RUNNER_SHORT_TOKEN: gfbyL6sY
CI_BUILDS_DIR: /builds
CI_PROJECT_DIR: /builds/gfbyL6sY/0/fpcprojects/fpcdab
CI_CONCURRENT_ID: 0
CI_CONCURRENT_PROJECT_ID: 0
CI_SERVER: yes
CI_JOB_STATUS: running
CI_SERVER_TLS_CA_FILE: -----BEGIN CERTIFICATE-----
MIIGBzCCBO+gAwIBAgIQXCLSMilzZJR9TSABzbgKzzANBgkqhkiG9w0BAQsFADCB
woE3CsYogf1SfQM=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIGEzCCA/ugAwIBAgIQfVtRJrR2uhHbdBYLvFMNpzANBgkqhkiG9w0BAQwFADCB
00u/I5sUKUErmgQfky3xxzlIPK1aEn8=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
jjxDah2nGN59PRbxYvnKkKj9
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFgTCCBGmgAwIBAgIQOXJEOvkit1HX02wQ3TE1lTANBgkqhkiG9w0BAQwFADB7
vGp4z7h/jnZymQyd/teRCBaho1+V
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEMjCCAxqgAwIBAgIBATANBgkqhkiG9w0BAQUFADB7MQswCQYDVQQGEwJHQjEb
smPi9WIsgtRqAEFQ8TmDn5XpNpaYbg==
-----END CERTIFICATE-----
CI_PIPELINE_ID: 357473838
CI_PIPELINE_URL: https://gitlab.com/fpcprojects/fpcdab/-/pipelines/357473838
CI_JOB_ID: 1524304017
CI_JOB_URL: https://gitlab.com/fpcprojects/fpcdab/-/jobs/1524304017
CI_JOB_STARTED_AT: 2021-08-22T23:12:40Z
CI_BUILD_ID: 1524304017
CI_REGISTRY_USER: gitlab-ci-token
CI_DEPENDENCY_PROXY_USER: gitlab-ci-token
CI_JOB_NAME: windows build
CI_JOB_STAGE: build
CI_NODE_TOTAL: 1
CI_BUILD_NAME: windows build
CI_BUILD_STAGE: build
CI: true
GITLAB_CI: true
CI_SERVER_URL: https://gitlab.com
CI_SERVER_HOST: gitlab.com
CI_SERVER_PORT: 443
CI_SERVER_PROTOCOL: https
CI_SERVER_NAME: GitLab
CI_SERVER_VERSION: 14.3.0-pre
CI_SERVER_VERSION_MAJOR: 14
CI_SERVER_VERSION_MINOR: 3
CI_SERVER_VERSION_PATCH: 0
CI_SERVER_REVISION: af8e04d574a
GITLAB_FEATURES: audit_events,blocked_issues,board_iteration_lists,code_owners,code_review_analytics,contribution_analytics,description_diffs,elastic_search,full_codequality_report,group_activity_analytics,group_bulk_edit,group_webhooks,issuable_default_templates,issue_weights,iterations,ldap_group_sync,member_lock,merge_request_approvers,milestone_charts,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,multiple_merge_request_reviewers,project_merge_request_analytics,protected_refs_for_users,push_rules,repository_mirrors,repository_size_limit,resource_access_token,seat_link,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,send_emails_from_admin_area,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,ci_secrets_management,cluster_agents,cluster_deployments,code_owner_approval_required,commit_committer_check,compliance_framework,custom_compliance_frameworks,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,cycle_analytics_for_groups,cycle_analytics_for_projects,db_load_balancing,default_branch_protection_restriction_in_groups,default_project_deletion_protection,disable_name_update_for_users,email_additional_text,epics,extended_audit_events,external_authorization_service_api_management,feature_flags_related_issues,file_locks,geo,generic_alert_fingerprinting,git_two_factor_enforcement,github_project_service_integration,group_allowed_email_domains,group_coverage_reports,group_forking_protection,group_ip_restriction,group_merge_request_analytics,group_merge_request_approval_settings,group_milestone_project_releases,group_project_templates,group_repository_analytics,group_saml,group_saml_group_sync,group_scoped_ci_variables,group_wikis,incident_sla,incident_metric_upload,ide_schema_config,issues_analytics,jira_issues_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,admin_merge_request_approvers_rules,merge_trains,metrics_reports,multiple_alert_http_integrations,multiple_approval_rules,multiple_group_issue_boards,multiple_iteration_cadences,object_storage,operations_dashboard,package_forwarding,pages_size_limit,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,smartcard_auth,swimlanes,type_of_work_analytics,minimal_access_role,unprotection_restrictions,ci_project_subscriptions,incident_timeline_view,oncall_schedules,escalation_policies,export_user_permissions,api_fuzzing,auto_rollback,cilium_alerts,cluster_image_scanning,external_status_checks,container_scanning,coverage_fuzzing,credentials_inventory,dast,dependency_scanning,devops_adoption,dora4_analytics,enforce_personal_access_token_expiration,enforce_ssh_key_expiration,enterprise_templates,environment_alerts,evaluate_group_level_compliance_pipeline,group_ci_cd_analytics,group_level_compliance_dashboard,group_level_devops_adoption,incident_management,inline_codequality,insights,instance_level_devops_adoption,issuable_health_status,jira_vulnerabilities_integration,jira_issue_association_enforcement,license_scanning,personal_access_token_expiration_policy,prometheus_alerts,pseudonymizer,quality_management,release_evidence_test_artifacts,report_approver_rules,requirements,sast,sast_custom_rulesets,sast_fp_reduction,secret_detection,security_dashboard,security_on_demand_scans,security_orchestration_policies,status_page,subepics,threat_monitoring,vulnerability_auto_fix,vulnerability_finding_signatures
CI_PROJECT_ID: 28515712
CI_PROJECT_NAME: fpcdab
CI_PROJECT_TITLE: fpcdab
CI_PROJECT_PATH: fpcprojects/fpcdab
CI_PROJECT_PATH_SLUG: fpcprojects-fpcdab
CI_PROJECT_NAMESPACE: fpcprojects
CI_PROJECT_ROOT_NAMESPACE: fpcprojects
CI_PROJECT_URL: https://gitlab.com/fpcprojects/fpcdab
CI_PROJECT_VISIBILITY: public
CI_PROJECT_REPOSITORY_LANGUAGES: pascal
CI_PROJECT_CLASSIFICATION_LABEL:
CI_DEFAULT_BRANCH: trunk
CI_CONFIG_PATH: .gitlab-ci.yml
CI_PAGES_DOMAIN: gitlab.io
CI_PAGES_URL: https://fpcprojects.gitlab.io/fpcdab
CI_REGISTRY: registry.gitlab.com
CI_REGISTRY_IMAGE: registry.gitlab.com/fpcprojects/fpcdab
CI_DEPENDENCY_PROXY_SERVER: gitlab.com:443
CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: gitlab.com:443/fpcprojects/dependency_proxy/containers
CI_API_V4_URL: https://gitlab.com/api/v4
CI_PIPELINE_IID: 6
CI_PIPELINE_SOURCE: push
CI_PIPELINE_CREATED_AT: 2021-08-22T23:00:01Z
CI_COMMIT_SHA: 29b9e6c8f0a3243143ea2f1783881b9253984ab4
CI_COMMIT_SHORT_SHA: 29b9e6c8
CI_COMMIT_BEFORE_SHA: c7614b9ddf5834e119ec4b992161f4092699ce82
CI_COMMIT_REF_NAME: trunk
CI_COMMIT_REF_SLUG: trunk
CI_COMMIT_BRANCH: trunk
CI_COMMIT_MESSAGE: * Windows and Linux build
CI_COMMIT_TITLE: * Windows and Linux build
CI_COMMIT_DESCRIPTION:
CI_COMMIT_REF_PROTECTED: true
CI_COMMIT_TIMESTAMP: 2021-08-23T00:59:56+02:00
CI_COMMIT_AUTHOR: Joost van der Sluis <joost@cnoc.nl>
CI_BUILD_REF: 29b9e6c8f0a3243143ea2f1783881b9253984ab4
CI_BUILD_BEFORE_SHA: c7614b9ddf5834e119ec4b992161f4092699ce82
CI_BUILD_REF_NAME: trunk
CI_BUILD_REF_SLUG: trunk
CI_RUNNER_ID: 9781130
CI_RUNNER_DESCRIPTION: aks_cnoc_jvdsluis_windows
CI_RUNNER_TAGS: windows
GITLAB_USER_ID: 9100338
GITLAB_USER_EMAIL: joost@cnoc.nl
GITLAB_USER_LOGIN: jvdsluis
GITLAB_USER_NAME: Joost van der Sluis
CI_DISPOSABLE_ENVIRONMENT: true
CI_RUNNER_VERSION: 14.0.0
CI_RUNNER_REVISION: 3b6f852e
CI_RUNNER_EXECUTABLE_ARCH: linux/amd64
Mounts:
/builds from repo (rw)
/logs-28515712-1524304017 from logs (rw)
/scripts-28515712-1524304017 from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4dhfs (ro)
helper:
Container ID:
Image: gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809
Image ID:
Port: <none>
Host Port: <none>
Command:
pwsh
-NoProfile
-NoLogo
-InputFormat
text
-OutputFormat
text
-NonInteractive
-ExecutionPolicy
Bypass
-Command
-
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION: false
FF_NETWORK_PER_BUILD: false
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: false
FF_USE_DIRECT_DOWNLOAD: true
FF_SKIP_NOOP_BUILD_STAGES: true
FF_USE_FASTZIP: false
FF_GITLAB_REGISTRY_HELPER_IMAGE: true
FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR: false
FF_ENABLE_BASH_EXIT_CODE_CHECK: false
FF_USE_WINDOWS_LEGACY_PROCESS_STRATEGY: true
FF_SKIP_DOCKER_MACHINE_PROVISION_ON_CREATION_FAILURE: false
FF_USE_NEW_BASH_EVAL_STRATEGY: false
FF_USE_POWERSHELL_PATH_RESOLVER: false
CI_JOB_IMAGE: fppkg.azurecr.io/fpc-322-windows-buildimage
CI_RUNNER_SHORT_TOKEN: gfbyL6sY
CI_BUILDS_DIR: /builds
CI_PROJECT_DIR: /builds/gfbyL6sY/0/fpcprojects/fpcdab
CI_CONCURRENT_ID: 0
CI_CONCURRENT_PROJECT_ID: 0
CI_SERVER: yes
CI_JOB_STATUS: running
CI_SERVER_TLS_CA_FILE: -----BEGIN CERTIFICATE-----
woE3CsYogf1SfQM=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIGEzCCA/ugAwIBAgIQfVtRJrR2uhHbdBYLvFMNpzANBgkqhkiG9w0BAQwFADCB
00u/I5sUKUErmgQfky3xxzlIPK1aEn8=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIF3jCCA8agAwIBAgIQAf1tMPyjylGoG7xkDjUDLTANBgkqhkiG9w0BAQwFADCB
jjxDah2nGN59PRbxYvnKkKj9
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFgTCCBGmgAwIBAgIQOXJEOvkit1HX02wQ3TE1lTANBgkqhkiG9w0BAQwFADB7
vGp4z7h/jnZymQyd/teRCBaho1+V
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEMjCCAxqgAwIBAgIBATANBgkqhkiG9w0BAQUFADB7MQswCQYDVQQGEwJHQjEb
smPi9WIsgtRqAEFQ8TmDn5XpNpaYbg==
-----END CERTIFICATE-----
CI_PIPELINE_ID: 357473838
CI_PIPELINE_URL: https://gitlab.com/fpcprojects/fpcdab/-/pipelines/357473838
CI_JOB_ID: 1524304017
CI_JOB_URL: https://gitlab.com/fpcprojects/fpcdab/-/jobs/1524304017
CI_JOB_STARTED_AT: 2021-08-22T23:12:40Z
CI_BUILD_ID: 1524304017
CI_REGISTRY_USER: gitlab-ci-token
CI_DEPENDENCY_PROXY_USER: gitlab-ci-token
CI_JOB_NAME: windows build
CI_JOB_STAGE: build
CI_NODE_TOTAL: 1
CI_BUILD_NAME: windows build
CI_BUILD_STAGE: build
CI: true
GITLAB_CI: true
CI_SERVER_URL: https://gitlab.com
CI_SERVER_HOST: gitlab.com
CI_SERVER_PORT: 443
CI_SERVER_PROTOCOL: https
CI_SERVER_NAME: GitLab
CI_SERVER_VERSION: 14.3.0-pre
CI_SERVER_VERSION_MAJOR: 14
CI_SERVER_VERSION_MINOR: 3
CI_SERVER_VERSION_PATCH: 0
CI_SERVER_REVISION: af8e04d574a
GITLAB_FEATURES: audit_events,blocked_issues,board_iteration_lists,code_owners,code_review_analytics,contribution_analytics,description_diffs,elastic_search,full_codequality_report,group_activity_analytics,group_bulk_edit,group_webhooks,issuable_default_templates,issue_weights,iterations,ldap_group_sync,member_lock,merge_request_approvers,milestone_charts,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,multiple_merge_request_reviewers,project_merge_request_analytics,protected_refs_for_users,push_rules,repository_mirrors,repository_size_limit,resource_access_token,seat_link,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,send_emails_from_admin_area,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,ci_secrets_management,cluster_agents,cluster_deployments,code_owner_approval_required,commit_committer_check,compliance_framework,custom_compliance_frameworks,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,cycle_analytics_for_groups,cycle_analytics_for_projects,db_load_balancing,default_branch_protection_restriction_in_groups,default_project_deletion_protection,disable_name_update_for_users,email_additional_text,epics,extended_audit_events,external_authorization_service_api_management,feature_flags_related_issues,file_locks,geo,generic_alert_fingerprinting,git_two_factor_enforcement,github_project_service_integration,group_allowed_email_domains,group_coverage_reports,group_forking_protection,group_ip_restriction,group_merge_request_analytics,group_merge_request_approval_settings,group_milestone_project_releases,group_project_templates,group_repository_analytics,group_saml,group_saml_group_sync,group_scoped_ci_variables,group_wikis,incident_sla,incident_metric_upload,ide_schema_config,issues_analytics,jira_issues_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,admin_merge_request_approvers_rules,merge_trains,metrics_reports,multiple_alert_http_integrations,multiple_approval_rules,multiple_group_issue_boards,multiple_iteration_cadences,object_storage,operations_dashboard,package_forwarding,pages_size_limit,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,smartcard_auth,swimlanes,type_of_work_analytics,minimal_access_role,unprotection_restrictions,ci_project_subscriptions,incident_timeline_view,oncall_schedules,escalation_policies,export_user_permissions,api_fuzzing,auto_rollback,cilium_alerts,cluster_image_scanning,external_status_checks,container_scanning,coverage_fuzzing,credentials_inventory,dast,dependency_scanning,devops_adoption,dora4_analytics,enforce_personal_access_token_expiration,enforce_ssh_key_expiration,enterprise_templates,environment_alerts,evaluate_group_level_compliance_pipeline,group_ci_cd_analytics,group_level_compliance_dashboard,group_level_devops_adoption,incident_management,inline_codequality,insights,instance_level_devops_adoption,issuable_health_status,jira_vulnerabilities_integration,jira_issue_association_enforcement,license_scanning,personal_access_token_expiration_policy,prometheus_alerts,pseudonymizer,quality_management,release_evidence_test_artifacts,report_approver_rules,requirements,sast,sast_custom_rulesets,sast_fp_reduction,secret_detection,security_dashboard,security_on_demand_scans,security_orchestration_policies,status_page,subepics,threat_monitoring,vulnerability_auto_fix,vulnerability_finding_signatures
CI_PROJECT_ID: 28515712
CI_PROJECT_NAME: fpcdab
CI_PROJECT_TITLE: fpcdab
CI_PROJECT_PATH: fpcprojects/fpcdab
CI_PROJECT_PATH_SLUG: fpcprojects-fpcdab
CI_PROJECT_NAMESPACE: fpcprojects
CI_PROJECT_ROOT_NAMESPACE: fpcprojects
CI_PROJECT_URL: https://gitlab.com/fpcprojects/fpcdab
CI_PROJECT_VISIBILITY: public
CI_PROJECT_REPOSITORY_LANGUAGES: pascal
CI_PROJECT_CLASSIFICATION_LABEL:
CI_DEFAULT_BRANCH: trunk
CI_CONFIG_PATH: .gitlab-ci.yml
CI_PAGES_DOMAIN: gitlab.io
CI_PAGES_URL: https://fpcprojects.gitlab.io/fpcdab
CI_REGISTRY: registry.gitlab.com
CI_REGISTRY_IMAGE: registry.gitlab.com/fpcprojects/fpcdab
CI_DEPENDENCY_PROXY_SERVER: gitlab.com:443
CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: gitlab.com:443/fpcprojects/dependency_proxy/containers
CI_API_V4_URL: https://gitlab.com/api/v4
CI_PIPELINE_IID: 6
CI_PIPELINE_SOURCE: push
CI_PIPELINE_CREATED_AT: 2021-08-22T23:00:01Z
CI_COMMIT_SHA: 29b9e6c8f0a3243143ea2f1783881b9253984ab4
CI_COMMIT_SHORT_SHA: 29b9e6c8
CI_COMMIT_BEFORE_SHA: c7614b9ddf5834e119ec4b992161f4092699ce82
CI_COMMIT_REF_NAME: trunk
CI_COMMIT_REF_SLUG: trunk
CI_COMMIT_BRANCH: trunk
CI_COMMIT_MESSAGE: * Windows and Linux build
CI_COMMIT_TITLE: * Windows and Linux build
CI_COMMIT_DESCRIPTION:
CI_COMMIT_REF_PROTECTED: true
CI_COMMIT_TIMESTAMP: 2021-08-23T00:59:56+02:00
CI_COMMIT_AUTHOR: Joost van der Sluis <joost@cnoc.nl>
CI_BUILD_REF: 29b9e6c8f0a3243143ea2f1783881b9253984ab4
CI_BUILD_BEFORE_SHA: c7614b9ddf5834e119ec4b992161f4092699ce82
CI_BUILD_REF_NAME: trunk
CI_BUILD_REF_SLUG: trunk
CI_RUNNER_ID: 9781130
CI_RUNNER_DESCRIPTION: aks_cnoc_jvdsluis_windows
CI_RUNNER_TAGS: windows
GITLAB_USER_ID: 9100338
GITLAB_USER_EMAIL: joost@cnoc.nl
GITLAB_USER_LOGIN: jvdsluis
GITLAB_USER_NAME: Joost van der Sluis
CI_DISPOSABLE_ENVIRONMENT: true
CI_RUNNER_VERSION: 14.0.0
CI_RUNNER_REVISION: 3b6f852e
CI_RUNNER_EXECUTABLE_ARCH: linux/amd64
Mounts:
/builds from repo (rw)
/logs-28515712-1524304017 from logs (rw)
/scripts-28515712-1524304017 from scripts (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4dhfs (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
repo:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
scripts:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: runner-gfbyl6sy-project-28515712-concurrent-0-scripts658n6
Optional: false
logs:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
default-token-4dhfs:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-4dhfs
Optional: false
QoS Class: BestEffort
Node-Selectors: kubernetes.io/arch=amd64
kubernetes.io/os=windows
node.kubernetes.io/windows-build=10.0.17763
Tolerations: kubernetes.azure.com/scalesetpriority=spot:NoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> Successfully assigned gitlab/runner-gfbyl6sy-project-28515712-concurrent-0rlfs6 to aksmspool00000b
Normal Pulled 10s kubelet, aksmspool00000b Container image "gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809" already present on machine
Normal Created 9s kubelet, aksmspool00000b Created container init-logs
Warning Failed 3s kubelet, aksmspool00000b Error: failed to start container "init-logs": Error response from daemon: container init-logs encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2)
Kubernetes-runner on Azure Kubernetes Service.
Everything can be found here: https://gitlab.com/fpcprojects/aks-runners
The projects and some builds here: https://gitlab.com/fpcprojects/fpcdab
config.toml contents
[[runners]]
shell = "pwsh"
executor = "kubernetes"
[runners.feature_flags]
FF_USE_POWERSHELL_PATH_RESOLVER = true
[runners.kubernetes]
helper_image = "gitlab/gitlab-runner-helper:x86_64-307aeab7-servercore1809"
poll_timeout = 3600
[runners.kubernetes.node_selector]
"kubernetes.io/arch" = "amd64"
"kubernetes.io/os" = "windows"
"node.kubernetes.io/windows-build" = "10.0.17763"
[runners.kubernetes.node_tolerations]
"kubernetes.azure.com/scalesetpriority=spot" = "NoSchedule"Used GitLab Runner version
Helm-charts that do not work: 0.30.0 and 0.31.0.
Possible fixes
Adapt the init-logs container so that it uses Powershell instead of bash? Not that with version 0.29.0 I do not see any sign of this init-logs container.