crash on Merge Request Pipeline
I run the step-runner inside a docker container with a local Gitlab v17.8.1-ee
The step-runner is compiled from main branch with latest commit 23145804 The only modification was to link against our internal Gitlab Server to circumvent #161 (closed)
It is working correctly for normal CI Triggered Jobs. But it seems to crash when it is running in a Merge Request Pipeline. I wrote this wrapper script around step-runner:
#!/bin/sh
# Dump environment variables to a file
env
# Log all parameters
echo "Called with parameters: $@"
# Execute the original step-runner with the same parameters
/step-runner "$@"
And this is the anonymized output
$ step-runner ci
CI_SERVER_REVISION=a303ad659b5
FF_USE_WINDOWS_LEGACY_PROCESS_STRATEGY=false
GITLAB_USER_LOGIN=group_179_bot_28d6e8b47d856f098f71eeedc4f10ad4
CI_COMMIT_SHORT_SHA=4da35bb4
FF_SCRIPT_SECTIONS=false
CI_SERVER_VERSION_PATCH=1
FF_POSIXLY_CORRECT_ESCAPES=false
CI_DEPENDENCY_PROXY_PASSWORD=[MASKED]
CI_DEPENDENCY_PROXY_SERVER=internal.sub.example.group:443
CI_MERGE_REQUEST_TARGET_BRANCH_PROTECTED=true
LANGUAGE=en
GIT_SUBMODULE_DEPTH=0
FF_USE_DUMB_INIT_WITH_KUBERNETES_EXECUTOR=false
CI_PROJECT_NAME=ci-test
FF_USE_LEGACY_S3_CACHE_ADAPTER=false
CI_SERVER_PROTOCOL=https
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY=false
CI_RUNNER_REVISION=690ce25c
FF_LOG_IMAGES_CONFIGURED_FOR_JOB=false
CI=true
HOSTNAME=runner-t2b29nqv-project-188-concurrent-0
CI_COMMIT_DESCRIPTION=
CI_JOB_STAGE=merge-request
CI_PROJECT_DESCRIPTION=
CI_SERVER_VERSION=17.8.1-ee
CI_INSTANCE_TOKEN=[MASKED]
CI_MERGE_REQUEST_SOURCE_PROJECT_ID=188
SHLVL=1
CI_DEPENDENCY_PROXY_USER=gitlab-ci-token
FF_DISABLE_POWERSHELL_STDIN=false
HOME=/root
FF_DISABLE_AUTOMATIC_TOKEN_ROTATION=false
CI_PROJECT_ROOT_NAMESPACE=project
FF_CLEAN_UP_FAILED_CACHE_EXTRACT=false
OLDPWD=/
FF_NETWORK_PER_BUILD=false
COMMIT_MESSAGE_PREFIX=CI File Upload:
CI_COMMIT_REF_NAME=test
CI_SERVER_HOST=internal.sub.example.group
CI_JOB_ID=28001
CI_RUNNER_VERSION=17.8.3
CI_MERGE_REQUEST_TARGET_BRANCH_SHA=
CI_PIPELINE_SOURCE=merge_request_event
FF_RESOLVE_FULL_TLS_CHAIN=false
FF_SKIP_NOOP_BUILD_STAGES=true
CI_MERGE_REQUEST_SOURCE_BRANCH_NAME=test
CI_MERGE_REQUEST_SOURCE_PROJECT_PATH=path/to/this/repo/ci-test
FF_USE_WINDOWS_JOB_OBJECT=false
CI_DEFAULT_BRANCH=main
CI_BUILDS_DIR=/builds
CI_SERVER_VERSION_MAJOR=17
FF_USE_FASTZIP=false
CI_REGISTRY_PASSWORD=[MASKED]
CI_COMMIT_REF_PROTECTED=false
CI_SERVER_URL=https://internal.sub.example.group
CI_TEMPLATE_REGISTRY_HOST=registry.gitlab.com
CI_MERGE_REQUEST_DIFF_ID=3904
STEPS=[{"name":"create_merge_request","step":"gitlab-ci-token:${{job.CI_JOB_TOKEN}}@internal.sub.example.group/path/to/this/roles/steps/create-merge-request@main"}]
GITLAB_FEATURES=audit_events,blocked_issues,blocked_work_items,board_iteration_lists,code_owners,code_review_analytics,full_codequality_report,group_activity_analytics,group_bulk_edit,issuable_default_templates,issue_weights,iterations,ldap_group_sync,merge_request_approvers,milestone_charts,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,multiple_merge_request_reviewers,project_merge_request_analytics,protected_refs_for_users,push_rules,resource_access_token,seat_link,seat_usage_quotas,pipelines_usage_quotas,transfer_usage_quotas,product_analytics_usage_quotas,wip_limits,zoekt_code_search,description_diffs,send_emails_from_admin_area,repository_size_limit,maintenance_mode,scoped_issue_board,contribution_analytics,group_webhooks,member_lock,elastic_search,repository_mirrors,ai_chat,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_secrets_management,ci_pipeline_cancellation_restrictions,cluster_agents_ci_impersonation,cluster_agents_user_impersonation,cluster_deployments,code_owner_approval_required,code_suggestions,commit_committer_check,commit_committer_name_check,compliance_framework,custom_compliance_frameworks,custom_fields,custom_file_templates,custom_project_templates,cycle_analytics_for_groups,cycle_analytics_for_projects,db_load_balancing,default_branch_protection_restriction_in_groups,default_project_deletion_protection,delete_unconfirmed_users,dependency_proxy_for_packages,disable_extensions_marketplace_for_enterprise_users,disable_name_update_for_users,disable_personal_access_tokens,domain_verification,epic_colors,epics,extended_audit_events,external_authorization_service_api_management,feature_flags_code_references,file_locks,geo,generic_alert_fingerprinting,git_two_factor_enforcement,group_allowed_email_domains,group_coverage_reports,group_forking_protection,group_level_compliance_dashboard,group_milestone_project_releases,group_project_templates,group_repository_analytics,group_saml,group_scoped_ci_variables,ide_schema_config,incident_metric_upload,instance_level_scim,jira_issues_integration,ldap_group_sync_filter,linked_items_epics,merge_request_performance_metrics,admin_merge_request_approvers_rules,merge_trains,metrics_reports,multiple_alert_http_integrations,multiple_approval_rules,multiple_group_issue_boards,object_storage,microsoft_group_sync,operations_dashboard,package_forwarding,pages_size_limit,pages_multiple_versions,productivity_analytics,project_aliases,protected_environments,reject_non_dco_commits,reject_unsigned_commits,related_epics,remote_development,saml_group_sync,service_accounts,scoped_labels,smartcard_auth,ssh_certificates,swimlanes,target_branch_rules,type_of_work_analytics,minimal_access_role,unprotection_restrictions,ci_project_subscriptions,incident_timeline_view,oncall_schedules,escalation_policies,zentao_issues_integration,coverage_check_approval_rule,issuable_resource_links,group_protected_branches,group_level_merge_checks_setting,oidc_client_groups_claim,disable_deleting_account_for_users,disable_private_profiles,group_saved_replies,requested_changes_block_merge_request,project_saved_replies,default_roles_assignees,ci_component_usages_in_projects,branch_rule_squash_options,group_ip_restriction,issues_analytics,password_complexity,group_wikis,email_additional_text,custom_file_templates_for_namespace,incident_sla,export_user_permissions,cross_project_pipelines,feature_flags_related_issues,merge_pipelines,ci_cd_projects,github_integration
ANSIBLE_VERSION=2.15.5
CI_PROJECT_ID=188
CI_MERGE_REQUEST_SOURCE_BRANCH_PROTECTED=false
CI_COMMIT_SHA=4da35bb41047add29ff1ce4f81569e1fbe4ea7d4
GITLAB_CI=true
FF_USE_LEGACY_GCS_CACHE_ADAPTER=false
FF_RETRIEVE_POD_WARNING_EVENTS=true
CI_CONCURRENT_ID=0
CI_REGISTRY_USER=gitlab-ci-token
FF_USE_DOCKER_AUTOSCALER_DIAL_STDIO=true
CI_SERVER_PORT=443
CI_MERGE_REQUEST_SQUASH_ON_MERGE=true
FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR=false
CI_PROJECT_PATH=path/to/this/repo/ci-test
CI_MERGE_REQUEST_PROJECT_URL=https://internal.sub.example.group/path/to/this/repo/ci-test
CI_PROJECT_DIR=/builds/path/to/this/repo/ci-test
CI_PROJECT_NAMESPACE=path/to/this/repo
FF_WAIT_FOR_POD_TO_BE_REACHABLE=false
FF_ENABLE_JOB_CLEANUP=false
CI_COMMIT_TIMESTAMP=2025-03-06T10:35:25+00:00
_=/usr/local/bin/step-runner
CI_INSTANCE_USER=group_179_bot_28d6e8b47d856f098f71eeedc4f10ad4
CI_SERVER_NAME=GitLab
FF_USE_DIRECT_DOWNLOAD=true
CI_MERGE_REQUEST_DESCRIPTION=
CI_JOB_TOKEN=[MASKED]
FF_USE_DYNAMIC_TRACE_FORCE_SEND_INTERVAL=false
CI_NODE_TOTAL=1
SSH_PUBLIC_KEY=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDmNWGYPfyDhfKtlisEH2hObnbPUXWRAI/DDt/zJ6rfxR6LkluK8OTUUNFUIFjl3O4RMOhrebsZPVV8dXdlHn83okQftwvXqbXzUUY0kpV/zp72y6QmpXHk/xT5a+zNeWgLTlIiTPlSk3U1+he2zs45EkPkrEKealvjrCG0DAHmCSN0RA/ymrmtxeyVtGFeKbXQ6Wc0kEzk8WNCETmrdi8PAbmesUgJzdJQO6q/aVljQmoSySBwdQ+4vFKNXEMrVEam3Ti+2n5b0sCxYRYvbb3U4Zkaclhx0AfgpPhIDBqKHNfylGa67Sn1Sf8HUQJUBnevmL2AHMwuVvH8tk+P2pJ0Qtw5MnSbzbVJ/LNBZa4kjLP1l3HOASaiNmDU9jWfJA5UNPdioFZg4ZAUT5YhtMxSHbydMeBA/FlXL/9U9RGyw0tC/b6y6FiAnpwLvJQfwprh/i8LvWA0sOhNyXs90eRN7QmhC3jjodnknM/gbNS92EIAtJpOX4kY3pTxhugeGt2uZyZJF74tSz0REjnj3glykHRj/Q5rrNxW8cx+k69TcNPzD9bhglx1WPAWRoJY7W8B8st0MTNcxFDE2HeD6KKZrsSDEFIFSLOox/xi7VXOhb5B22y6R1e9fdxDzmRD5qNBOKhtWc5vVCfk3Z58A3vJBMVajhJGbtsZFhxTr6oocQ== Apache-Reverse-Proxy-Deployment-Key
CI_PROJECT_NAMESPACE_ID=179
CI_MERGE_REQUEST_SOURCE_BRANCH_SHA=
CI_PIPELINE_CREATED_AT=2025-03-06T11:07:52Z
CI_JOB_NAME_SLUG=create-mr
RUNNER_TEMP_PROJECT_DIR=/builds/path/to/this/repo/ci-test.tmp
GIT_STRATEGY=clone
CI_CONCURRENT_PROJECT_ID=0
CI_MERGE_REQUEST_IID=2
CI_PIPELINE_URL=https://internal.sub.example.group/path/to/this/repo/ci-test/-/pipelines/17459
FF_KUBERNETES_HONOR_ENTRYPOINT=false
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CI_RUNNER_DESCRIPTION=Adappp033 limited to Reverse Proxy Group
CI_SERVER_VERSION_MINOR=8
CI_JOB_STARTED_AT=2025-03-06T11:08:14Z
SSH_PRIVATE_KEY_PW=[MASKED]
GITLAB_USER_EMAIL=group_179_bot_28d6e8b47d856f098f71eeedc4f10ad4@noreply.internal.sub.example.group
FF_USE_NEW_BASH_EVAL_STRATEGY=false
FF_DISABLE_UMASK_FOR_KUBERNETES_EXECUTOR=false
CI_PROJECT_VISIBILITY=internal
CI_COMMIT_TITLE=test
GIT_SUBMODULE_STRATEGY=none
CI_PROJECT_TITLE=ci-test
CI_SERVER=yes
FF_USE_GIT_BUNDLE_URIS=true
FF_USE_NATIVE_STEPS=false
CI_MERGE_REQUEST_EVENT_TYPE=detached
DISPLAY=1
FF_TIMESTAMPS=false
LANG=en_US.UTF-8
CI_MERGE_REQUEST_ID=3899
BOT_EMAIL=gitlab-runner-bot@example.com
CI_MERGE_REQUEST_APPROVED=true
CI_PROJECT_REPOSITORY_LANGUAGES=jinja,shell,python
FF_SET_PERMISSIONS_BEFORE_CLEANUP=true
FF_PRINT_POD_EVENTS=false
CI_SERVER_FQDN=internal.sub.example.group
CI_COMMIT_AUTHOR=Marco Hald <marco.hald@example.com>
CI_RUNNER_SHORT_TOKEN=t2_b29nqv
CI_JOB_IMAGE=docker.devops.example.group/project/ansible-revproxy-image:latest
CI_PIPELINE_NAME=
CI_JOB_TIMEOUT=3600
CI_REPOSITORY_URL=https://gitlab-ci-token:[MASKED]@internal.sub.example.group/path/to/this/repo/ci-test.git
GITLAB_ENV=/builds/path/to/this/repo/ci-test.tmp/gitlab_runner_env
CI_PROJECT_CLASSIFICATION_LABEL=
CI_INSTANCE_MAIL=group_179_bot_28d6e8b47d856f098f71eeedc4f10ad4@noreply.internal.sub.example.group
FF_USE_POWERSHELL_PATH_RESOLVER=false
CI_API_GRAPHQL_URL=https://internal.sub.example.group/api/graphql
FF_GIT_URLS_WITHOUT_TOKENS=false
CI_RUNNER_ID=1
CI_API_V4_URL=https://internal.sub.example.group/api/v4
CI_MERGE_REQUEST_SOURCE_PROJECT_URL=https://internal.sub.example.group/path/to/this/repo/ci-test
CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX=internal.sub.example.group:443/path/to/this/repo/dependency_proxy/containers
GITLAB_USER_NAME=INSTANCE_TOKEN
CI_MERGE_REQUEST_PROJECT_ID=188
CI_PIPELINE_IID=188
CI_RUNNER_EXECUTABLE_ARCH=linux/amd64
CI_MERGE_REQUEST_DESCRIPTION_IS_TRUNCATED=false
BOT_NAME=GitLab Runner Bot
FF_USE_POD_ACTIVE_DEADLINE_SECONDS=true
FF_USE_ADVANCED_POD_SPEC_CONFIGURATION=false
CI_SERVER_SHELL_SSH_HOST=internal.sub.example.group
CI_JOB_URL=https://internal.sub.example.group/path/to/this/repo/ci-test/-/jobs/28001
FF_USE_INIT_WITH_DOCKER_EXECUTOR=false
SSH_PRIVATE_KEY=/builds/path/to/this/repo/ci-test.tmp/SSH_PRIVATE_KEY
FF_TEST_FEATURE=false
CI_MERGE_REQUEST_REF_PATH=refs/merge-requests/2/head
CI_COMMIT_REF_SLUG=test
PWD=/builds/path/to/this/repo/ci-test
GIT_PULL_TOKEN=[MASKED]
CI_DISPOSABLE_ENVIRONMENT=true
LC_ALL=en_US.UTF-8
CI_RUNNER_TAGS=[]
CI_MERGE_REQUEST_PROJECT_PATH=path/to/this/repo/ci-test
FF_SECRET_RESOLVING_FAILS_IF_MISSING=true
CI_PIPELINE_ID=17459
CI_SERVER_TLS_CA_FILE=/builds/path/to/this/repo/ci-test.tmp/CI_SERVER_TLS_CA_FILE
CI_MERGE_REQUEST_TITLE=test : test
CI_PROJECT_PATH_SLUG=path-to-this-repo-ci-test
CI_COMMIT_BEFORE_SHA=0000000000000000000000000000000000000000
CI_CONFIG_PATH=.gitlab-ci.yml
CI_PROJECT_URL=https://internal.sub.example.group/path/to/this/repo/ci-test
CI_COMMIT_MESSAGE=test
GITLAB_USER_ID=45
CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX=internal.sub.example.group:443/project/dependency_proxy/containers
FF_ENABLE_BASH_EXIT_CODE_CHECK=false
CI_JOB_NAME=create mr
CI_OPEN_MERGE_REQUESTS=path/to/this/repo/ci-test!2
CI_JOB_STATUS=running
CI_MERGE_REQUEST_DIFF_BASE_SHA=8c465ba56a83ef9348b780c533525abf387e96cf
CI_SERVER_SHELL_SSH_PORT=22
GIT_DEPTH=0
CI_MERGE_REQUEST_TARGET_BRANCH_NAME=main
Called with parameters: ci
Step Runner version: e5ec7e46
See https://gitlab.com/gitlab-org/step-runner/-/blob/main/CHANGELOG.md for changes.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xb72a6b]
goroutine 1 [running]:
gitlab.com/gitlab-org/step-runner/pkg/runner.(*Environment).Values(0x0)
/builds/marcohald/step-runner/pkg/runner/environment.go:105 +0x4b
gitlab.com/gitlab-org/step-runner/pkg/runner.(*StepResultBuilder).ObserveEnv(...)
/builds/marcohald/step-runner/pkg/runner/step_result_builder.go:64
gitlab.com/gitlab-org/step-runner/pkg/runner.(*SequenceOfSteps).Run(0xc0003a8500, {0x13cda18, 0x198b9c0}, 0xc0003a8540)
/builds/marcohald/step-runner/pkg/runner/sequence_of_steps.go:41 +0x165
gitlab.com/gitlab-org/step-runner/pkg/runner.(*LazilyLoadedStep).Run(0xc0004361e0, {0x13cda18, 0x198b9c0}, 0xc0004f4540)
/builds/marcohald/step-runner/pkg/runner/lazily_loaded_step.go:57 +0x57b
gitlab.com/gitlab-org/step-runner/pkg/runner.(*SequenceOfSteps).Run(0xc0004f4500, {0x13cda18, 0x198b9c0}, 0xc0004f4540)
/builds/marcohald/step-runner/pkg/runner/sequence_of_steps.go:46 +0x269
gitlab.com/gitlab-org/step-runner/cmd/ci.run(0xc0004fc4b0)
/builds/marcohald/step-runner/cmd/ci/ci.go:107 +0x7d4
gitlab.com/gitlab-org/step-runner/cmd/ci.NewCmd.func1(0xc0004c6000?, {0xd58d2a?, 0x4?, 0xd58d2e?})
/builds/marcohald/step-runner/cmd/ci/ci.go:38 +0x17
github.com/spf13/cobra.(*Command).execute(0xc000298c08, {0x198b9c0, 0x0, 0x0})
/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:985 +0xaaa
github.com/spf13/cobra.(*Command).ExecuteC(0xc000298908)
/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(0xc000298908?)
/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041 +0x13
main.main()
/builds/marcohald/step-runner/main.go:21 +0x2c9