Fix testKubernetesPodEvents integration test
What does this MR do?
Before the fix, testKubernetesPodEvents constantly fails locally for the following reasons:
- Some lines were expected in exec mode instead of attach mode
- Some lines weren't printed when the cluster cache has already pulled the image
In order to improve the current test, assert.Regexp
is now used to handle the pulled vs already pulled images.
Why was this MR needed?
To fix a failing integration test.
What's the best way to test this MR?
No images cached
❯ go test -tags integration,kubernetes -timeout=60m -run "^TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents*" -v gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes
=== RUN TestRunIntegrationTestsWithFeatureFlag
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== PAUSE TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== CONT TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_true
test.go:24: Running with gitlab-runner development version (HEAD)
feature flags: FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY:true, FF_PRINT_POD_EVENTS:true
Preparing the "kubernetes" executor
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine:3.14.2 ...
Preparing environment
Using FF_USE_POD_ACTIVE_DEADLINE_SECONDS, the Pod activeDeadlineSeconds will be set to the job timeout: 2h0m0s...
Subscribing to Kubernetes Pod events...
Type Reason Message
Normal Scheduled Successfully assigned default/runner--project-0-concurrent-0-5yudjny4 to minikube
Normal Pulling Pulling image "alpine:3.14.2"
Normal Pulled Successfully pulled image "alpine:3.14.2" in 964ms (964ms including waiting). Image size: 2715479 bytes.
Normal Created Created container build
Normal Started Started container build
Normal Pulling Pulling image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest"
Normal Pulled Successfully pulled image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" in 1.752s (1.752s including waiting). Image size: 27734922 bytes.
Normal Created Created container helper
Normal Started Started container helper
Running on runner--project-0-concurrent-0-5yudjny4 via ratchade-MBP...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gitlab-org/ci-cd/gitlab-runner-pipeline-tests/gitlab-test/.git/
Created fresh repository.
Checking out 69b18e5e as detached HEAD (ref is main)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ echo Hello World
Hello World
Job succeeded
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_false
test.go:24: Running with gitlab-runner development version (HEAD)
feature flags: FF_PRINT_POD_EVENTS:true
Preparing the "kubernetes" executor
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine:3.14.2 ...
Using attach strategy to execute scripts...
Preparing environment
Using FF_USE_POD_ACTIVE_DEADLINE_SECONDS, the Pod activeDeadlineSeconds will be set to the job timeout: 2h0m0s...
Subscribing to Kubernetes Pod events...
Type Reason Message
Normal Scheduled Successfully assigned default/runner--project-0-concurrent-0-ub9kd28r to minikube
Normal Pulled Container image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" already present on machine
Normal Created Created container init-permissions
Normal Started Started container init-permissions
Normal Pulled Container image "alpine:3.14.2" already present on machine
Normal Created Created container build
Normal Started Started container build
Normal Pulled Container image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" already present on machine
Normal Created Created container helper
Normal Started Started container helper
Running on runner--project-0-concurrent-0-ub9kd28r via ratchade-MBP...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gitlab-org/ci-cd/gitlab-runner-pipeline-tests/gitlab-test/.git/
Created fresh repository.
Checking out 69b18e5e as detached HEAD (ref is main)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ echo Hello World
Hello World
Job succeeded
--- PASS: TestRunIntegrationTestsWithFeatureFlag (0.00s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY (13.81s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_true (7.84s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_false (5.98s)
PASS
ok gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes 14.442s
Images cached
❯ go test -tags integration,kubernetes -timeout=60m -run "^TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents*" -v gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes
=== RUN TestRunIntegrationTestsWithFeatureFlag
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== PAUSE TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== CONT TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_true
test.go:24: Running with gitlab-runner development version (HEAD)
feature flags: FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY:true, FF_PRINT_POD_EVENTS:true
Preparing the "kubernetes" executor
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine:3.14.2 ...
Preparing environment
Using FF_USE_POD_ACTIVE_DEADLINE_SECONDS, the Pod activeDeadlineSeconds will be set to the job timeout: 2h0m0s...
Subscribing to Kubernetes Pod events...
Type Reason Message
Normal Scheduled Successfully assigned default/runner--project-0-concurrent-0-6ijbnfop to minikube
Normal Pulled Container image "alpine:3.14.2" already present on machine
Normal Created Created container build
Normal Started Started container build
Normal Pulled Container image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" already present on machine
Normal Created Created container helper
Normal Started Started container helper
Running on runner--project-0-concurrent-0-6ijbnfop via ratchade-MBP...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gitlab-org/ci-cd/gitlab-runner-pipeline-tests/gitlab-test/.git/
Created fresh repository.
Checking out 69b18e5e as detached HEAD (ref is main)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ echo Hello World
Hello World
Job succeeded
=== RUN TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_false
test.go:24: Running with gitlab-runner development version (HEAD)
feature flags: FF_PRINT_POD_EVENTS:true
Preparing the "kubernetes" executor
WARNING: Namespace is empty, therefore assuming 'default'.
Using Kubernetes namespace: default
Using Kubernetes executor with image alpine:3.14.2 ...
Using attach strategy to execute scripts...
Preparing environment
Using FF_USE_POD_ACTIVE_DEADLINE_SECONDS, the Pod activeDeadlineSeconds will be set to the job timeout: 2h0m0s...
Subscribing to Kubernetes Pod events...
Type Reason Message
Normal Scheduled Successfully assigned default/runner--project-0-concurrent-0-duanfvlr to minikube
Normal Pulled Container image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" already present on machine
Normal Created Created container init-permissions
Normal Started Started container init-permissions
Normal Pulled Container image "alpine:3.14.2" already present on machine
Normal Created Created container build
Normal Started Started container build
Normal Pulled Container image "registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest" already present on machine
Normal Created Created container helper
Normal Started Started container helper
Running on runner--project-0-concurrent-0-duanfvlr via ratchade-MBP...
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gitlab-org/ci-cd/gitlab-runner-pipeline-tests/gitlab-test/.git/
Created fresh repository.
Checking out 69b18e5e as detached HEAD (ref is main)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ echo Hello World
Hello World
Job succeeded
--- PASS: TestRunIntegrationTestsWithFeatureFlag (0.00s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY (10.78s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_true (4.78s)
--- PASS: TestRunIntegrationTestsWithFeatureFlag/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY/testKubernetesPodEvents_FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY_false (5.99s)
PASS
ok gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes 11.401s
What are the relevant issue numbers?
Edited by Romuald Atchadé