Skip to content

Fix check_modules command

Steve Xuereb requested to merge 27229-check-modules-job-not-failing into master

What does this MR do?

The check_modules should check if there are any changes not committed in go.sum or in the vendor directory. However, this requires to have no changes in go.sum, if there are changes this wouldn't work because we compare the diff before the command and after the command.

The variable OUR_PACKAGES is running go list which runs go mod tidy behind the scenes. This results into a dirty go.sum and when we run check_modules the changes are already present so we don't see any new changes.

Remove the usage of OUR_PACKAGES since it's no longer needed, we don't have to filter out the vendor directory anymore and using ./... is enough in newer versions of Go.

This will also make our Makefile faster, because we don't have to run go list for every command.

Make sure we reset the go.sum and the vendor/ dir because we run the git diff command to prevent any failures like these in the future.

Why was this MR needed?

check module isn't failing when it should.

What's the best way to test this MR?

make check_modules

  1. Check out the current master SHA: git checkout 11e965397800fa0a3cd457953fc7e18568651e95
  2. Run make check_modules no failures.
  3. Run go mod tidy
  4. You should see a change in go.sum which means that the check_modules command should fail
  5. Check out specific SHA to fix check_modules command: git checkout 1a6b666a0842de3fdb958a33058e6fa32cbcf36e.
  6. Run make check_modules. You should get a failure now, as shown in https://gitlab.com/gitlab-org/gitlab-runner/-/jobs/853478892

make simple-test

  1. Run make simple-test

  2. You should see the go test command output all the packages from go list ./..

    make simple-test
    go test gitlab.com/gitlab-org/gitlab-runner gitlab.com/gitlab-org/gitlab-runner/apps/gitlab-runner-helper     gitlab.com/gitlab-org/gitlab-runner/cache gitlab.com/gitlab-org/gitlab-runner/cache/azure gitlab.com/gitlab-    org/gitlab-runner/cache/gcs gitlab.com/gitlab-org/gitlab-runner/cache/s3 gitlab.com/gitlab-org/gitlab-    runner/cache/test gitlab.com/gitlab-org/gitlab-runner/commands gitlab.com/gitlab-org/gitlab-runner/commands/helpers gitlab.com/gitlab-org/gitlab-runner/commands/helpers/archive gitlab.com/gitlab-org/gitlab-runner/commands/helpers/archive/fastzip gitlab.com/gitlab-org/gitlab-runner/commands/helpers/archive/gziplegacy gitlab.com/gitlab-org/gitlab-runner/commands/helpers/archive/raw gitlab.com/gitlab-org/gitlab-runner/commands/helpers/archive/ziplegacy gitlab.com/gitlab-org/gitlab-runner/common gitlab.com/gitlab-org/gitlab-runner/common/buildtest gitlab.com/gitlab-org/gitlab-runner/executors gitlab.com/gitlab-org/gitlab-runner/executors/custom gitlab.com/gitlab-org/gitlab-runner/executors/custom/api gitlab.com/gitlab-org/gitlab-runner/executors/custom/command gitlab.com/gitlab-org/gitlab-runner/executors/docker gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/labels gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/networks gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/volumes gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/volumes/parser gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/volumes/permission gitlab.com/gitlab-org/gitlab-runner/executors/docker/internal/wait gitlab.com/gitlab-org/gitlab-runner/executors/docker/machine gitlab.com/gitlab-org/gitlab-runner/executors/kubernetes gitlab.com/gitlab-org/gitlab-runner/executors/parallels gitlab.com/gitlab-org/gitlab-runner/executors/shell gitlab.com/gitlab-org/gitlab-runner/executors/ssh gitlab.com/gitlab-org/gitlab-runner/executors/virtualbox gitlab.com/gitlab-org/gitlab-runner/helpers gitlab.com/gitlab-org/gitlab-runner/helpers/archives gitlab.com/gitlab-org/gitlab-runner/helpers/certificate gitlab.com/gitlab-org/gitlab-runner/helpers/cli gitlab.com/gitlab-org/gitlab-runner/helpers/container/helperimage gitlab.com/gitlab-org/gitlab-runner/helpers/container/services gitlab.com/gitlab-org/gitlab-runner/helpers/container/services/test gitlab.com/gitlab-org/gitlab-runner/helpers/container/windows gitlab.com/gitlab-org/gitlab-runner/helpers/dns gitlab.com/gitlab-org/gitlab-runner/helpers/dns/test gitlab.com/gitlab-org/gitlab-runner/helpers/docker gitlab.com/gitlab-org/gitlab-runner/helpers/docker/auth gitlab.com/gitlab-org/gitlab-runner/helpers/docker/errors gitlab.com/gitlab-    org/gitlab-runner/helpers/docker/test gitlab.com/gitlab-org/gitlab-runner/helpers/featureflags     gitlab.com/gitlab-org/gitlab-runner/helpers/gitlab_ci_yaml_parser gitlab.com/gitlab-org/gitlab-runner/helpers/parallels gitlab.com/gitlab-org/gitlab-runner/helpers/path gitlab.com/gitlab-org/gitlab-runner/helpers/process gitlab.com/gitlab-org/gitlab-runner/helpers/prometheus gitlab.com/gitlab-org/gitlab-runner/helpers/retry gitlab.com/gitlab-org/gitlab-runner/helpers/secrets gitlab.com/gitlab-org/gitlab-runner/helpers/secrets/resolvers/vault gitlab.com/gitlab-org/gitlab-runner/helpers/sentry gitlab.com/gitlab-org/gitlab-runner/helpers/service gitlab.com/gitlab-org/gitlab-runner/helpers/service/mocks gitlab.com/gitlab-org/gitlab-runner/helpers/ssh gitlab.com/gitlab-org/gitlab-runner/helpers/test gitlab.com/gitlab-org/gitlab-runner/helpers/timeperiod gitlab.com/gitlab-org/gitlab-runner/helpers/tls gitlab.com/gitlab-org/gitlab-runner/helpers/tls/ca_chain gitlab.com/gitlab-org/gitlab-runner/helpers/trace gitlab.com/gitlab-org/gitlab-runner/helpers/url gitlab.com/gitlab-org/gitlab-runner/helpers/vault gitlab.com/gitlab-org/gitlab-runner/helpers/vault/auth_methods gitlab.com/gitlab-org/gitlab-runner/helpers/vault/auth_methods/jwt gitlab.com/gitlab-org/gitlab-runner/helpers/vault/internal/registry gitlab.com/gitlab-org/gitlab-runner/helpers/vault/secret_engines gitlab.com/gitlab-org/gitlab-runner/helpers/vault/secret_engines/kv_v1 gitlab.com/gitlab-org/gitlab-runner/helpers/vault/secret_engines/kv_v2 gitlab.com/gitlab-org/gitlab-runner/helpers/vault/service gitlab.com/gitlab-org/gitlab-runner/helpers/virtualbox gitlab.com/gitlab-org/gitlab-runner/log gitlab.com/gitlab-org/gitlab-runner/log/test gitlab.com/gitlab-org/gitlab-runner/network     gitlab.com/gitlab-org/gitlab-runner/referees gitlab.com/gitlab-org/gitlab-runner/scripts/pull-images-for-tests     gitlab.com/gitlab-org/gitlab-runner/scripts/update-feature-flags-docs gitlab.com/gitlab-org/gitlab-    runner/session gitlab.com/gitlab-org/gitlab-runner/session/proxy gitlab.com/gitlab-org/gitlab-    runner/session/terminal gitlab.com/gitlab-org/gitlab-runner/shells gitlab.com/gitlab-org/gitlab-    runner/shells/shellstest -cover -ldflags "-X gitlab.com/gitlab-org/gitlab-runner/common.NAME=gitlab-runner -X         gitlab.com/gitlab-org/gitlab-runner/common.VERSION=13.7.0~beta.328.g1a6b666a -X gitlab.com/gitlab-org/gitlab-    runner/common.REVISION=1a6b666a -X gitlab.com/gitlab-org/gitlab-runner/common.BUILT=2020-11-17T07:44:33+0000 -X 
    gitlab.com/gitlab-org/gitlab-runner/common.BRANCH=27229-check-modules-job-not-failing -s -w"

make git1.7-test

  1. Run make git1.7-test
  2. You should only see gitlab.com/gitlab-org/gitlab-runner/executors/shell and gitlab.com/gitlab-org/gitlab-runner/shells run, just like the master branch job

make parallel_test_prepare

  1. Run make parallel_test_prepare
  2. Download the latest artifacts from the last pipeline on master.
  3. Compare the two files and make sure there is no difference: diff testsdefinitions.txt ~/Downloads/testsdefinitions.txt

What are the relevant issue numbers?

Closes #27229 (closed)

Edited by Steve Xuereb

Merge request reports