ci: use cache properly

The cache has not been used properly due to a few of reasons:

1. the $GOPATH variable is not being read properly by the job as the runner has no knowledge of this variable value. This turns into the cache upload stage to fail because /pkg/mod/ does not exist.
2. The cache files need to be relative to the working directory!
3. The default cache policy is pull-push, which first downloads the cache and then uploads it again.
4. Most jobs do not depend on any other jobs ( i.e. needs: []), especially the tests, when this is combined with the pull-push cache policy, it means every job needs to work inefficiently by downloading an empty cache, downloading all the modules it needs and then failing to upload the cache due to reason 1.
5. The cache key is the branch name, this is fine but we can be more efficient by using files as keys.

To fix this:

1. We define the default cache policy to be pull
2. Change the cache key to use file names, such as go.mod & go.sum and a prefix by $GO_VERSION
3. We add a new job modules:download and set the cache policy to push
4. We make all tests depend on the modules:download job.
5. Each job downloads the cache for its own go version

You can see from the job logs that we no longer need to download the modules 🎉

Fixes ci: cache is not being used properly (#872 - closed)