<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "type::bug" label: - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary Getting 404s from [Download the artifacts archive](https://docs.gitlab.com/ee/api/job_artifacts.html#download-the-artifacts-archive) API endpoint (both in CI jobs and from my workstation) after `n` other jobs (not the one we're querying for) have also run on the project. It seems to correlate to when the job we're looking for is no longer on the first page of the UI. It would be great if the Artifacts Archive API endpoint would fetch the latest matched artifact for a given job name + `:ref`, even if it wasn't the very last job run. Since this endpoint provides such great functionality that it does already exist in the Gitlab API, I probably don't need to convince you this is a valuable capability. However, how it currently works would limit us to only having a single job for a given repository, and it would also break if there was a failure (from what I've gathered from the other ticket). ### Steps to reproduce 1. Create a job (test job) that tries to fetch the artifact from its own previous job run using [Download the artifacts archive](https://docs.gitlab.com/ee/api/job_artifacts.html#download-the-artifacts-archive) 2. Run a different job (or jobs - control job(s)) enough times that the job you're testing is well into the second page of the the Artifacts UI 3. Run the "test job" again The [Download the artifacts archive](https://docs.gitlab.com/ee/api/job_artifacts.html#download-the-artifacts-archive) call will work until enough other jobs run that the "test job" latest run (on the artifacts page) doesn't appear until somewhere down page 2. ### Example Project <!--If possible, please create an example project here on GitLab.com that exhibits the problematic behavior, and link to it here in the bug report. If you are using an older version of GitLab, this will also determine whether the bug is fixed in a more recent version.--> I've recreated it in this public project https://gitlab.com/mschencksap1/mschencksap I've tried it both with JOB-TOKEN and PRIVATE-TOKEN auth headers. The JOB-TOKEN approach doesn't work but I expect that since this is a free-tier account. You can see a successful run here (4 runs in a row showing the accumulated history): - job run: https://gitlab.com/mschencksap1/mschencksap/-/jobs/8999776190 - artifact: https://gitlab.com/mschencksap1/mschencksap/-/jobs/9000350835/artifacts/external_file/test.txt Then I ran the "filler" stage 3 times, then the "test" stage again: https://gitlab.com/mschencksap1/mschencksap/-/jobs The very next run of the "test" stage shows it 404ing: - job run: https://gitlab.com/mschencksap1/mschencksap/-/jobs/9000648698 - artifact: https://gitlab.com/mschencksap1/mschencksap/-/jobs/9000648698/artifacts/external_file/test.txt What's also odd is that the call fails when I just remove the (not working) `testjobtoken` from the `.gitlab-ci.yml`. So I'm really not sure what's going on but this seems really flakey. ### What is the current _bug_ behavior? 404ing when the artifact is still available (even when I set `expire_in` to several weeks. The 404ing behavior can occur as soon as 10 minutes after the "test job" completes. ### What is the expected _correct_ behavior? I expect to still be able to fetch the latest artifact of the given job, as long as the artifact hasn't expired. Just like I can fetch it via [get artifact by job id](https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts) ### Relevant logs and/or screenshots <!--Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.--> ### Output of checks <!--If you are reporting a bug on GitLab.com, uncomment below--> <!--This bug happens on GitLab.com--> <!--and uncomment below if you have /label privileges--> <!--/label ~"reproduced on GitLab.com"--> <!--or follow up with an issue comment of `@gitlab-bot label ~"reproduced on GitLab.com"` if you do not--> #### Results of GitLab environment info <!--Input any relevant GitLab environment information if needed.--> <details> <summary>Expand for output related to GitLab environment info</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: \`sudo gitlab-rake gitlab:env:info\`) (For installations from source run and paste the output of: \`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production\`) </pre> </details> #### Results of GitLab application Check <!--Input any relevant GitLab application check information if needed.--> <details> <summary>Expand for output related to the GitLab application check</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:check SANITIZE=true`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`) (we will only investigate if the tests are passing) </pre> </details> ### Possible fixes <!--If you can, link to the line of code that might be responsible for the problem.--> <!--If you don't have /label privileges, follow up with an issue comment of `@gitlab-bot label ~"type::bug"`-->