Add check to ensure data.results is not empty
What does this MR do and why?
While investigating #378174 (closed) I noted that we do not do any type of assertion on the data.results
array to
ensure there's actually data to validate against.
In some situations we may get an empty result array back - {"status":"success","data":{"resultType":"vector","result":[]}}
and in these cases this test should error out to indicate the environment isn't as expected.
- on a brand new instance where no requests have happened
- if the metric stops updating for some reason
- if we misconfigure prometheus
In the mentioned E2E failures - the issue is a misconfiguration - and the spec 'reads from each node'
should have also failed as a result.
However because it's an empty array - it manages to slip through and appears to be a passing test -- this is incorrect and this spec should have also failed in addition to the 'when a node is unhealthy'
spec.
How to set up and validate locally
- For a failing job see https://gitlab.com/gitlab-org/gitlab/-/jobs/3359580967 test failed as expected
- This fails with builds of gitlab-qa <8.13.1
- For a passing job, see https://gitlab.com/gitlab-org/gitlab/-/jobs/3372443890
- This passes with builds of gitlab-qa >=8.13.1 due to the fix Fix bug in config for prometheus_listen_addr (gitlab-qa!1087 - merged)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %15.7
added Quality bugfunctional typebug labels
assigned to @john.mcdonnell
- A deleted user
added QA label
- Resolved by Mark Lapierre
1 Message CHANGELOG missing: If you want to create a changelog entry for GitLab FOSS, add the
Changelog
trailer to the commit message you want to add to the changelog.If you want to create a changelog entry for GitLab EE, also add the
EE: true
trailer to your commit message.If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer QA Careem Ahamed (
@cahamed
) (UTC+8, 1 hour ahead of@john.mcdonnell
)Mark Lapierre (
@mlapierre
) (UTC+11, 4 hours ahead of@john.mcdonnell
)To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
DangerEdited by Ghost User
Allure report
allure-report-publisher
generated test report!e2e-package-and-test:
test report for ba4192cdexpand test summary
+---------------------------------------------------------------------------+ | suites summary | +----------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +----------------------+--------+--------+---------+-------+-------+--------+ | Manage | 357 | 0 | 27 | 3 | 384 | ❗ | | Create | 818 | 1 | 27 | 6 | 846 | ❌ | | Plan | 312 | 0 | 0 | 0 | 312 | ✅ | | Verify | 229 | 0 | 42 | 0 | 271 | ✅ | | Version sanity check | 0 | 0 | 7 | 7 | 7 | ➖ | | Package | 144 | 0 | 37 | 15 | 181 | ❗ | | Govern | 209 | 1 | 0 | 0 | 210 | ❌ | | Systems | 18 | 0 | 1 | 1 | 19 | ❗ | | Fulfillment | 12 | 0 | 75 | 0 | 87 | ✅ | | Secure | 29 | 1 | 10 | 5 | 40 | ❌ | | Release | 30 | 0 | 0 | 0 | 30 | ✅ | | Configure | 1 | 0 | 15 | 0 | 16 | ✅ | | ModelOps | 0 | 0 | 5 | 0 | 5 | ➖ | | Analytics | 11 | 0 | 0 | 0 | 11 | ✅ | | GitLab Metrics | 2 | 0 | 1 | 1 | 3 | ❗ | | Data Stores | 13 | 0 | 1 | 0 | 14 | ✅ | | Monitor | 5 | 0 | 0 | 0 | 5 | ✅ | +----------------------+--------+--------+---------+-------+-------+--------+ | Total | 2190 | 3 | 248 | 38 | 2441 | ❌ | +----------------------+--------+--------+---------+-------+-------+--------+
e2e-review-qa:
test report for ba4192cdexpand test summary
+-----------------------------------------------------------------------------------------+ | suites summary | +------------------------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------------------------+--------+--------+---------+-------+-------+--------+ | Manage | 39 | 0 | 4 | 2 | 43 | ❗ | | Create | 28 | 0 | 1 | 0 | 29 | ✅ | | Verify | 12 | 0 | 1 | 0 | 13 | ✅ | | Plan | 49 | 0 | 1 | 0 | 50 | ✅ | | Govern | 10 | 0 | 5 | 1 | 15 | ❗ | | Configure | 0 | 0 | 1 | 0 | 1 | ➖ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | | Feature flag handler sanity checks | 9 | 0 | 0 | 0 | 9 | ✅ | | Version sanity check | 0 | 0 | 1 | 0 | 1 | ➖ | +------------------------------------+--------+--------+---------+-------+-------+--------+ | Total | 147 | 0 | 15 | 3 | 162 | ❗ | +------------------------------------+--------+--------+---------+-------+-------+--------+
Edited by Ghost Useradded 403 commits
-
cb20dc16...98b098b2 - 402 commits from branch
master
- ba4192cd - Add check to ensure data.results is not empty
-
cb20dc16...98b098b2 - 402 commits from branch
requested review from @cahamed
requested review from @mlapierre
@mlapierre, did you forget to run a pipeline before you merged this work? Based on our code review process, if the latest pipeline finished more than 2 hours ago, you should:
- Ensure the merge request is not in Draft status.
- Start a pipeline (especially important for Community contribution merge requests).
- Set the merge request to merge when pipeline succeeds.
This is a guideline, not a rule. Please consider replying to this comment for transparency.
This message was generated automatically. You're welcome to improve it.
mentioned in commit f4c87752
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-staging label and removed workflowproduction label
added workflowpost-deploy-db-production label and removed workflowpost-deploy-db-staging label
added releasedcandidate label
added releasedpublished label and removed releasedcandidate label