QA E2E Pipeline Setup Questions
Hi, there
JH is trying to setup E2E test pipeline according to this doc and quality projects, after some spike we still have questions about the pipeline setup, it will be great to have someone answer our questions
ReCaptcha on Login
When we run QA tests on SaaS production, we found sometimes user login will be block by ReCaptcha after multiple logins, not sure how you guys solve this kind of issue? whitelist user? whitelist IP address?
Require Admin Account
How do you deal with test cases that run in SaaS production which require an admin account? From what I understand, in some tests, admin user may change application settings, which may change SaaS behavior.
Total time of Pipeline
According to this doc, we found smoke & reliable
tests are running every 4 hours. However it take us one hour to run smoke tests only, not sure whether it will be longer than 4 hours if we run smoke & reliable tests both.
Just curious how long the pipeline runs on your side? like smoke & reliable on production
, full on production
and so on.(test report only shows tests total time, not the pipeline total time). What's more, it will be great to share some specifications about the job runners, I guess our runner need more resources.
How to pass correct docker image version to pipeline
According to this doc, there are many pipeline and environments, different env has different release frequency, eg SaaS env
are more like update to latest, but release env
will update monthly. So then pipeline runs , we didn't figure out the way to pass docker image(contains E2E tests) version into pipeline for different scenarios.
gitlab-qa/.gitlab/ci
How to use yamls in We found there are some yamls in gitlab-qa/.gitlab/ci, not sure it is related with qa test pipeline. Which pipeline is using these files?
ops.gitlab.net account?
Do you think JH team member can have a lowest level permission account on https://ops.gitlab.net/, we just want to know more about pipeline setup, such as pipeline variables and so on.
Hi, @kwiebers please help us find the coordinator, thanks
cc: @qianzhangxa