Add nightly CI tests for production images
Clarification and motivation
We have Docker images in the container registry. It is used by our scripts/morley.sh
script that fetches an image and runs morley
. After !463 (merged) we will be able to run some automated tests for these images. However, AFAIU those tests won't fully test end2end user experience because they get an image based on the MORLEY_IMAGE
variable which normal people are not supposed to use.
For example, more than a year ago we had an issue when there was no image in the registry. We spotted it quite accidentally.
Acceptance criteria
There is a scheduled pipeline (can run once per day for example) that runs our tests for morley.sh
using it more or less the same way as regular users would do it. It should fetch the latest production image from registry.gitlab.com/morley-framework/morley
. If it fails, notifications are sent the same way as for other scheduled pipelines.