Skip to content

Run CustomersDot end-to-end tests per CustomersDot MR

Problem

We do not run fulfillment end-to-end tests against CustomersDot per-MR. This leads to potential regressions and defects making it into the application.

Proposal

Run fulfillment end-to-end tests that exist within the CustomersDot/qa project, per MR against an ephemeral CustomersDot application spun up in Docker.

Current Approach

image

CustomersDot Docker container

We use docker-compose setup to create all necessary services.

The CDot image defined in Dockerfile contains dependencies to provision the app. App code is mounted through a docker volume that doesn't get copied into the image. This approach is great for local development by being able to liveload changes without rebuilding the image. However in CI, this leads to the app provision step being necessary for all parallel jobs that require the CDot container. Although the provision step takes about 4 mins, but we should look to making the jobs more sufficient.

CDot-GDK image

Currently a pre-configured GDK image is published every 24 hours. This GDK image can serve as a SaaS environment required for CDot SSO and SaaS E2E tests. The image is configured to work with a default CustomersDot development environment, with a url of http://customers:5000.

CI/CD setup

We use a predefined set of environment variables to pass in required credentials/secrets that's used for communication between services (Zuora, ZSim etc). A manual job is created in MR pipeline to run any E2E tests that are tagged with :run_in_mr. For each E2E job, due to the limitation in CDot image, we need to run dip provision to provision the CDot app first.

The job is running on e2e runners with FF_NETWORK_PER_BUILD set to false.

Iteration 1

  • Refined docker-compose setup to have CustomersDot app successfully run in container in CI
  • Created and publish the cdot-gdk image that's ready to use with CDot container
  • Setup e2e test job in CDot's MR pipeline that successfully run SaaS E2E tests

Iteration 2

  • Create and publish CDot image per MR with app code built into the image, tagged with feature. Consolidate web, webpacker and sidekiq into single container in CI runs
  • Move away from docker-compose structure in CI, and move the e2e jobs to docker runners in CI job, and use GitLab's built-in services with FF_NETWORK_PER_BUILD=true.
  • Discuss options with TE team to decide the strategy for SM E2E tests.
    1. Using existing GDK image to serve as both SaaS and SM instance
    2. Continue to use gitlab-ee production image
Edited by Chloe Liu