Skip to content

Resolve "Improve e2e test"

Michael Leopard requested to merge 240-improve-e2e-test into master

Overview

This MR covers improving the end to end test framework in congregate. The aspects covered in this MR are the following:

  • Modifying the do_all capability to run a full migration (users, groups, and projects) completely automatically
  • Updating the setup and execution of the end to end tests
  • Creating the base diff mechanisms for verifying the accuracy of the migrations
  • Creating a specific user diff for verifying the accuracy of migrated users
  • Creating a specific group diff for verifying the accuracy of migrated groups
    • The following endpoints are used in this diff:
      • /groups/:id
      • /groups/:id/variables
      • /groups/:id/members
  • Creating a specific project diff for verifying the accuracy of migrated projects
    • The following endpoints are used in this diff:
      • /projects/:id
      • /projects/:id/variables
      • /projects/:id/members
  • Generate a report containing the diff results and accuracy of the migration in an HTML/PDF format
  • Allowing the end to end tests to fail in the pipeline due to how we test the accuracy of the migration. If the accuracy is below a specified percentage, the test fails. This shouldn't fail the entire pipeline, though, because this would cause extensive scope creep. We now have a baseline for accuracy and it will get better over time.

This MR does not cover all of the GET endpoints we have for groups and projects. They will be addressed in followup issues once this is merged.

Note: this MR involves switching from pipenv to poetry and all the changes to files mentioning or using pipenv

Tasks

  • [ ]

Closes #240 (closed) #70 (closed)

Edited by Michael Leopard

Merge request reports