Consider renaming `gitlab-qa` to `gitlab-qa-orchestrator` since it's what it is

While working on !135 (merged), I start to have a clear view of all the pieces that we currently put under the generic "GitLab QA" term:

In https://gitlab.com/gitlab-org/gitlab-ce / https://gitlab.com/gitlab-org/gitlab-ee:

  • GitLab QA framework: A framework that allows developer to write end-to-end tests simply and efficiently. Located at gitlab-org/gitlab-ce@qa/qa/.
  • GitLab QA instance-level scenarios: RSpec scenarios that are using GitLab QA framework and Capybara to setup and perform individual end-to-end tests against a live GitLab instance. Located at gitlab-org/gitlab-ce@qa/qa/specs/features/.

In https://gitlab.com/gitlab-org/gitlab-qa:

  • GitLab QA orchestrator (this project): An orchestration tool that allows to run various QA test suites in a simple manner.
  • GitLab QA orchestrated scenarios: Scenarios where containers are started, configured and that run instance-level scenarios against a live GitLab instance. Located at gitlab-org/gitlab-qa@lib/gitlab/qa/scenario/test/.

That said, it occurred to me that the tree structure under https://gitlab.com/gitlab-org/gitlab-ce/qa/ is very misleading because it mixes the framework and the actual scenarios (aka test cases or examples)!

I think we could improve it and reduce the confusion by adopting a structure like the following:

Screen_Shot_2018-05-24_at_12.56.16

Proposal

  1. Rename "GitLab QA" (this project) to "GitLab QA orchestrator" (and its path from gitlab-qa to gitlab-qa-orchestrator, probably no need to rename the Ruby gem)
  2. Improve the structure under https://gitlab.com/gitlab-org/gitlab-ce/qa/ to clearly separate the framework and the actual scenarios

@grzesiek @rspeicher @godfat @meks WDYT?