Okta ASA - Phase 1 deploy: qa-tunnel

1. Using the okta_asa environment in terraform:
   • Create a qa_tunnel project with an enrollment token
   • Add GitLab - Backend Engineers to this project, as normal users (not admins)
2. Copy the enrollment token created for this project by terraform from the Okta ASA web UI, and store in GKMS for use by the Okta ASA cookbook (manual step):
   • Create a new vault called 'gitlab-okta-asa' in the 'gitlab-ci' GCP project: ./bin/gkms-vault-create gitlab-okta-asa ci. The contents is a dictionary/hash with a top level 'enrollment_tokens' key, with the value being a dictionary/hash of project_name => token pairs (just the one to start with)
3. Make chef changes:
   1. To the qa-tunnel.gitlabinfo role, add:
      • The gitlab_okta_asa::default recipe
      • The attribute okta_asa.project_name with value 'qa_tunnel'
      • The group sft_gitlab_backend_engineers to the openssh.server.allow_groups list. Note it does not need to be added to gitlab_users.groups; SFTD will be responsible for creating the group
   2. To the ci-base role add GKMS vault secret configuration in the okta_asa.secrets key. See other GKMS vault secret config on the same role for a starting point; this config uses the 'okta_asa' key, with the path being 155816-gitlab-ci-secrets/gitlab-okta-asa
4. After chef has run on the server, verify that the okta_$USERNAME users have been created on the qa-tunnel server.
5. Write instructions for devs on how to install/use the Okta ASA client
6. Test with a friendly neighborhood Backend Engineer to ensure it works and instructions are usable.
   1. Test with a regular user of the qa-tunnel, to ensure GDK behaves
7. Update any existing documentation given to backend engineers for how to use the qa-tunnel server.
8. Create a new group in Okta for as-requested accounts (list noted in a comment below), and add it to the terraform configuration
   1. Ensure group is populated with extra users.
9. Inform all affected users to start using the new accounts, and how to do so; give a one week grace period before the next step.
10. Regularly monitor active users (ssh logs) to see who is not converting and make contact half-way through the grace period.
11. Remove the gitlab-qa-tunnel group from chef-repo.
    1. Looking at data_bags/users/*.json:
       1. For any user that has only the gitlab-qa-tunnel group, modify their 'action' to 'remove'
       2. For any others with gitlab-qa-tunnel in their groups, remove that group but leave the rest.
    2. Remove references to the gitlab-qa-tunnel group from roles/gitlab-qa-tunnel.json (gitlab_users.groups, and openssh.server.allow_groups)
    3. Commit/merge + have reviewed
    4. For each modified user file, upload the changes to the chef server: knife data bag from file users $FILE
    5. Run chef manually on the qa-tunnel server, validate that all the legacy accounts are removed, and gitlab-qa-tunnel group is gone.