Skip to content

[SE-2273] Update keystone calls (OpenStack Identity API) from v2 to v3

Daniel Clemente Laboreo requested to merge clemente/keystone-v3-api into master

Update keystone (OpenStack Identity Service) API v2.0 to v3. See deprecation notice

The API change affects the way Ocim authenticates with OpenStack. I don't think it affects other cases (e.g. the SWIFT usage done from an edxapp instance), though I didn't particularly test it since we've deprecated it. Likewise for the migrate_swift_to_s3 command: it passes some parameters to rclone to create a configuration. This call may need some update if we ever use the script again and it doesn't work.

Notes

  • This PR changes the authorization part. It doesn't change other OpenStack APIs. For instance, this mustn't change from 2 to 3: def get_nova_client(region_name, api_version=2):. It's a different setting, it's the client version, not the „auth_version“. If I change it, I get: novaclient.exceptions.UnsupportedVersion: Invalid client version '3.0'. Major part should be '2'.

About this part:

+            user_domain_id="default",
+            user_domain_name="Default",

The names default and Default are extracted from a settings file downloaded from OVH, which contains:

export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="default"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi

We don't require customizing them and we can keep them at default.

Note that e.g. domain ID appears twice: user domain ID, and project domain ID. This is because the expected v3 request contains this information twice too: the "domain": { "id": "default" } appears twice. The code didn't work when the request only set the user domain ID but not the project domain ID.

As for user_domain_name/project_domain_name, note that it's not the same as the „project name“ (aka „tenant“) which is set in another line, project_name=settings.OPENSTACK_TENANT, to a private numeric ID.

Finally, the authorization code is repeated in two places because we use two different variables: openstacksdk (in get_openstack_connection) and python-novaclient (in get_nova_client). Luckily, the authorization works in the same way (I think both of them use keystone).

Testing protocol

  • Phase 1: testing in CI
    • the tests should pass. The flaky tests should flake
    • you should check that the change covers APIs mentioned in the deprecation notice
    • This PR is the final one. But in this other branch I'm using a temporary variable, OPENSTACK_AUTH_URL_V3, to avoid changing the real one OPENSTACK_AUTH_URL while I test. I added OPENSTACK_AUTH_URL_V3 to CircleCI. The code in that branch uses OPENSTACK_AUTH_URL_V3
  • Phase 2: in Ocim stage
    • remove the OPENSTACK_AUTH_URL_V3 in the code (update: done in this PR), use OPENSTACK_AUTH_URL. CircleCI will not pass (we didn't change OPENSTACK_AUTH_URL yet), but it's not important in this phase
    • the PR should be the final one
    • put it in stage
    • try to deploy an instance
    • in particular, check that it's created in the right place: right account, right security group, right SSH key (i.e. you can access), etc. Try to archive it too, just in case
    • if the deployment works, and the CircleCI tests from phase 1 passed, it's ready to be deployed
  • Phase 3: in Ocim production
    • update OPENSTACK_AUTH_URL in CircleCI to https://auth.cloud.ovh.net/v3. Remove OPENSTACK_AUTH_URL_V3
    • (update it in Vault too)
    • re-run the CircleCI tests from github. They should be reading OPENSTACK_AUTH_URL now, and they should pass now
    • (note that up to now, CircleCI is broken for everyone else who isn't using this branch)
    • merge the PR
    • deploy to Ocim production. This shouldn't change any behaviour. No .env change is required
    • ask people to rebase on master

Merge request reports