[SE-2273] Update keystone calls (OpenStack Identity API) from v2 to v3
Update keystone (OpenStack Identity Service) API v2.0 to v3. See deprecation notice
The API change affects the way Ocim authenticates with OpenStack.
I don't think it affects other cases (e.g. the SWIFT usage done from an edxapp instance), though I didn't particularly test it since we've deprecated it. Likewise for the migrate_swift_to_s3
command: it passes some parameters to rclone
to create a configuration. This call may need some update if we ever use the script again and it doesn't work.
Notes
- This PR changes the authorization part. It doesn't change other OpenStack APIs. For instance, this mustn't change from 2 to 3:
def get_nova_client(region_name, api_version=2):
. It's a different setting, it's the client version, not the „auth_version“. If I change it, I get:novaclient.exceptions.UnsupportedVersion: Invalid client version '3.0'. Major part should be '2'
.
About this part:
+ user_domain_id="default",
+ user_domain_name="Default",
The names default
and Default
are extracted from a settings file downloaded from OVH, which contains:
export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="default"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi
We don't require customizing them and we can keep them at default
.
Note that e.g. domain ID appears twice: user domain ID, and project domain ID. This is because the expected v3 request contains this information twice too: the "domain": { "id": "default" }
appears twice. The code didn't work when the request only set the user domain ID but not the project domain ID.
As for user_domain_name/project_domain_name, note that it's not the same as the „project name“ (aka „tenant“) which is set in another line, project_name=settings.OPENSTACK_TENANT
, to a private numeric ID.
Finally, the authorization code is repeated in two places because we use two different variables: openstacksdk
(in get_openstack_connection
) and python-novaclient
(in get_nova_client
). Luckily, the authorization works in the same way (I think both of them use keystone).
Testing protocol
- Phase 1: testing in CI
- the tests should pass. The flaky tests should flake
- you should check that the change covers APIs mentioned in the deprecation notice
- This PR is the final one. But in this other branch I'm using a temporary variable,
OPENSTACK_AUTH_URL_V3
, to avoid changing the real oneOPENSTACK_AUTH_URL
while I test. I addedOPENSTACK_AUTH_URL_V3
to CircleCI. The code in that branch usesOPENSTACK_AUTH_URL_V3
- Phase 2: in Ocim stage
- remove the
OPENSTACK_AUTH_URL_V3
in the code (update: done in this PR), useOPENSTACK_AUTH_URL
. CircleCI will not pass (we didn't changeOPENSTACK_AUTH_URL
yet), but it's not important in this phase - the PR should be the final one
- put it in stage
- try to deploy an instance
- in particular, check that it's created in the right place: right account, right security group, right SSH key (i.e. you can access), etc. Try to archive it too, just in case
- if the deployment works, and the CircleCI tests from phase 1 passed, it's ready to be deployed
- remove the
- Phase 3: in Ocim production
- update
OPENSTACK_AUTH_URL
in CircleCI tohttps://auth.cloud.ovh.net/v3
. RemoveOPENSTACK_AUTH_URL_V3
- (update it in Vault too)
- re-run the CircleCI tests from github. They should be reading
OPENSTACK_AUTH_URL
now, and they should pass now - (note that up to now, CircleCI is broken for everyone else who isn't using this branch)
- merge the PR
- deploy to Ocim production. This shouldn't change any behaviour. No
.env
change is required - ask people to rebase on master
- update