Skip to content

BB-690 - Add new integration cleanup script WIP

Giovanni Cimolin da Silva requested to merge giovanni/bb-690-improve-integration-cleanup into master

Add new integration cleanup script to delete all AWS provisioned resources for CircleCI build that where left behind. Also cleans up OpenStack instances and DNS records.

Features:

  • Deletion of S3 buckets, policies, keys and users
  • Shutdown of VM's older than 3 days
  • Deletion of DNS records associated to integration

JIRA tickets: BB-690

Merge deadline: Sprint 183.

Testing instructions:

  1. Clone this branch of OCIM
  2. Create a virtualenv and install requirements with pip install -r requirements.txt
  3. Make sure you have the following environment variables set (from CircleCI env):
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
OPENSTACK_AUTH_URL
OPENSTACK_USER
OPENSTACK_PASSWORD
OPENSTACK_TENANT
OPENSTACK_REGION
GANDI_ZONE_ID
GANDI_API_KEY
  1. To test the cleanup tool without actually deleting anything, run:
# This will execute the tool, query every old S3 bucket, user, DNS records and OVH machines running that are related to CircleCI and are older than 3 days but not delete anything
./integration_cleanup.py --dry_run

Author notes and concerns:

  1. I'm aware that this cleanup tool doesn't clean up all DNS records.

Reviewers

  • @lgp171188

Merge request reports