Upgrade to boto3

Created by: viadanna

Description

This PR upgrades the requirement from boto2 to boto3.

It also implements versioning and lifecycle management of s3 buckets

Related tickets: BB-719 BB-329

Testing

1. Checkout the changes and load the shell using make shell (this can be done using the admin as well).

2. Create an instance using the shell as mentioned in the README:

from instance.factories import instance_factory, production_instance_factory

# Creating an instance with defaults appropriate for sandboxes:
instance = instance_factory(name="Sandbox instance", sub_domain="sandbox")

# Creating an instance with defaults appropriate for production:
production_instance = production_instance_factory(name="Production instance", sub_domain="production")

3. Spawn an AppServer using on stage.console.opencraft.com

4. Use either the shell or awscli to query the bucket exists, CORS and lifecycle are correct, i.e:

instance.s3.get_bucket_cors(Bucket=instance.s3_bucket_name)
instance.s3.get_bucket_lifecycle(Bucket=instance.s3_bucket_name)
instance.s3.get_bucket_lifecycle_configuration(Bucket=instance.s3_bucket_name)

5. Delete the instance and make sure the bucket is deleted as well, i.e:

bucket = instance.s3_bucket_name
instance.delete()
boto3.client('s3').list_objects(Bucket=bucket)  # Should not exist

Management Command

A management command called reprovision_buckets was added to reprovision and configure iam, versioning and lifecycle on existing instance.

This needs to be tested in stage using:

$ make manage reprovision_buckets

This can be tested with the step 4 shown above.