feat: implement ami filtering
Description
This MR implements AMI filtering for AWS EKS worker nodes. Previously, only fixed AMI IDs were supported, this MR allows fixed IDs OR AMI filtering.
If the fixed ID is set, that will be prefered over the filtered AMI, therefore the cluster operator have more control over the cluster.
Supporting information
https://tasks.opencraft.com/browse/SE-5940
Testing instructions
Steps to test the changes:
- Validate the output below corresponds to the changes
Note: Objects have changed outside of Terraform
Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:
# module.mysql.aws_db_instance.rds_instance has changed
~ resource "aws_db_instance" "rds_instance" {
id = "oc-stage-us-east-1-db"
~ latest_restorable_time = "2023-03-20T18:05:00Z" -> "2023-04-24T10:30:00Z"
+ listener_endpoint = []
+ storage_throughput = 0
tags = {
"ManagedBy" = "Grove"
"Terraform" = "true"
}
# (49 unchanged attributes hidden)
}
Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions
to undo or respond to these changes.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
<= read (data resources)
Terraform will perform the following actions:
# module.ingress.helm_release.nginx_ingress will be updated in-place
~ resource "helm_release" "nginx_ingress" {
id = "ingress-nginx"
name = "ingress-nginx"
~ timeout = 300 -> 600
# (27 unchanged attributes hidden)
# (2 unchanged blocks hidden)
}
# module.instance_edxapp_bucket["blueprint"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-blueprint-20230120120259057300000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "blueprint.staging.dev.grove-aws.opencraft.hosting",
+ "preview.blueprint.staging.dev.grove-aws.opencraft.hosting",
+ "studio.blueprint.staging.dev.grove-aws.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.instance_edxapp_bucket["grove-deployment-test-kshitij"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-grove-deployme20230120123521324000000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "app.grove-deployment-test-ksh.stage.opencraft.hosting",
+ "discovery.grove-deployment-test-ksh.stage.opencraft.hosting",
+ "ecommerce.grove-deployment-test-ksh.stage.opencraft.hosting",
+ "grove-deployment-test-ksh.stage.opencraft.hosting",
+ "preview.grove-deployment-test-ksh.stage.opencraft.hosting",
+ "studio.grove-deployment-test-ksh.stage.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.instance_edxapp_bucket["mtyaka"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-mtyaka-20230224112635906400000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "app.mtyaka.staging.dev.grove-aws.opencraft.hosting",
+ "mtyaka.staging.dev.grove-aws.opencraft.hosting",
+ "studio.mtyaka.staging.dev.grove-aws.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.instance_edxapp_bucket["nutmegone"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-nutmegone-20230220005711767800000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "app.nutmegone.staging.dev.grove-aws.opencraft.hosting",
+ "nutmegone.staging.dev.grove-aws.opencraft.hosting",
+ "studio.nutmegone.staging.dev.grove-aws.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.instance_edxapp_bucket["nutmegtwo"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-nutmegtwo-20230220062255828200000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "app.nutmegtwo.staging.dev.grove-aws.opencraft.hosting",
+ "nutmegtwo.staging.dev.grove-aws.opencraft.hosting",
+ "studio.nutmegtwo.staging.dev.grove-aws.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.instance_edxapp_bucket["test-video-settings"].aws_s3_bucket_cors_configuration.s3_cors will be updated in-place
~ resource "aws_s3_bucket_cors_configuration" "s3_cors" {
id = "edx-oc-stage-us-east-1-test-video-set20230316092128034900000001"
# (1 unchanged attribute hidden)
- cors_rule {
- allowed_headers = [
- "*",
] -> null
- allowed_methods = [
- "GET",
- "POST",
- "PUT",
] -> null
- allowed_origins = [
- "*",
] -> null
- expose_headers = [] -> null
- max_age_seconds = 0 -> null
}
+ cors_rule {
+ allowed_headers = [
+ "*",
]
+ allowed_methods = [
+ "GET",
+ "POST",
+ "PUT",
]
+ allowed_origins = [
+ "app.video-settings-test.stage.opencraft.hosting",
+ "discovery.video-settings-test.stage.opencraft.hosting",
+ "ecommerce.video-settings-test.stage.opencraft.hosting",
+ "preview.video-settings-test.stage.opencraft.hosting",
+ "studio.video-settings-test.stage.opencraft.hosting",
+ "video-settings-test.stage.opencraft.hosting",
]
+ expose_headers = []
}
}
# module.k8s_monitoring.data.external.opensearch_admin_key_pkcs8 will be read during apply
# (depends on a resource or a module with changes pending)
<= data "external" "opensearch_admin_key_pkcs8" {
+ id = (known after apply)
+ program = [
+ "/bin/sh",
+ "../provider-modules/k8s-monitoring/java_key.sh",
+ (sensitive value),
]
+ result = (known after apply)
}
# module.k8s_monitoring.kubernetes_secret.opensearch_http_certificates will be updated in-place
~ resource "kubernetes_secret" "opensearch_http_certificates" {
~ data = (sensitive value)
id = "monitoring/opensearch-http-certificates"
# (3 unchanged attributes hidden)
# (1 unchanged block hidden)
}
# module.eks.module.self_managed_node_group["worker_group"].aws_autoscaling_group.this[0] will be updated in-place
~ resource "aws_autoscaling_group" "this" {
id = "oc-stage-us-east-1-workers-20230120121401555400000016"
name = "oc-stage-us-east-1-workers-20230120121401555400000016"
# (24 unchanged attributes hidden)
~ launch_template {
id = "lt-022d2e42c380205a3"
name = "worker_group-20230120121400544500000014"
~ version = "2" -> (known after apply)
}
# (8 unchanged blocks hidden)
}
# module.eks.module.self_managed_node_group["worker_group"].aws_launch_template.this[0] will be updated in-place
~ resource "aws_launch_template" "this" {
~ default_version = 2 -> (known after apply)
id = "lt-022d2e42c380205a3"
~ image_id = "ami-0eca1b299fdb31f00" -> "ami-0205ea3fdf1715b14"
~ latest_version = 2 -> (known after apply)
name = "worker_group-20230120121400544500000014"
tags = {
"ManagedBy" = "Grove"
"Terraform" = "true"
}
# (11 unchanged attributes hidden)
# (6 unchanged blocks hidden)
}
Plan: 0 to add, 10 to change, 0 to destroy.
Changes to Outputs:
~ kubeconfig = (sensitive value)
~ mysql = (sensitive value)
╷
│ Warning: "default_secret_name" is no longer applicable for Kubernetes v1.24.0 and above
│
│ with module.k8s_gitlab_container_registry["mtyaka"].kubernetes_default_service_account.k8s_default_service_account,
│ on ../provider-modules/k8s-gitlab-container-registry/main.tf line 38, in resource "kubernetes_default_service_account" "k8s_default_service_account":
│ 38: resource "kubernetes_default_service_account" "k8s_default_service_account" {
│
│ Starting from version 1.24.0 Kubernetes does not automatically generate a token for service accounts, in this case, "default_secret_name" will be empty
│
│ (and 5 more similar warnings elsewhere)
╵
❯ ./tf apply
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
There are some problems with the CLI configuration:
╷
│ Warning: Unable to open CLI configuration file
│
│ The CLI configuration file at "/home/grove/.terraformrc" does not exist.
╵
There are some problems with the CLI configuration:
╷
│ Warning: Unable to open CLI configuration file
│
│ The CLI configuration file at "/home/grove/.terraformrc" does not exist.
╵
Initializing the backend...
Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...
Initializing provider plugins...
- Reusing previous version of hashicorp/http from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/helm from the dependency lock file
- Reusing previous version of hashicorp/kubernetes from the dependency lock file
- Reusing previous version of loafoe/htpasswd from the dependency lock file
- Reusing previous version of gavinbunney/kubectl from the dependency lock file
- Reusing previous version of hashicorp/tls from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of gitlabhq/gitlab from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of mongodb/mongodbatlas from the dependency lock file
- Reusing previous version of hashicorp/cloudinit from the dependency lock file
- Using previously-installed hashicorp/cloudinit v2.3.2
- Using previously-installed hashicorp/kubernetes v2.18.1
- Using previously-installed gavinbunney/kubectl v1.14.0
- Using previously-installed hashicorp/tls v4.0.4
- Using previously-installed gitlabhq/gitlab v15.10.0
- Using previously-installed hashicorp/aws v4.59.0
- Using previously-installed mongodb/mongodbatlas v1.8.2
- Using previously-installed hashicorp/http v3.2.1
- Using previously-installed hashicorp/random v3.4.3
- Using previously-installed hashicorp/helm v2.9.0
- Using previously-installed loafoe/htpasswd v1.0.4
- Using previously-installed hashicorp/external v2.3.1
Terraform has been successfully initialized!
There are some problems with the CLI configuration:
╷
│ Warning: Unable to open CLI configuration file
│
│ The CLI configuration file at "/home/grove/.terraformrc" does not exist.
╵
Acquiring state lock. This may take a few moments...
module.instance_edxapp_bucket["nutmegtwo"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-nutmegtwo-20230220062255828200000001]
module.instance_edxapp_bucket["grove-deployment-test-kshitij"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-grove-deployme20230120123521324000000001]
module.instance_edxapp_bucket["test-video-settings"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-test-video-set20230316092128034900000001]
module.instance_edxapp_bucket["nutmegone"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-nutmegone-20230220005711767800000001]
module.instance_edxapp_bucket["blueprint"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-blueprint-20230120120259057300000001]
module.instance_edxapp_bucket["mtyaka"].aws_s3_bucket_cors_configuration.s3_cors: Modifying... [id=edx-oc-stage-us-east-1-mtyaka-20230224112635906400000001]
module.eks.module.self_managed_node_group["worker_group"].aws_launch_template.this[0]: Modifying... [id=lt-022d2e42c380205a3]
module.ingress.helm_release.nginx_ingress: Modifying... [id=ingress-nginx]
module.instance_edxapp_bucket["nutmegtwo"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-nutmegtwo-20230220062255828200000001]
module.instance_edxapp_bucket["nutmegone"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-nutmegone-20230220005711767800000001]
module.instance_edxapp_bucket["blueprint"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-blueprint-20230120120259057300000001]
module.instance_edxapp_bucket["test-video-settings"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-test-video-set20230316092128034900000001]
module.instance_edxapp_bucket["mtyaka"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-mtyaka-20230224112635906400000001]
module.instance_edxapp_bucket["grove-deployment-test-kshitij"].aws_s3_bucket_cors_configuration.s3_cors: Modifications complete after 2s [id=edx-oc-stage-us-east-1-grove-deployme20230120123521324000000001]
module.eks.module.self_managed_node_group["worker_group"].aws_launch_template.this[0]: Modifications complete after 3s [id=lt-022d2e42c380205a3]
module.eks.module.self_managed_node_group["worker_group"].aws_autoscaling_group.this[0]: Modifying... [id=oc-stage-us-east-1-workers-20230120121401555400000016]
module.eks.module.self_managed_node_group["worker_group"].aws_autoscaling_group.this[0]: Modifications complete after 3s [id=oc-stage-us-east-1-workers-20230120121401555400000016]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 10s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 20s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 30s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 40s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 50s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 1m0s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 1m10s elapsed]
module.ingress.helm_release.nginx_ingress: Still modifying... [id=ingress-nginx, 1m20s elapsed]
module.ingress.helm_release.nginx_ingress: Modifications complete after 1m25s [id=ingress-nginx]
module.k8s_monitoring.data.external.opensearch_admin_key_pkcs8: Reading...
module.k8s_monitoring.data.external.opensearch_admin_key_pkcs8: Read complete after 0s [id=-]
Apply complete! Resources: 0 added, 9 changed, 0 destroyed.Dependencies
N/A
Screenshots
N/A
Checklist
If any of the items below is not applicable, do not remove them, but put a check in it.
-
All providers include the new feature/change -
All affected providers can provision new clusters -
Unit tests are added/updated -
Documentation is added/updated -
The TOOLS_CONTAINER_IMAGE_VERSIONin ci_vars.yml is updated -
The grove-template repository is updated
Additional context
N/A
Edited by Boros Gábor