Use create_before_destroy to make some EKS node group changes less disruptive

Some EKS node group changes (e.g. disk size changes) are disruptive because they require a destroy/recreate of the node group. Launch templates help (avoid), but currently in GET require a custom AMI to select, and that AMI needs ongoing maintenance for upgrades etc.

However, it is possible to use the terraform lifecycle create_before_destroy; that will build a new node group first, then destroy the old one, during which EKS will cordon/drain the nodes and move the workloads to the new node group, keeping disruption as minimal as possible for this sort of node-replacement operation.

MR created; it contains a fuller description of how it works.

Edited by Craig Miskell