Set up GCP Workload Identity Federation for EKS workloads

Create a GCP Service Account for each EKS workload and set up Pod Identity and Workload Identity Federation so the EKS workload can easily authenticate to GCP APIs.

This will allow EKS workloads to send continuous profiling and tracing information to GCP without the need to copy tokens / credentials around.

Implementation outline

  • Provisioner: Provision a "Workload Identity Pool" (one each for gitlab-runway-staging and gitlab-runway-production).

  • Provisioner: Provision the "AWS Identity Provider" within the Pool.

  • Provisioner: Amend the flux-cluster-vars ConfigMap.

    • gcp_workload_identity_audience
    • gcp_project_id

  • Provisioner: Create a GCP Service Account for each Runway service as eks-${runway_service_id}.

  • Provisioner: Bind the GSA to the specific AWS identity using the roles/iam.workloadIdentityUser role.

    Format:

    principalSet://iam.googleapis.com/${var.gcp_workload_identity_pool}/attribute.aws_iam_role/${module.pod_identity.iam_role_name}

  • Provisioner: Create an IAM policy granting the roles/cloudprofiler.agent role to the service's GCP Service Account.

  • Provisioner: Add a Pod Identity

  • Provisioner: add per-service IAM role and Pod Identity association

    data "aws_iam_policy_document" "default_policy" {
  statement {
    effect = "Allow"
    actions = [
      # GetCallerIdentity is used by GCP to verify the IAM role identity.
      "sts:GetCallerIdentity",
    ]
    resources = ["*"]
  }
}

  • runway-eks Helm chart: Add a ConfigMap with the following content:

    {
  "universe_domain": "googleapis.com",
  "type": "external_account",
  "audience": "${gcp_workload_identity_audience}",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/eks-${runway_service_id}@${gcp_project_id}.iam.gserviceaccount.com:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  }
}

    Note: The ${…} placeholders will be substituted from the above ConfigMap by Flux.

  • runway-eks Helm chart: Add the ConfigMap to the Deployment as a volume mount, so that the credentials file is available as /etc/gcp-service-account/credentials.json.

  • runway-eks Helm chart: Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to /etc/gcp-service-account/credentials.json.

Edited by Florian Forster