feat!: Replace the use of aws-auth configmap with EKS cluster access entry (#2858)

* feat: Replace `resolve_conflicts` with `resolve_conflicts_on_create`/`delete`; raise MSV of AWS provider to `v5.0` to support * fix: Replace dynamic DNS suffix for `sts:AssumeRole` API calls for static suffix * feat: Add module tag * feat: Align Karpenter permissions with Karpenter v1beta1/v0.32 permissions from upstream * refactor: Move `aws-auth` ConfigMap functionality to its own sub-module * chore: Update examples * feat: Add state `moved` block for Karpenter Pod Identity role re-name * fix: Correct variable `create` description * feat: Add support for cluster access entries * chore: Bump MSV of Terraform to `1.3` * fix: Replace defunct kubectl provider with an updated forked equivalent * chore: Update and validate examples for access entry; clean up provider usage * docs: Correct double redundant variable descriptions * feat: Add support for Cloudwatch log group class argument * fix: Update usage tag placement, fix Karpenter event spelling, add upcoming changes section to upgrade guide * feat: Update Karpenter module to generalize naming used and align policy with the upstream Karpenter policy * feat: Add native support for Windows based managed nodegroups similar to AL2 and Bottlerocket * feat: Update self-managed nodegroup module to use latest features of ASG * docs: Update and simplify docs * fix: Correct variable description for AMI types * fix: Update upgrade guide with changes; rename Karpenter controller resource names to support migrating for users * docs: Complete upgrade guide docs for migration and changes applied * Update examples/karpenter/README.md Co-authored-by: Anton Babenko <anton@antonbabenko.com> * Update examples/outposts/README.md Co-authored-by: Anton Babenko <anton@antonbabenko.com> * Update modules/karpenter/README.md Co-authored-by: Anton Babenko <anton@antonbabenko.com> --------- Co-authored-by: Anton Babenko <anton@antonbabenko.com>
2026-03-11 21:11:32 +01:00 · 2024-02-02 09:36:25 -05:00
parent 2cb1fac31b
commit 6b40bdbb1d
71 changed files with 1809 additions and 2136 deletions
--- a/examples/self_managed_node_group/README.md
+++ b/examples/self_managed_node_group/README.md
@@ -25,24 +25,25 @@ Note that this example may create resources which cost money. Run `terraform des

 | Name | Version |
 |------|---------|
-| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.57 |
-| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.34 |

 ## Providers

 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.57 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.34 |

 ## Modules

 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_ebs_kms_key"></a> [ebs\_kms\_key](#module\_ebs\_kms\_key) | terraform-aws-modules/kms/aws | ~> 1.5 |
+| <a name="module_disabled_self_managed_node_group"></a> [disabled\_self\_managed\_node\_group](#module\_disabled\_self\_managed\_node\_group) | ../../modules/self-managed-node-group | n/a |
+| <a name="module_ebs_kms_key"></a> [ebs\_kms\_key](#module\_ebs\_kms\_key) | terraform-aws-modules/kms/aws | ~> 2.0 |
 | <a name="module_eks"></a> [eks](#module\_eks) | ../.. | n/a |
 | <a name="module_key_pair"></a> [key\_pair](#module\_key\_pair) | terraform-aws-modules/key-pair/aws | ~> 2.0 |
-| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 4.0 |
+| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | ~> 2.1 |
+| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |

 ## Resources

@@ -62,7 +63,7 @@ No inputs.

 | Name | Description |
 |------|-------------|
-| <a name="output_aws_auth_configmap_yaml"></a> [aws\_auth\_configmap\_yaml](#output\_aws\_auth\_configmap\_yaml) | Formatted yaml output for base aws-auth configmap containing roles used in cluster node groups/fargate profiles |
+| <a name="output_access_entries"></a> [access\_entries](#output\_access\_entries) | Map of access entries created and their attributes |
 | <a name="output_cloudwatch_log_group_arn"></a> [cloudwatch\_log\_group\_arn](#output\_cloudwatch\_log\_group\_arn) | Arn of cloudwatch log group created |
 | <a name="output_cloudwatch_log_group_name"></a> [cloudwatch\_log\_group\_name](#output\_cloudwatch\_log\_group\_name) | Name of cloudwatch log group created |
 | <a name="output_cluster_addons"></a> [cluster\_addons](#output\_cluster\_addons) | Map of attribute maps for all EKS cluster addons enabled |
--- a/examples/self_managed_node_group/main.tf
+++ b/examples/self_managed_node_group/main.tf
@@ -2,24 +2,12 @@ provider "aws" {
  region = local.region
 }

-provider "kubernetes" {
-  host                   = module.eks.cluster_endpoint
-  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
-
-  exec {
-    api_version = "client.authentication.k8s.io/v1beta1"
-    command     = "aws"
-    # This requires the awscli to be installed locally where Terraform is executed
-    args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
-  }
-}
-
 data "aws_caller_identity" "current" {}
 data "aws_availability_zones" "available" {}

 locals {
  name            = "ex-${replace(basename(path.cwd), "_", "-")}"
-  cluster_version = "1.27"
+  cluster_version = "1.29"
  region          = "eu-west-1"

  vpc_cidr = "10.0.0.0/16"
@@ -59,9 +47,12 @@ module "eks" {
  subnet_ids               = module.vpc.private_subnets
  control_plane_subnet_ids = module.vpc.intra_subnets

-  # Self managed node groups will not automatically create the aws-auth configmap so we need to
-  create_aws_auth_configmap = true
-  manage_aws_auth_configmap = true
+  # External encryption key
+  create_kms_key = false
+  cluster_encryption_config = {
+    resources        = ["secrets"]
+    provider_key_arn = module.kms.key_arn
+  }

  self_managed_node_group_defaults = {
    # enable discovery of autoscaling groups by cluster-autoscaler
@@ -141,36 +132,6 @@ module "eks" {
      }
    }

-    efa = {
-      min_size     = 1
-      max_size     = 2
-      desired_size = 1
-
-      # aws ec2 describe-instance-types --region eu-west-1 --filters Name=network-info.efa-supported,Values=true --query "InstanceTypes[*].[InstanceType]" --output text | sort
-      instance_type = "c5n.9xlarge"
-
-      post_bootstrap_user_data = <<-EOT
-        # Install EFA
-        curl -O https://efa-installer.amazonaws.com/aws-efa-installer-latest.tar.gz
-        tar -xf aws-efa-installer-latest.tar.gz && cd aws-efa-installer
-        ./efa_installer.sh -y --minimal
-        fi_info -p efa -t FI_EP_RDM
-
-        # Disable ptrace
-        sysctl -w kernel.yama.ptrace_scope=0
-      EOT
-
-      network_interfaces = [
-        {
-          description                 = "EFA interface example"
-          delete_on_termination       = true
-          device_index                = 0
-          associate_public_ip_address = false
-          interface_type              = "efa"
-        }
-      ]
-    }
-
    # Complete
    complete = {
      name            = "complete-self-mng"
@@ -287,12 +248,6 @@ module "eks" {
        additional                         = aws_iam_policy.additional.arn
      }

-      timeouts = {
-        create = "80m"
-        update = "80m"
-        delete = "80m"
-      }
-
      tags = {
        ExtraTag = "Self managed node group complete example"
      }
@@ -302,13 +257,19 @@ module "eks" {
  tags = local.tags
 }

+module "disabled_self_managed_node_group" {
+  source = "../../modules/self-managed-node-group"
+
+  create = false
+}
+
 ################################################################################
 # Supporting Resources
 ################################################################################

 module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
-  version = "~> 4.0"
+  version = "~> 5.0"

  name = local.name
  cidr = local.vpc_cidr
@@ -364,7 +325,7 @@ module "key_pair" {

 module "ebs_kms_key" {
  source  = "terraform-aws-modules/kms/aws"
-  version = "~> 1.5"
+  version = "~> 2.0"

  description = "Customer managed key to encrypt EKS managed node group volumes"

@@ -386,6 +347,18 @@ module "ebs_kms_key" {
  tags = local.tags
 }

+module "kms" {
+  source  = "terraform-aws-modules/kms/aws"
+  version = "~> 2.1"
+
+  aliases               = ["eks/${local.name}"]
+  description           = "${local.name} cluster encryption key"
+  enable_default_policy = true
+  key_owners            = [data.aws_caller_identity.current.arn]
+
+  tags = local.tags
+}
+
 resource "aws_iam_policy" "additional" {
  name        = "${local.name}-additional"
  description = "Example usage of node additional policy"
--- a/examples/self_managed_node_group/outputs.tf
+++ b/examples/self_managed_node_group/outputs.tf
@@ -47,6 +47,15 @@ output "cluster_primary_security_group_id" {
  value       = module.eks.cluster_primary_security_group_id
 }

+################################################################################
+# Access Entry
+################################################################################
+
+output "access_entries" {
+  description = "Map of access entries created and their attributes"
+  value       = module.eks.access_entries
+}
+
 ################################################################################
 # KMS Key
 ################################################################################
@@ -200,12 +209,3 @@ output "self_managed_node_groups_autoscaling_group_names" {
  description = "List of the autoscaling group names created by self-managed node groups"
  value       = module.eks.self_managed_node_groups_autoscaling_group_names
 }
-
-################################################################################
-# Additional
-################################################################################
-
-output "aws_auth_configmap_yaml" {
-  description = "Formatted yaml output for base aws-auth configmap containing roles used in cluster node groups/fargate profiles"
-  value       = module.eks.aws_auth_configmap_yaml
-}
--- a/examples/self_managed_node_group/versions.tf
+++ b/examples/self_managed_node_group/versions.tf
@@ -1,14 +1,10 @@
 terraform {
-  required_version = ">= 1.0"
+  required_version = ">= 1.3"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
-      version = ">= 4.57"
-    }
-    kubernetes = {
-      source  = "hashicorp/kubernetes"
-      version = ">= 2.10"
+      version = ">= 5.34"
    }
  }
 }