github-mirrors/terraform-aws-eks
1
0
Fork 0
mirror of https://github.com/ysoftdevs/terraform-aws-eks.git synced 2026-02-24 03:24:50 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat!: Replace the use of aws-auth configmap with EKS cluster access entry (#2858)

Browse Source 
* feat: Replace `resolve_conflicts` with `resolve_conflicts_on_create`/`delete`; raise MSV of AWS provider to `v5.0` to support

* fix: Replace dynamic DNS suffix for `sts:AssumeRole` API calls for static suffix

* feat: Add module tag

* feat: Align Karpenter permissions with Karpenter v1beta1/v0.32 permissions from upstream

* refactor: Move `aws-auth` ConfigMap functionality to its own sub-module

* chore: Update examples

* feat: Add state `moved` block for Karpenter Pod Identity role re-name

* fix: Correct variable `create` description

* feat: Add support for cluster access entries

* chore: Bump MSV of Terraform to `1.3`

* fix: Replace defunct kubectl provider with an updated forked equivalent

* chore: Update and validate examples for access entry; clean up provider usage

* docs: Correct double redundant variable descriptions

* feat: Add support for Cloudwatch log group class argument

* fix: Update usage tag placement, fix Karpenter event spelling, add upcoming changes section to upgrade guide

* feat: Update Karpenter module to generalize naming used and align policy with the upstream Karpenter policy

* feat: Add native support for Windows based managed nodegroups similar to AL2 and Bottlerocket

* feat: Update self-managed nodegroup module to use latest features of ASG

* docs: Update and simplify docs

* fix: Correct variable description for AMI types

* fix: Update upgrade guide with changes; rename Karpenter controller resource names to support migrating for users

* docs: Complete upgrade guide docs for migration and changes applied

* Update examples/karpenter/README.md

Co-authored-by: Anton Babenko <anton@antonbabenko.com>

* Update examples/outposts/README.md

Co-authored-by: Anton Babenko <anton@antonbabenko.com>

* Update modules/karpenter/README.md

Co-authored-by: Anton Babenko <anton@antonbabenko.com>

---------

Co-authored-by: Anton Babenko <anton@antonbabenko.com>
This commit is contained in:
Bryant Biggs
2024-02-02 09:36:25 -05:00
committed by GitHub
parent 2cb1fac31b
commit 6b40bdbb1d
71 changed files with 1809 additions and 2136 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
examples/eks_managed_node_group/README.md
View File

@@ -29,31 +29,31 @@ Note that this example may create resources which cost money. Run `terraform des
| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.57 |
| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.34 |
## Providers
| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.57 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.34 |
## Modules
| Name | Source | Version |
|------|--------|---------|
| <a name="module_ebs_kms_key"></a> [ebs\_kms\_key](#module\_ebs\_kms\_key) | terraform-aws-modules/kms/aws | ~> 1.5 |
| <a name="module_disabled_eks"></a> [disabled\_eks](#module\_disabled\_eks) | ../.. | n/a |
| <a name="module_disabled_eks_managed_node_group"></a> [disabled\_eks\_managed\_node\_group](#module\_disabled\_eks\_managed\_node\_group) | ../../modules/eks-managed-node-group | n/a |
| <a name="module_ebs_kms_key"></a> [ebs\_kms\_key](#module\_ebs\_kms\_key) | terraform-aws-modules/kms/aws | ~> 2.1 |
| <a name="module_eks"></a> [eks](#module\_eks) | ../.. | n/a |
| <a name="module_eks_managed_node_group"></a> [eks\_managed\_node\_group](#module\_eks\_managed\_node\_group) | ../../modules/eks-managed-node-group | n/a |
| <a name="module_key_pair"></a> [key\_pair](#module\_key\_pair) | terraform-aws-modules/key-pair/aws | ~> 2.0 |
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 4.0 |
| <a name="module_vpc_cni_irsa"></a> [vpc\_cni\_irsa](#module\_vpc\_cni\_irsa) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 5.0 |
| <a name="module_vpc"></a> [vpc](#module\_vpc) | terraform-aws-modules/vpc/aws | ~> 5.0 |
## Resources
| Name | Type |
|------|------|
| [aws_autoscaling_group_tag.cluster_autoscaler_label_tags](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group_tag) | resource |
| [aws_iam_policy.node_additional](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_security_group.remote_access](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_ami.eks_default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) | data source |
@@ -70,7 +70,7 @@ No inputs.
| Name | Description |
|------|-------------|
| <a name="output_aws_auth_configmap_yaml"></a> [aws\_auth\_configmap\_yaml](#output\_aws\_auth\_configmap\_yaml) | Formatted yaml output for base aws-auth configmap containing roles used in cluster node groups/fargate profiles |
| <a name="output_access_entries"></a> [access\_entries](#output\_access\_entries) | Map of access entries created and their attributes |
| <a name="output_cloudwatch_log_group_arn"></a> [cloudwatch\_log\_group\_arn](#output\_cloudwatch\_log\_group\_arn) | Arn of cloudwatch log group created |
| <a name="output_cloudwatch_log_group_name"></a> [cloudwatch\_log\_group\_name](#output\_cloudwatch\_log\_group\_name) | Name of cloudwatch log group created |
| <a name="output_cluster_addons"></a> [cluster\_addons](#output\_cluster\_addons) | Map of attribute maps for all EKS cluster addons enabled |

173
examples/eks_managed_node_group/main.tf
View File

@@ -2,18 +2,6 @@ provider "aws" {
region = local.region
}
provider "kubernetes" {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = "client.authentication.k8s.io/v1beta1"
command = "aws"
# This requires the awscli to be installed locally where Terraform is executed
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
}
}
data "aws_caller_identity" "current" {}
data "aws_availability_zones" "available" {}
@@ -44,14 +32,7 @@ module "eks" {
cluster_endpoint_public_access = true
# IPV6
cluster_ip_family = "ipv6"
# We are using the IRSA created below for permissions
# However, we have to deploy with the policy attached FIRST (when creating a fresh cluster)
# and then turn this off after the cluster/node group is created. Without this initial policy,
# the VPC CNI fails to assign IPs and nodes cannot join the cluster
# See https://github.com/aws/containers-roadmap/issues/1666 for more context
# TODO - remove this policy once AWS releases a managed version similar to AmazonEKS_CNI_Policy (IPv4)
cluster_ip_family = "ipv6"
create_cni_ipv6_iam_policy = true
cluster_addons = {
@@ -62,9 +43,8 @@ module "eks" {
most_recent = true
}
vpc-cni = {
most_recent = true
before_compute = true
service_account_role_arn = module.vpc_cni_irsa.iam_role_arn
most_recent = true
before_compute = true
configuration_values = jsonencode({
env = {
# Reference docs https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html
@@ -79,18 +59,9 @@ module "eks" {
subnet_ids = module.vpc.private_subnets
control_plane_subnet_ids = module.vpc.intra_subnets
manage_aws_auth_configmap = true
eks_managed_node_group_defaults = {
ami_type = "AL2_x86_64"
instance_types = ["m6i.large", "m5.large", "m5n.large", "m5zn.large"]
# We are using the IRSA created below for permissions
# However, we have to deploy with the policy attached FIRST (when creating a fresh cluster)
# and then turn this off after the cluster/node group is created. Without this initial policy,
# the VPC CNI fails to assign IPs and nodes cannot join the cluster
# See https://github.com/aws/containers-roadmap/issues/1666 for more context
iam_role_attach_cni_policy = true
}
eks_managed_node_groups = {
@@ -264,27 +235,6 @@ module "eks" {
additional = aws_iam_policy.node_additional.arn
}
schedules = {
scale-up = {
min_size = 2
max_size = "-1" # Retains current max size
desired_size = 2
start_time = "2023-03-05T00:00:00Z"
end_time = "2024-03-05T00:00:00Z"
time_zone = "Etc/GMT+0"
recurrence = "0 0 * * *"
},
scale-down = {
min_size = 0
max_size = "-1" # Retains current max size
desired_size = 0
start_time = "2023-03-05T12:00:00Z"
end_time = "2024-03-05T12:00:00Z"
time_zone = "Etc/GMT+0"
recurrence = "0 12 * * *"
}
}
tags = {
ExtraTag = "EKS managed node group complete example"
}
@@ -294,13 +244,59 @@ module "eks" {
tags = local.tags
}
module "disabled_eks" {
source = "../.."
create = false
}
################################################################################
# Sub-Module Usage on Existing/Separate Cluster
################################################################################
module "eks_managed_node_group" {
source = "../../modules/eks-managed-node-group"
name = "separate-eks-mng"
cluster_name = module.eks.cluster_name
cluster_version = module.eks.cluster_version
subnet_ids = module.vpc.private_subnets
cluster_primary_security_group_id = module.eks.cluster_primary_security_group_id
vpc_security_group_ids = [
module.eks.cluster_security_group_id,
]
ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"
# this will get added to what AWS provides
bootstrap_extra_args = <<-EOT
# extra args added
[settings.kernel]
lockdown = "integrity"
[settings.kubernetes.node-labels]
"label1" = "foo"
"label2" = "bar"
EOT
tags = merge(local.tags, { Separate = "eks-managed-node-group" })
}
module "disabled_eks_managed_node_group" {
source = "../../modules/eks-managed-node-group"
create = false
}
################################################################################
# Supporting Resources
################################################################################
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 4.0"
version = "~> 5.0"
name = local.name
cidr = local.vpc_cidr
@@ -333,27 +329,9 @@ module "vpc" {
tags = local.tags
}
module "vpc_cni_irsa" {
source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
version = "~> 5.0"
role_name_prefix = "VPC-CNI-IRSA"
attach_vpc_cni_policy = true
vpc_cni_enable_ipv6 = true
oidc_providers = {
main = {
provider_arn = module.eks.oidc_provider_arn
namespace_service_accounts = ["kube-system:aws-node"]
}
}
tags = local.tags
}
module "ebs_kms_key" {
source = "terraform-aws-modules/kms/aws"
version = "~> 1.5"
version = "~> 2.1"
description = "Customer managed key to encrypt EKS managed node group volumes"
@@ -458,52 +436,3 @@ data "aws_ami" "eks_default_bottlerocket" {
values = ["bottlerocket-aws-k8s-${local.cluster_version}-x86_64-*"]
}
}
################################################################################
# Tags for the ASG to support cluster-autoscaler scale up from 0
################################################################################
locals {
# We need to lookup K8s taint effect from the AWS API value
taint_effects = {
NO_SCHEDULE = "NoSchedule"
NO_EXECUTE = "NoExecute"
PREFER_NO_SCHEDULE = "PreferNoSchedule"
}
cluster_autoscaler_label_tags = merge([
for name, group in module.eks.eks_managed_node_groups : {
for label_name, label_value in coalesce(group.node_group_labels, {}) : "${name}|label|${label_name}" => {
autoscaling_group = group.node_group_autoscaling_group_names[0],
key = "k8s.io/cluster-autoscaler/node-template/label/${label_name}",
value = label_value,
}
}
]...)
cluster_autoscaler_taint_tags = merge([
for name, group in module.eks.eks_managed_node_groups : {
for taint in coalesce(group.node_group_taints, []) : "${name}|taint|${taint.key}" => {
autoscaling_group = group.node_group_autoscaling_group_names[0],
key = "k8s.io/cluster-autoscaler/node-template/taint/${taint.key}"
value = "${taint.value}:${local.taint_effects[taint.effect]}"
}
}
]...)
cluster_autoscaler_asg_tags = merge(local.cluster_autoscaler_label_tags, local.cluster_autoscaler_taint_tags)
}
resource "aws_autoscaling_group_tag" "cluster_autoscaler_label_tags" {
for_each = local.cluster_autoscaler_asg_tags
autoscaling_group_name = each.value.autoscaling_group
tag {
key = each.value.key
value = each.value.value
propagate_at_launch = false
}
}

18
examples/eks_managed_node_group/outputs.tf
View File

@@ -47,6 +47,15 @@ output "cluster_primary_security_group_id" {
value = module.eks.cluster_primary_security_group_id
}
################################################################################
# Access Entry
################################################################################
output "access_entries" {
description = "Map of access entries created and their attributes"
value = module.eks.access_entries
}
################################################################################
# KMS Key
################################################################################
@@ -200,12 +209,3 @@ output "self_managed_node_groups_autoscaling_group_names" {
description = "List of the autoscaling group names created by self-managed node groups"
value = module.eks.self_managed_node_groups_autoscaling_group_names
}
################################################################################
# Additional
################################################################################
output "aws_auth_configmap_yaml" {
description = "Formatted yaml output for base aws-auth configmap containing roles used in cluster node groups/fargate profiles"
value = module.eks.aws_auth_configmap_yaml
}

8
examples/eks_managed_node_group/versions.tf
View File

@@ -1,14 +1,10 @@
terraform {
required_version = ">= 1.0"
required_version = ">= 1.3"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.57"
}
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.10"
version = ">= 5.34"
}
}
}