A good start is to follow AWS documentations (https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html)
Pre-requisites
- eksctl (https://github.com/weaveworks/eksctl)
- Metrics Server (https://alexlogy.io/how-to-install-metrics-server-in-aws-eks/)
Create IAM Policy
Save the following content to a file.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/k8s.io/cluster-autoscaler/<my-cluster>": "owned"
}
}
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeAutoScalingGroups",
"ec2:DescribeLaunchTemplateVersions",
"autoscaling:DescribeTags",
"autoscaling:DescribeLaunchConfigurations"
],
"Resource": "*"
}
]
}Create IAM role and attach IAM policy
eksctl create iamserviceaccount \
--cluster=<my-cluster> \
--namespace=kube-system \
--name=cluster-autoscaler \
--attach-policy-arn=arn:aws:iam::<111122223333>:policy/<AmazonEKSClusterAutoscalerPolicy> \
--override-existing-serviceaccounts \
--approveDownload Cluster Autoscaler YAML
curl -o cluster-autoscaler-autodiscover.yaml https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yamlApply the YAML
kubectl apply -f cluster-autoscaler-autodiscover.yamlAnnotate the Service Account
kubectl annotate serviceaccount cluster-autoscaler \
-n kube-system \
eks.amazonaws.com/role-arn=arn:aws:iam::<ACCOUNT_ID>:role/<AmazonEKSClusterAutoscalerRole>Patch the deployment
kubectl patch deployment cluster-autoscaler \
-n kube-system \
-p '{"spec":{"template":{"metadata":{"annotations":{"cluster-autoscaler.kubernetes.io/safe-to-evict": "false"}}}}}'Edit the Deployment
kubectl -n kube-system edit deployment.apps/cluster-autoscalerAppend the following under spec.containers.command:
- --balance-similar-node-groups
- --skip-nodes-with-system-pods=false
Save and close the file with ESC and then wq to apply the changes.
Upgrade to minor release
Find the latest minor release for your kubernetes cluster version from https://github.com/kubernetes/autoscaler/releases. For example, if your cluster is running 1.22, look for the highest minor version with 1.22.x.
Upgrade to minor release with the following command:
kubectl set image deployment cluster-autoscaler \
-n kube-system \
cluster-autoscaler=k8s.gcr.io/autoscaling/cluster-autoscaler:v<1.22.x>View your Cluster Autoscaler logs
kubectl -n kube-system logs -f deployment.apps/cluster-autoscaler