A good start is to follow AWS documentations (https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html)
Pre-requisites
- eksctl (https://github.com/weaveworks/eksctl)
- Metrics Server (https://alexlogy.io/how-to-install-metrics-server-in-aws-eks/)
Create IAM Policy
Save the following content to a file.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/k8s.io/cluster-autoscaler/<my-cluster>": "owned"
}
}
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeAutoScalingGroups",
"ec2:DescribeLaunchTemplateVersions",
"autoscaling:DescribeTags",
"autoscaling:DescribeLaunchConfigurations"
],
"Resource": "*"
}
]
}
Create IAM role and attach IAM policy
eksctl create iamserviceaccount \
--cluster=<my-cluster> \
--namespace=kube-system \
--name=cluster-autoscaler \
--attach-policy-arn=arn:aws:iam::<111122223333>:policy/<AmazonEKSClusterAutoscalerPolicy> \
--override-existing-serviceaccounts \
--approve
Download Cluster Autoscaler YAML
curl -o cluster-autoscaler-autodiscover.yaml https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml
Apply the YAML
kubectl apply -f cluster-autoscaler-autodiscover.yaml
Annotate the Service Account
kubectl annotate serviceaccount cluster-autoscaler \
-n kube-system \
eks.amazonaws.com/role-arn=arn:aws:iam::<ACCOUNT_ID>:role/<AmazonEKSClusterAutoscalerRole>
Patch the deployment
kubectl patch deployment cluster-autoscaler \
-n kube-system \
-p '{"spec":{"template":{"metadata":{"annotations":{"cluster-autoscaler.kubernetes.io/safe-to-evict": "false"}}}}}'
Edit the Deployment
kubectl -n kube-system edit deployment.apps/cluster-autoscaler
Append the following under spec.containers.command:
- --balance-similar-node-groups
- --skip-nodes-with-system-pods=false
Save and close the file with ESC and then wq to apply the changes.
Upgrade to minor release
Find the latest minor release for your kubernetes cluster version from https://github.com/kubernetes/autoscaler/releases. For example, if your cluster is running 1.22, look for the highest minor version with 1.22.x.
Upgrade to minor release with the following command:
kubectl set image deployment cluster-autoscaler \
-n kube-system \
cluster-autoscaler=k8s.gcr.io/autoscaling/cluster-autoscaler:v<1.22.x>
View your Cluster Autoscaler logs
kubectl -n kube-system logs -f deployment.apps/cluster-autoscaler