Recently AWS released EKS Pod Identity which simplifies how cluster administrators can configure Kubernetes applications to obtain AWS IAM permissions. With Pod Identity you can assign an IAM role to a K8s service account using the new AWS API create-pod-identity-association and the EKS pod identity agent. When a pod starts with the associated service account, the identity agent will inject credentials so the pod can call AWS services.
Here we will create a simple demo allowing a pod to access S3 using EKS Pod Identity. Let’s get started!
EKS Pod Identity Restrictions
- Since we will be using new APIs, you will need to update aws-cli to the latest version.
- These are the compatible EKS Pod Identity cluster versions.
- Worker nodes in the cluster have to be Linux Amazon EC2 instances. Fargate and pods that run on Windows Amazon EC2 instances aren’t supported.
Install an EKS Cluster
Install EKS using whatever method you choose. I am using GitOps Playground to quickly spin up an EKS cluster for testing and then tear it down when done. I will name the cluster my-cluster.
Create a Test S3 Bucket
Create a simple, non-public bucket. Here I’m using the name lets-test-aws-pod-identity, but since bucket names must be unique, you’ll need to pick something else. Upload a text file to the root directory called test.txt.
Create Pod IAM Role
Create a new role called pod-idenity-s3-example with the new trust principal pods.eks.amazonaws.com:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
Attach an inline policy for read-only access to our bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetObject",
"Effect": "Allow",
"Resource": "arn:aws:s3:::lets-test-aws-pod-identity/*",
"Sid": "PodIdentity"
}
]
}
Install the EKS Pod Identity Agent
Use AWS create-addon to install the pod identity agent:
aws eks create-addon \ --cluster-name my-cluster \ --addon-name eks-pod-identity-agent \ --addon-version v1.0.0-eksbuild.1
Once the addon is installed, you can see the new DaemonSet running in EKS:
$ kubectl get pods -l app.kubernetes.io/instance=eks-pod-identity-agent -n kube-system NAME READY STATUS RESTARTS AGE eks-pod-identity-agent-k469t 1/1 Running 0 3h58m eks-pod-identity-agent-lvw6h 1/1 Running 0 3h58m eks-pod-identity-agent-pzjgl 1/1 Running 0 3h58m
Pod Identity Associations
Use the new AWS API create-pod-identity-association to attach our IAM role to an EKS service account and namespace:
aws eks create-pod-identity-association \ --cluster-name my-cluster \ --service-account pod-identity \ --role-arn arn:aws:iam:::role/pod-idenity-s3-example \ --namespace default
Now we are ready for testing!
Spin up a Test Pod
Create a pod with the service account pod-identity that will not run anything so we can access the pod’s shell for testing:
apiVersion: apps/v1
kind: Deployment
metadata:
name: pod-identity
labels:
app: pod-identity
spec:
replicas: 1
selector:
matchLabels:
app: pod-identity
template:
metadata:
labels:
app: pod-identity
spec:
serviceAccountName: pod-identity
containers:
- name: pod-identity
image: makoreactor/debug:latest
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: pod-identity
labels:
app: pod-identity
Once the test pod spawns, let’s jump into it using kubectl exec:
$ kubectl get pods NAME READY STATUS RESTARTS AGE pod-identity-6798cdbd9d-drswx 1/1 Running 0 39s $ kubectl exec -ti pod-identity-6798cdbd9d-drswx -- bash root@pod-identity-6798cdbd9d-drswx:~#
The eks-pod-identity-agent injected environment variables and a volume containing a service token into our test pod:
$ env | sort AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token AWS_CONTAINER_CREDENTIALS_FULL_URI=http://169.254.170.23/v1/credentials ... $ cat $AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE eyJhbGciOiJSUzI1N.....
Here the aws-cli will contact http://169.254.170.23/v1/credentials using the token and grab the credentials to assume the IAM role. However the current version of aws-cli doesn’t support the new metadata endpoint, so we will have to do it ourselves for now.
$ SERVICE_TOKEN=`cat /var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token`
$ curl 169.254.170.23/v1/credentials -H "Authorization: $SERVICE_TOKEN" | jq .
{
"AccessKeyId": "ASI...",
"SecretAccessKey": "Kmfg...",
"Token": "IQoJb3...",
"AccountId": "111111111111111",
"Expiration": "2023-12-01T03:41:43Z"
}
Now let’s export these credentials and check our caller-identity:
$ export AWS_ACCESS_KEY_ID=ASI...
$ export AWS_SECRET_ACCESS_KEY=Kmfg...
$ export AWS_SESSION_TOKEN="IQoJb3..."
$ aws sts get-caller-identity
{
"UserId": "AROA24SMTVXLXTKHVJCJU:my-cluster-act-pod-identity-06feb0a1-a0f3-4555-b509-6f14f1b59d05",
"Account": "111111111111111",
"Arn": "arn:aws:sts::111111111111111:assumed-role/pod-identity-s3-example/my-cluster-act-pod-identity-053..."
}
Now we should be good to go testing S3 access! Let’s try and pull the test.txt file to our pod:
$ aws s3 cp s3://lets-test-aws-pod-idenity/test.txt . download: s3://lets-test-aws-pod-idenity-j20a/test.txt to ./test.txt $ touch put.txt $ aws s3 cp put.txt s3://lets-test-aws-pod-idenity-j20a/ upload failed: ./put.txt to s3://lets-test-aws-pod-idenity-j20a/put.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Our IAM role worked as intended; we can only pull objects from S3, but cannot put objects.
Conclusion
EKS Pod Identity makes it much easier for EKS administrators to manage IAM roles within their clusters using an easy, straightforward method. As AWS updates the addon agent, I’ll make updates to this blog.
Don’t forget to tear down any resources you created for testing.