Leveraging EKS Pod Identity to Inject ASM Secrets: A Step-by-Step Guide

EKS Pod Identity is a feature that enables applications running on EKS to securely access AWS services, such as AWS Secrets Manager, without the need for hardcoding or managing access credentials. Instead, EKS Pod Identity uses IAM roles to grant permissions to pods, allowing them to interact with AWS services seamlessly.

In my last post, I showed an example of a pod fetching objects from S3 using pod identity. But let’s create a more real world example: using pod identity to inject ASM secrets into a pod.

Prerequisites

Before we dive into the implementation, ensure you have the following prerequisites in place:

• An active AWS account with an EKS cluster.
• AWS CLI installed and configured.
• Kubernetes command-line tool (kubectl) installed.
• Basic knowledge of deploying pods to EKS with a service account.

1. Create an ASM secret and IAM role

Here I created an ASM secret with the credentials to connect to my test database:

 $ cat <<EOF > db_secret.json 
{ 
  "DB_USERNAME":"super_admin", 
  "DB_PASSWORD":"Super_P@assword", 
  "DB_HOST":"test.coxy34rrmtiu.us-east-1.rds.amazonaws.com", 
  "DB_NAME":"test" 
}
EOF 

$ aws secretsmanager create-secret \ 
  --name db_creds \ 
  --secret-string file://db_secret.json

Now I’ll create the read-only IAM role to access this secret.

 $ cat <<EOF > trust-policy.json 
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "pods.eks.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
    ]
}
EOF 

$ aws iam create-role \ 
  --role-name Pod-ASM-Role \ 
  --assume-role-policy-document file://trust-policy.json

 $ cat <<EOF > pod-policy.json
{                           
    "Version": "2012-10-17",
    "Statement": [                        
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "arn:aws:secretsmanager:<region>:<account_id>:secret:db_creds"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "secretsmanager:ListSecrets",
            "Resource": "*"
        }
    ]
}
EOF

$ aws iam put-role-policy \ 
  --role-name Pod-ASM-Role \ 
  --policy-name Pod-ASM-RO \ 
  --policy-document file://pod-policy.json

2. Install the pod identity add-on

I’m adding the required EKS add-on called eks-pod-identity-agent.

$ aws eks create-addon \ 
  --cluster-name my-cluster \ 
  --addon-name eks-pod-identity-agent \ 
  --addon-version v1.0.0-eksbuild.1

3. Create a pod identity association to a Kubernetes service account and namespace

 
$ aws eks create-pod-identity-association \ 
  --cluster-name my-cluster \ 
  --service-account pod-identity \ 
  --role-arn arn:aws:iam::<account_id>:role/Pod-ASM-Role \ --namespace default 

4. Start pod and inject ASM secret

Now that the pod has access to the ASM secret, we can inject and use the secret as we see fit. For my example, I am running a simple Django application that uses environment variables to connect to a database. I created a pod in EKS using the service account I’ve named in the create-pod-identity-association command: pod-identity

Here is the entrypoint for the test Django image running in the pod:

#!/bin/bash

# Get ASM Secret
JSON_SECRET=`aws secretsmanager get-secret-value \
--secret-id db_creds  \
--query SecretString \
--output text`

# Convert JSON to environment variables
$( echo "$JSON_SECRET" | jq -r 'keys[] as $k | "export \($k)=\(.[$k])"' )


# Check env (demo only!)
env | sort

# Run app logic
python3 manage.py migrate 

When the pod runs, it will fetch the secret using aws secretsmanager get-secret-value, export the secrets as environment variables, and run Django migrations. Here is the output:

 $ env | sort 
DB_HOST=test.coxy34rrmtiu.us-east-1.rds.amazonaws.com 
DB_NAME=test 
DB_PASSWORD=[zOYFe?jVG$jH[.G!<LU4]c7|LL: 
DB_USERNAME=super_admin 
HOME=/root 
HOSTNAME=pod-identity-5c64d7d6cb-n7xrm 
.... 

$ python3 manage.py migrate 
Operations to perform: 
Apply all migrations: admin, auth, contenttypes, sessions 
Running migrations: Applying contenttypes.0001_initial... OK 
Applying auth.0001_initial... OK 
Applying admin.0001_initial... OK 
Applying admin.0002_logentry_remove_auto_add... OK 
Applying admin.0003_logentry_add_action_flag_choices... OK 
Applying contenttypes.0002_remove_content_type_name... OK 
Applying auth.0002_alter_permission_name_max_length... OK 
Applying auth.0003_alter_user_email_max_length... OK 
Applying auth.0004_alter_user_username_opts... OK 
Applying auth.0005_alter_user_last_login_null... OK 
Applying auth.0006_require_contenttypes_0002... OK 
Applying auth.0007_alter_validators_add_error_messages... OK 
Applying auth.0008_alter_user_username_max_length... OK 
Applying auth.0009_alter_user_last_name_max_length... OK 
Applying auth.0010_alter_group_name_max_length... OK 
Applying auth.0011_update_proxy_permissions... OK 
Applying auth.0012_alter_user_first_name_max_length... OK 
Applying sessions.0001_initial... OK 

Conclusion

EKS Pod Identity simplifies the process of integrating AWS Secrets Manager, providing a seamless solution for managing and retrieving secrets securely in a Kubernetes environment.

Please feel free to share your strategies on how you implemented injecting secrets for your app!