Automating Let’s Encrypt Certificate Renewal using DNS Challenge Type

by
Tags: , ,
Category:

Let’s Encrypt makes the automation of renewing certificates easy using certbot and the HTTP-01 challenge type. However when using the HTTP challenge type, you are restricted to port 80 on the target running certbot. This can be cumbersome if you have multiple certificates, and personally I don’t like having port 80 open inside my network. The DNS challenge type fixes these issues, however automating the process is not as straightforward. With DNS, certbot will ask the enduser to manually create a TXT record with a token in their domain, then click enter so letsencrypt can validate if that record exists. Obviously this won’t work if you want to automate the process, luckily certbot comes with the --manual-auth-hook and --manual-cleanup-hook options which will let you run a custom script that will create the TXT record with token automatically. The structure of your script will depend on how you create/change records for your domain.

In the following examples, I’ll show how to renew certs with domains hosted on AWS/Route53 and GoDaddy. I run certbot with scripts within a docker container (to simplify automation), however you can use CLI.

Note: I put line breaks in my scripts for clarity.

AWS/Route53

Create the following scripts in a single directory:

  1. aws.sh – Script will create the TXT validation record
  2. aws.json – JSON required by aws-cli
  3. aws-clean.sh – Script will remove the TXT record after validation.
  4. aws-clean.json – JSON required by aws-cli
  5. Dockerfile – (Optional) build container to run certbot
  6. entrypoint.sh – (Option) entrypoint file for the container

aws.sh

#!/bin/bash
#aws.sh
 
export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|
	 
DNS_REC_NAME="_acme-challenge"
DNS_REC_DATA="$CERTBOT_VALIDATION"
 
# Replace string with validation and TXT name
sed -i "s/CHANGEME/$DNS_REC_DATA/g" /aws.json
sed -i "s/NAME/$DNS_REC_NAME.$CERTBOT_DOMAIN/g" /aws.json
 
# Create TXT record
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws.json

# Sleep to make sure DNS propagates before lets encrypt validates.
sleep 30

Certbot will run this script to create the TXT validation record in Route53. The script will use aws-cli, so you will need an AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

certbot will provide to the script the following variables:
1. $CERTBOT_DOMAIN: The domain(The -d option in your certbot command).
2. $CERTBOT_VALIDATION: The token let’s encrypt will supply for validation.

Lets go through the script:

export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|

DNS_REC_NAME="_acme-challenge."
DNS_REC_DATA="$CERTBOT_VALIDATION"

All the environment variables for the script. You will need to add your AWS ID and KEY. Note: If you are adding your AWS ID and secret in plaintext, be careful where you store the script.

# Create TXT record
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws.json

The aws-cli command that will create the TXT record with the proper validation. You will need to change the |domain_zone_id| to your own.

aws.json

{
  "Comment": "aws.json",
  "Changes": [ {
     "Action": "CREATE",
     "ResourceRecordSet": {
        "Name": "NAME",
            "Type": "TXT",
             "TTL": 600,
          "ResourceRecords": [{"Value": "\"CHANGEME\""}]
              }}
]
}

The JSON file required by the aws route53 change-resource-record-sets command. The NAME and CHANGEME values will be updated by aws.sh.

aws-clean.sh

#!/bin/bash
#aws-clean.sh

export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|

############################################################

DNS_REC_NAME="_acme-challenge"
DNS_REC_DATA="$CERTBOT_VALIDATION"
 
# Replace string with TXT name
sed -i "s/CHANGEME/$DNS_REC_DATA/g" /aws-clean.json
sed -i "s/NAME/$DNS_REC_NAME.$CERTBOT_DOMAIN/g" /aws-clean.json
 
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws-clean.json

aws-clean.sh will run after let’s encrypt validates the domain and remove the validation TXT record. Again, you will need to change the |domain_zone_id|.

aws-clean.json

{
  "Comment": "aws-clean.json",
  "Changes": [ {
      "Action": "DELETE",
      "ResourceRecordSet": {
          "Name": "NAME",
              "Type": "TXT",
	      "TTL": 600,
	      "ResourceRecords": [{"Value": "\"CHANGEME\""}]
      }}
  ]
}

The JSON file required by the aws route53 change-resource-record-sets command. The NAME and CHANGEME values will be updated by aws-clean.sh.

Dockerfile

FROM ubuntu:bionic-20191029

RUN apt update

#Enter your timezone
RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone
RUN apt -y install certbot curl awscli

COPY aws.sh /aws.sh
COPY aws.json /aws.json
COPY aws-clean.sh /aws-clean.sh
COPY aws-clean.json /aws-clean.json

COPY entrypoint.sh /usr/bin/
ENTRYPOINT entrypoint.sh

You’ll have to change the timezone in the RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone section to your own timezone. 2 Methods To Change TimeZone in Linux will help you.

entrypoint.sh

#!/bin/bash

certbot -d example.com --agree-tos --register-unsafely-without-email --manual \
--preferred-challenges dns --manual-auth-hook /aws.sh \
--manual-cleanup-hook /aws-clean.sh --manual-public-ip-logging-ok \
--force-renewal certonly
# Add logic to handle the certs/keys when they are issued.

Add your domain to the -d flag. This should be the full CN part of the certificate. Certs will be saved to /etc/letsencrypt/live/example.com/. Add to entrypoint.sh how you want to handle the certificates and keys after they are issued (copy them to volume, add them to your vault, etc.)

Now lets build and run!

$ chmod +x *.sh
$ docker build -t certbot-manager .
$ docker run certbot-manager

GoDaddy

The Godaddy scripts will update the TXT records via Godaddy’s API. When creating keys, make sure to choose the production environment.

Create the following scripts in a single directory:

  1. gdaddy.sh – Script will create the TXT validation record
  2. gdaddy-clean.sh – Script will change the TXT record value to “clean”.
  3. Dockerfile – (Optional) build container to run certbot
  4. entrypoint.sh – (Option) entrypoint file for the container

Work At Chariot

If you value continual learning, a culture of flexibility and trust, and being surrounded by colleagues who are curious and love to share what they’re learning (with articles like this one, for example!) we encourage you to join our team. Many positions are remote — browse open positions, benefits, and learn more about our interview process below.

Careers

gdaddy.sh

#!/bin/bash

GODADDY_API_KEY="|godaddy_api_key|"
GODADDY_API_SECRET="|godaddy_api_secret|"
GODADDY_URL="https://api.godaddy.com/"
############################################################

# Replace all of a Domain's type of DNS Records

DNS_REC_TYPE=TXT
DNS_REC_NAME="_acme-challenge"
DNS_REC_DATA="$CERTBOT_VALIDATION"
DNS_REC_TTL="600"

IFS='.' read -ra ADDR <<< $CERTBOT_DOMAIN
if [[ ${#ADDR[@]} > 2 ]]; then
        DNS_REC_NAME=`echo $CERTBOT_DOMAIN | cut --delimiter="." -f 1`;
        DNS_REC_NAME=`echo _acme-challenge.$DNS_REC_NAME`;
fi

curl -X PUT \
"${GODADDY_URL}/v1/domains/${CERTBOT_DOMAIN}/records/${DNS_REC_TYPE}/${DNS_REC_NAME}" \
-H  "accept: application/json" -H  "Content-Type: application/json" \
-H  "Authorization: sso-key ${GODADDY_API_KEY}:${GODADDY_API_SECRET}" \
-d "[{ \"data\": \"${DNS_REC_DATA}\", \"name\": \"${DNS_REC_NAME}\", \"ttl\": ${DNS_REC_TTL} }]"

# Sleep to make sure DNS propagates before lets encrypt validates.
sleep 30

Certbot will run this script to create the TXT validation record in Godaddy.

certbot will provide to the script the following variables:
1. $CERTBOT_DOMAIN: The domain(The -d option in your certbot command).
2. $CERTBOT_VALIDATION: The token let’s encrypt will supply for validation.

Lets go through the script:

GODADDY_API_KEY="|godaddy_api_key|"
GODADDY_API_SECRET="|godaddy_api_secret|"
GODADDY_URL="https://api.godaddy.com/"

Replace GODADDY_API_KEY and GODADDY_API_SECRET with your Godaddy API credentials.

curl -X PUT \
"${GODADDY_URL}/v1/domains/${CERTBOT_DOMAIN}/records/${DNS_REC_TYPE}/${DNS_REC_NAME}" \
-H  "accept: application/json" -H  "Content-Type: application/json" \
-H  "Authorization: sso-key ${GODADDY_API_KEY}:${GODADDY_API_SECRET}" \
-d "[{ \"data\": \"${DNS_REC_DATA}\", \"name\": \"${DNS_REC_NAME}\", \"ttl\": ${DNS_REC_TTL} }]"

Curl will authenticate to Godaddy and create/update the TXT record with vaild name and token.

gdaddy-clean.sh

#!/bin/bash
GODADDY_API_KEY="|godaddy_api_key|"
GODADDY_API_SECRET="|godaddy_api_secret|"
GODADDY_URL="https://api.godaddy.com/"
############################################################
 
# Replace all of a Domain's type of DNS Records
 
DNS_REC_TYPE=TXT
DNS_REC_NAME="_acme-challenge"
DNS_REC_TTL="600"

IFS='.' read -ra ADDR <<< $CERTBOT_DOMAIN
if [[ ${#ADDR[@]} > 2 ]]; then
        DNS_REC_NAME=`echo $CERTBOT_DOMAIN | cut --delimiter="." -f 1`;
        DNS_REC_NAME=`echo _acme-challenge.$DNS_REC_NAME`;
fi

curl -X PUT \
"${GODADDY_URL}/v1/domains/${CERTBOT_DOMAIN}/records/${DNS_REC_TYPE}/${DNS_REC_NAME}" \
-H  "accept: application/json" -H  "Content-Type: application/json" \
-H  "Authorization: sso-key ${GODADDY_API_KEY}:${GODADDY_API_SECRET}" \
-d "[{ \"data\": \"clean\", \"name\": \"${DNS_REC_NAME}\", \"ttl\": ${DNS_REC_TTL} }]"

Certbot will run this script to change the TXT validation token to “clean”.

Dockerfile

FROM ubuntu:bionic-20191029

RUN apt update

#Enter your timezone
RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone
RUN apt -y install certbot curl 

COPY gdaddy.sh /gdaddy.sh
COPY gdaddy-clean.sh /gdaddy-clean.sh

COPY entrypoint.sh /usr/bin/
ENTRYPOINT entrypoint.sh

You’ll have to change the timezone in the RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone section to your own timezone. 2 Methods To Change TimeZone in Linux will help you.

entrypoint.sh

#!/bin/bash

certbot -d example.com --agree-tos --register-unsafely-without-email --manual \
--preferred-challenges dns --manual-auth-hook /gdaddy.sh \
--manual-cleanup-hook /gdaddy-clean.sh --manual-public-ip-logging-ok \
--force-renewal certonly
# Add logic to handle the certs/keys when they are issued.

Add your domain to the -d flag. This should be the full CN part of the certificate. Certs will be saved to /etc/letsencrypt/live/example.com/. Add to entrypoint.sh how you want to handle the certificates and keys after they are issued (copy them to volume, add them to your vault, etc.)

Now lets build and run!

$ chmod +x *.sh
$ docker build -t certbot-manager .
$ docker run certbot-manager

Conclusion

Hopefully this will steer you in the right direction automating your Let’s Encrypt renewal process. Good luck!