Blog

Delving into CloudTrail events

CloudTrail provides you with an audit log of every successful API call made in your AWS account. It can be invaluable for security auditing, as well as answering general questions such as “who started this instance and when?” The chief drawbacks are that it produces a large number and variety of events, making analysis challoenging. However, a search engine such as Elasticsearch with Kibana lets you explore your audit log using simple filters and search terms.

Using AWS CostExplorer to find forgotten services

One of the risks with giving developers their own sandboxes is that they’ll forget to shut down EC2 instances, or RDS databases, or any of the other AWS services that come with a per-hour charge. It happens. I’ve done it, as have most of the developers I know. But there’s no reason to be surprised when the bill arrives. In this post I’ll give an introduction to Budgets, and walk through using Cost Explorer to find a forgotten Sagemaker notebook.

Automating Let’s Encrypt Certificate Renewal using DNS Challenge Type

Let’s Encrypt makes the automation of renewing certificates easy using certbot and the HTTP-01 challenge type. However when using the HTTP challenge type, you are restricted to port 80 on the target running certbot. This can be cumbersome if you have multiple certificates, and personally I don’t like having port 80 open inside my network. The DNS challenge type fixes these issues, however automating the process is not as straightforward. With DNS, certbot will ask the enduser to manually create a TXT record with a token in their domain, then click enter so letsencrypt can validate if that record exists. Obviously this won’t work if you want to automate the process, luckily certbot comes with the --manual-auth-hook and --manual-cleanup-hook options which will let you run a custom script that will create the TXT record with token automatically. The structure of your script will depend on how you create/change records for your domain.

In the following examples, I’ll show how to renew certs with domains hosted on AWS/Route53 and GoDaddy. I run certbot with scripts within a docker container (to simplify automation), however you can use CLI.

Note: I put line breaks in my scripts for clarity.

AWS/Route53

Create the following scripts in a single directory:

  1. aws.sh – Script will create the TXT validation record
  2. aws.json – JSON required by aws-cli
  3. aws-clean.sh – Script will remove the TXT record after validation.
  4. aws-clean.json – JSON required by aws-cli
  5. Dockerfile – (Optional) build container to run certbot
  6. entrypoint.sh – (Option) entrypoint file for the container

aws.sh

#!/bin/bash
#aws.sh
 
export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|
	 
DNS_REC_NAME="_acme-challenge"
DNS_REC_DATA="$CERTBOT_VALIDATION"
 
# Replace string with validation and TXT name
sed -i "s/CHANGEME/$DNS_REC_DATA/g" /aws.json
sed -i "s/NAME/$DNS_REC_NAME.$CERTBOT_DOMAIN/g" /aws.json
 
# Create TXT record
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws.json

# Sleep to make sure DNS propagates before lets encrypt validates.
sleep 30

Certbot will run this script to create the TXT validation record in Route53. The script will use aws-cli, so you will need an AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

certbot will provide to the script the following variables:
1. $CERTBOT_DOMAIN: The domain(The -d option in your certbot command).
2. $CERTBOT_VALIDATION: The token let’s encrypt will supply for validation.

Lets go through the script:

export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|

DNS_REC_NAME="_acme-challenge."
DNS_REC_DATA="$CERTBOT_VALIDATION"

All the environment variables for the script. You will need to add your AWS ID and KEY. Note: If you are adding your AWS ID and secret in plaintext, be careful where you store the script.

# Create TXT record
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws.json

The aws-cli command that will create the TXT record with the proper validation. You will need to change the |domain_zone_id| to your own.

aws.json

{
  "Comment": "aws.json",
  "Changes": [ {
     "Action": "CREATE",
     "ResourceRecordSet": {
        "Name": "NAME",
            "Type": "TXT",
             "TTL": 600,
          "ResourceRecords": [{"Value": "\"CHANGEME\""}]
              }}
]
}

The JSON file required by the aws route53 change-resource-record-sets command. The NAME and CHANGEME values will be updated by aws.sh.

aws-clean.sh

#!/bin/bash
#aws-clean.sh

export AWS_ACCESS_KEY_ID=|aws_id|
export AWS_SECRET_ACCESS_KEY=|aws_secret|

############################################################

DNS_REC_NAME="_acme-challenge"
DNS_REC_DATA="$CERTBOT_VALIDATION"
 
# Replace string with TXT name
sed -i "s/CHANGEME/$DNS_REC_DATA/g" /aws-clean.json
sed -i "s/NAME/$DNS_REC_NAME.$CERTBOT_DOMAIN/g" /aws-clean.json
 
aws route53 change-resource-record-sets --hosted-zone-id |domain_zone_id| \
--change-batch file://aws-clean.json

aws-clean.sh will run after let’s encrypt validates the domain and remove the validation TXT record. Again, you will need to change the |domain_zone_id|.

aws-clean.json

{
  "Comment": "aws-clean.json",
  "Changes": [ {
      "Action": "DELETE",
      "ResourceRecordSet": {
          "Name": "NAME",
              "Type": "TXT",
	      "TTL": 600,
	      "ResourceRecords": [{"Value": "\"CHANGEME\""}]
      }}
  ]
}

The JSON file required by the aws route53 change-resource-record-sets command. The NAME and CHANGEME values will be updated by aws-clean.sh.

Dockerfile

FROM ubuntu:bionic-20191029

RUN apt update

#Enter your timezone
RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone
RUN apt -y install certbot curl awscli

COPY aws.sh /aws.sh
COPY aws.json /aws.json
COPY aws-clean.sh /aws-clean.sh
COPY aws-clean.json /aws-clean.json

COPY entrypoint.sh /usr/bin/
ENTRYPOINT entrypoint.sh

You’ll have to change the timezone in the RUN ln -snf /usr/share/zoneinfo/America/New_York /etc/localtime && echo America/New_York > /etc/timezone section to your own timezone. 2 Methods To Change TimeZone in Linux will help you.

entrypoint.sh

#!/bin/bash

certbot -d example.com --agree-tos --register-unsafely-without-email --manual \
--preferred-challenges dns --manual-auth-hook /aws.sh \
--manual-cleanup-hook /aws-clean.sh --manual-public-ip-logging-ok \
--force-renewal certonly
# Add logic to handle the certs/keys when they are issued.

Add your domain to the -d flag. This should be the full CN part of the certificate. Certs will be saved to /etc/letsencrypt/live/example.com/. Add to entrypoint.sh how you want to handle the certificates and keys after they are issued (copy them to volume, add them to your vault, etc.)

Now lets build and run!

$ chmod +x *.sh
$ docker build -t certbot-manager .
$ docker run certbot-manager

Building Developer Sandboxes on AWS

The ability to experiment is one of the unsung benefits of cloud computing. It was, in fact what drew me to AWS in 2008. At Chariot, we have multiple sandbox environments, some for specific projects and some for general play, and recommend that our clients do the same. However, sandboxes need some controls, to ensure that they don’t become a source of runaway costs.

Talk Tech To Me – User Centered Software Design

Here is Sue Spolan’s summary of my Talk Tech To Me interview: User Centered Software Design: The Secret to Success  Consumers interact with software many times a day. It might be a mobile app like Instagram, or a web app like Gmail, and while users don’t think a whole lot about the underlying design, it’s an essential part of getting the customer experience right. It’s the secret to a successful software solution.  Peter Fleming is the head of user experience…

GraphQL, the new Contender to REST

The Representative State Transfer (REST) protocol has been the king of remote access protocols for web applications for well over a decade. The general pattern: expose “nouns” (Customers, Activities, Employees, Tasks, Sasquatches) as URLs (/api/sasquatch/32) and access them via HTTP “verbs” such as “GET”, “POST” (create), “PUT” (update), or “DELETE” (umm, well…). The content type is specified via HTTP headers such as Content-Type (for data being received by the client) and Accepts (for a data request). The reason this works…

Amazon Workspaces – Desktops in the Cloud

Let’s say you’re traveling to a conference, and you want to leave your heavy laptop behind along with all of its power requirements. Or you have a long-running project that you manage from just about anywhere and want to keep a desktop active or suspended/resumed whenever you want. Or, you’re a company that wants to roll out the same desktop to many different people but doesn’t want to worry about the user’s computer horsepower. Amazon Workspaces fits those parameters. I’ve…

re:Invent Recap

AWS re:Invent is over, and I finally have time to breathe. I’ve been going to the NY Summit since it was “only” a few thousand people and watched it grow into a 12,000-person behemoth, but this was my first re:Invent. I assumed it would be like the Summit, but bigger. I was wrong. Summits are single-day events, and half of that day is taken by the keynote, leaving time for just a few sessions for the afternoon. By comparison, re:Invent…

How to supercharge your wireframes

Why go to the effort? The thought and preparation that go into wireframes make a drastic difference on the outcome. Wireframes come in a spectrum of resolutions, but the important thing is to find the right balance of fidelity, examples, and annotation for the project and team. Wireframes should be thought of as sketches for stakeholders to see the vision and instructions for developers to build the idea. They should not be thought of as works of art or a…

1 2 35