Keith Gregory

Serverless Databasing with the Aurora Data API

Earlier this year I wrote that Amazon Aurora Serverless allows you to implement a fully serverless application with a relational database. In this post I show you how to use the Data API to do just that.

Rightsizing Data for Athena

Amazon Athena is a service that lets you run SQL queries against structured data files stored in S3. It takes a “divide and conquer” approach, spinning up parallel query execution engines that each examine only a portion of your data. The performance of these queries, however, depends on how you consolidate and partition your data. In this post I compare query times for a moderately large dataset, looking for the “sweet spot” between number of files and individual file size.

Using CodeArtifact with Poetry

In my last post I discussed how an artifact server is the best way to publish locally-developed Python packages. In this post, I show you how to set up the AWS CodeArtifact service and use it with pip and Poetry.

Building Lambdas with Poetry

Coming from a Java background, I consider the Python development process to be a bit of a mess. The pieces are all there: a central repository for publicly-available packages, a way to install the packages you want, and several ways to run your program with only those packages. But it seems that everybody has a different way to combine those pieces. So when a colleague introduced me to Poetry, my first reaction was “oh great, another tool that solves part of my problem.” But after spending time with it, I don’t want to build Lambdas any other way.

Three Approaches to Deploying Lambdas

“Traditional” deployment patterns separate the application from its infrastructure. Lambda deployments turn this model on its head, binding the infrastructure tightly to the running code. This can be a challenge, especially when developing in a team: it is all too easy for one developer to accidentally overwrite another’s work. In this post I look at several deployment options, and how they impact a development team.

Building and Deploying Lambdas from a Docker Container

In my last post, I looked at how you could package your Lambda as a Docker image. In this post, I show how you can use the base Amazon images to build a “traditional” Lambda and ensure that it has libraries that are appropriate for the Lambda runtime environment.

Getting Started with Lambda Container Images

Lambda Container Images were announced at re:Invent 2020, providing a new way to build and deploy Lambda functions. They arrived just in time to solve an annoying build problem for me, so got my attention. And there weren’t any tutorials floating around when I first Googled, so I figured it was worth writing one. But … Read More

Running the SAM CLI on Linux

I find the SAM (Serverless Application Model) CLI extremely frustrating to use on Linux, starting with installation. But this week I learned two things that simplify both installation and operation. I’m passing them on in the hopes that they’ll be useful to you as well.