data lakes

S3 Table Buckets vs Redshift

AWS released S3 Table Buckets at re:Invent 2024, and at release they were pretty much only usable with Elastic Map Reduce. However, over the past year, the S3 Tables team has been making improvements. And while there are still some limitations, S3 Tables with Athena gives a user experience similar to traditional data warehouses such as Redshift.

Which leads to the question: can Athena and S3 Tables be a cost-effective replacement for Redshift? In this post I show how to use S3 Tables, and run some performance comparisons to answer that question.

D2C242: Data Engineering and its Streams, Rivers, and Lakes

Keith Gregory teaches Day Two Cloud about data engineering in a way DevOps folks (and hydrologists) can understand. He explains that the role of a data engineer is to create pipelines to transport data from metaphorical rivers and make it usable for data analysts. Keith walks us through the testing process; the difference between streaming … Read More