Abstract
In this talk, we’ll present dbt, an open-source data transformation framework that lets analysts and engineers collaborate using their shared knowledge of SQL.
We’ll dive deep into the analyst persona and how dbt empowers them to produce data that their entire organization can trust independently of software engineers.
We’ll explore how dbt embraces software engineering best practices like modularity, version control, testing, and documentation to help analytics teams ship as effectively as software teams.
Using a fictional e-commerce store as an example, we’ll also illustrate how a transformation written in dbt is simpler and cleaner than its Python equivalent.
We’ll then look at how dbt serves as the nexus for analytics workflows at Spotify. We’ll explore what Spotify’s tech stack used to look like, why they adopted dbt, and what the dbt experience looks like for different stakeholders within the organization.
Finally, we’ll cover what’s on dbt’s roadmap – power user tools like multi-project deployments and a Python SDK – built for software engineers.
About Kshitij
Aranke Kshitij is a senior software engineer at dbt Labs, working on the popular open-source data transformation tool dbt. Previously, he worked in analytics at Vouch Insurance (using dbt!) and in big data at Amazon and LinkedIn. He is based in San Francisco and listens to podcasts on long walks in his spare time.
About Brandon Segal
Brandon Segal is a Philadelphian working as a Senior Analytics Engineer at Spotify, working within their internal data platform team to understand how the teams at Spotify are collecting data and transforming it into insights. His ways of working have been greatly influenced by his previous experience as a data scientist and software engineer at Capital One with their ML feature platform. Since joining Spotify, he has worked on bringing software engineering best practices to his insights teams by introducing standard practices like semantic versioning, integration testing, and monitoring to improve data product quality.
About the Conference
The Philly Emerging Technologies for the Enterprise (ETE) is the Mid-Atlantic’s premier developer’s conference. Entering its 17th year, we’ve brought world-class speakers — including some local favorites — to speak about leading-edge technologies being used today, and emerging technologies that will be important for attendees to know about in the near future.
Watch More
Check out our YouTube playlist to watch all the talks from Emerging Technologies for the Enterprise 2023.