Data Engineering
Data platforms, warehouses, CDC and streaming ingestion, data contracts and analytics pipelines.
RDS Postgres to Snowflake: Migration Patterns and Choosing the Right Approach
Getting data from an RDS Postgres database into Snowflake sounds like a solved problem. It mostly is, but there are at least five patterns to choose from, and the wrong one will cost you in freshness, money or on-call pain. Here's how I think about each.
Snowpipe Streaming: Getting Data into Snowflake in Seconds, Not Minutes
Classic Snowpipe loads files from cloud storage and is measured in minutes. Snowpipe Streaming writes rows directly into tables with seconds of latency, and its newer high-performance architecture changes both the throughput ceiling and the pricing model. Here's when it's worth it.
GTM vs Segment: Tag Manager or Customer Data Pipeline (and Why Many Teams Run Both)
Google Tag Manager and Segment get compared as if they were alternatives. They solve different problems: one decides which tags fire in the browser, the other is a pipeline for your event data. Here's how I think about each, the costs, and when to use both.
Data Contracts: Stopping Application Changes from Breaking the Warehouse
Most broken dashboards I've seen weren't caused by bad SQL. They were caused by a perfectly reasonable application change that nobody told the data team about. Data contracts are how I stop that happening, and they're less bureaucratic than they sound.
Building a Data Platform with Redshift, Looker, and dbt
In 2018-19, TouchNote needed to move from ad-hoc SQL queries to a proper analytics platform. Here's how we built our data stack with Redshift, Looker, and dbt, and what we'd do differently.