Tagged “pipelines”
3 posts
Data freshness and schema-drift alerting
A pipeline can run successfully and still be wrong. The two failures that matter most are silent by nature — the data stopped arriving, or it quietly changed shape — and neither shows up in a green DAG. Here's what to measure, and how lineage turns an alarm into an assignment.
|7 min readdlt REST API sources: declarative pagination without the glue
The schema glue is gone, but most teams still hand-write the other half of every API pipeline — pagination loops, auth headers, and "what changed since last time." dlt's REST API source turns all of it into configuration. Here's the declarative shape, end to end.
|7 min readData pipelines with dlt and DuckDB
Most pipeline code is glue nobody wants to maintain. dlt and DuckDB let you skip the glue and keep the parts that matter — schema inference, incremental loading, and contracts that fail loudly instead of silently corrupting your warehouse.
|6 min read