Datafusion

dbt Core v2: 77 Rust crates, 18 Python files, no Python runtime

dbt rewrote itself in Rust. The speed is the boring part.

I opened a bug report against dbt and found a codebase nobody had told me about: 77 Rust crates where the Python used to be. This is what the rewrite is actually for, why the marketing undersells it, and what I broke loose while poking at it — two fixes, both for bugs that never printed an error.

Guardrails for Tabular ML: A Data Engineer's Take on Data Leakage, Poisoning, and Brittle Pipelines

Most ML pipeline failures are not exotic model bugs — they are data issues that nobody encoded as checks. This article walks through building guardrails using pandas, Apache DataFusion, data contracts, and the Arrow C Data Interface.

Designing a Data Profiler Around Apache Arrow: Lessons from dataprof

A design story of dataprof: why I built a profiler around Apache Arrow, how it changed the architecture, and how this journey led to contributions to arrow-rs’ Parquet reader.