arrow-abi-doctor

Differential testing for the Arrow C Data and C Stream interfaces.

A test harness that builds portable, minimized reproducers of Arrow C structures, hands them to real consumers in isolated processes, and observes the whole lifecycle: imports, moves, releases, leaks, and the data that comes back.

Updated from public sources · 2026-10-04

One corpus, six consumers

Milestone 1 runs a frozen conformance model of 5,650 cells against a null control, nanoarrow as the reference validator, PyArrow, Arrow C++ and DuckDB built with sanitizers, and dataprof. A consumer agrees when the lifecycle is clean, nothing leaks, and the returned data has the digest of what was sent. Cells a consumer cannot run are reported as not run, with the reason.

Findings that went upstream

The harness found two FFI defects in dataprof, both fixed, and a zero-length Utf8 slice at a non-zero offset that overran its buffer in arrow-rs, fixed in apache/arrow-rs#10916. Misaligned loads on unaligned buffers were fixed in nanoarrow and are reproduced but still open in DuckDB. It tests trusted-but-defective producers; it does not harden against malicious ones.