[rustyconover]

Builds on #27.

20M rows, 2M groups, 1 GB limit, 8 threads	before	after
datasketch_hll(12, v)	out of memory after 13.1 s	6.9 s, 0.82 GB peak
datasketch_cpc(12, v)	10.4 s, 3.6 GB	7.7 s, 0.77 GB

HLL

• Buffer: the create aggregate keeps its first 7 distinct values in the arena: fixed-width values as raw bytes, and VARCHAR/BLOB values with their bytes copied in. They go to a union, in order, only when more values arrive, at finalize, or when merging with a state that already has a union.
• Why 7: a union in its simplest mode (LIST) holds up to 7 coupons, and in that mode merging feeds coupons in arrival order. So buffered values give the same union as merging two unions.
• Finalize and export: a buffered state finalizes through a temporary union on the heap and exports as raw values. Import rebuilds the buffer.

CPC

• Arena states: datasketch_cpc and datasketch_cpc_union now use arena states, and create states buffer 16 values.
• Merging: when both partial states hold values, they still merge through a union on the heap, as before, and the result is copied into the arena sketch. A merged CPC result is flagged as merged and loses its more accurate estimator, so appending one buffer to another would change the result.

Also in this PR

• Constant input: it is added once, because both sketches absorb a repeated value only once.
• Cleanup: the old heap HLL/CPC states are removed, and the generator now requires arena=True for counting sketches.

Tests

New test/sql/datasketch_spill_counting.test, under 128 MB with 4 threads:

• every input type family over 100k groups
• a single-thread run that checks spilled HLL sketches are byte-identical to unspilled ones
• merging stored sketches
• EXPORT_STATE with finalize and combine
• the constant-input path

All tests pass.

🤖 Generated with Claude Code