Repository navigation
perf: reduce glob traversal and Buffer write overheads - #29
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
test(performance): publish reproducible API comparisons and fix review findings
test(performance): compare shared upstream filesystem workloads
Problem and native value
Four compatible glob patterns previously scanned the same tree four times and locked shared result vectors per match. This change shares walkers for identical roots/hidden policies/root inclusion, collects results per worker, and skips cross-walk deduplication when only one walk remains. Separate roots retain their previous traversal semantics.
Buffer writes previously copied synchronous input once and async input twice. Sync calls now borrow bytes; async calls keep their entry snapshot and write it directly, preserving mutation isolation and slice boundaries. Snapshots are released on the worker after I/O, without waiting for JS Promise resolution. String encoding and public routing/signatures are unchanged.
Based on #28 (
feat/node-api-alignment). The integrated change also includes the reproducible peer comparisons and official documentation from #30/#31, with glob traversal/exclusion and Windows scan metadata corrections.Compatibility and tests
Performance evidence
Detailed report, Node 22 raw samples, Node 24 raw samples, reproduction harness.
Release/public-entry comparison against db835af on Apple M4 Pro, macOS arm64. Two warmups, ten measured samples, rotating Node/before/after order, median latency; warm local filesystem; exact results validated every sample. Glob concurrency 4, write concurrency 1, no explicit flush. Source/binary hashes and per-sample RSS deltas are recorded.
Both runtimes show improvement over the previous 8 MiB Buffer paths in final runs. Small writes have mixed results/regressions; exploratory disk-writeback noise also affected large writes. Single-pattern glob is broadly comparable; no independent speedup is claimed for the collector alone. RSS deltas are not peak memory; worker-local vectors may retain spare capacity and overlap the merged allocation. One payload-sized redundant allocation is removed from each Buffer operation, not all copies or filesystem work.
Documentation and validation
No package publication is included. The integrated stack has 468 passing local tests per Node 22/24 runtime plus 12 publication/adapter contract checks; final platform CI passed and source-tree equality is verified during integration. Current per-API evidence retains all wins, losses and Node fallback routes.
Final verified code tree:
27d9beaac4576bdb74ddafb4a7bfceb58ff7bb06. Passing cross-platform CI. Integration preserves this exact tree.