Repository navigation
backport: bitcoin#26684 (partial), #28921, #29458, #28339, #28770, #30344, #28263, #28209 - #8178
PastaPastaPasta wants to merge 8 commits into
Conversation
1c4b9cb bench: add readblock benchmark (Andrew Toth) Pull request description: Requested in bitcoin#13151 (comment). See bitcoin#26415 and bitcoin#21319. Benchmarking shows a >50x increase in speed on both nvme and spinning disk. Benchmark results: | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 5,377,375.00 | 185.96 | 0.2% | 60,125,513.00 | 11,633,676.00 | 5.168 | 3,588,800.00 | 0.4% | 0.09 | `ReadBlockFromDiskTest` | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 89,945.58 | 11,117.83 | 0.7% | 12,743.90 | 64,530.33 | 0.197 | 2,595.20 | 0.2% | 0.01 | `ReadRawBlockFromDiskTest` ACKs for top commit: maflcko: lgtm ACK 1c4b9cb achow101: ACK 1c4b9cb TheCharlatan: ACK 1c4b9cb Tree-SHA512: 71dbcd6c7e2be97eb3001e35d0a95ef8e0c9b10dc9193025c7f8e11a09017fa2fbf89489b686353cd88fb409fb729fe2c4a25c567d2988f64c9c164ab09fba9f (cherry picked from commit 00bf4a1) Dash adaptation: the benchmark reads Dash's bundled block813851 instead of block413567, deserializes it without TX_WITH_WITNESS, and is registered in src/bench/CMakeLists.txt instead of Makefile.bench.include. Partial: ReadRawBlockFromDiskTest is left out because Dash's BlockManager has no ReadRawBlockFromDisk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6acec6b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky) 0cc74fc multiprocess: Add type conversion code for serializable types (Ryan Ofsky) 4aaee23 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky) Pull request description: Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly. The non-test code in this PR was previously part of bitcoin#10102 and has been split off for easier review, but the test code is new. --- This PR is part of the [process separation project](bitcoin#28722). ACKs for top commit: achow101: ACK 6acec6b dergoegge: reACK 6acec6b Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6 (cherry picked from commit 2f218c6) Dash adaptations: - The Makefile.am / Makefile.test.include hunks become the IPC test block in src/test/CMakeLists.txt, restored as upstream's CMake tree has it (bitcoin#30454). The build_msvc hunk is dropped; Dash has no MSVC build. - Dash's SpanReader still takes a serialization type and version (bitcoin#28451 is not backported), so CustomReadField passes SER_NETWORK, CLIENT_VERSION. - Dash has no Txid type (bitcoin#28107), so the test builds the COutPoint from a plain uint256. - Dash's libmultiprocess pin already carries the update from bitcoin#31105, whose ipc_test.cpp hunk (m_rpc_system is now a std::optional) Dash's partial bitcoin#31105 left out because the file did not exist yet; it is applied here so the test builds against that pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…void resizing a19235c Preallocate result in `TryParseHex` to avoid resizing (Lőrinc) b7489ec Add benchmark for TryParseHex (Lőrinc) Pull request description: This pull request introduces optimizations to the `TryParseHex` function, focusing primarily on the ideal case (valid hexadecimal input without spaces). A new benchmark, `HexParse` was introduced in a separate commit. The main optimization preallocates the result vector based on the input string's length. This aims to completely avoid costly dynamic reallocations when no spaces are present. ------------ Before: ``` | ns/base16 | base16/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.60 | 623,238,893.11 | 0.3% | 0.01 | `HexParse` | 1.65 | 606,747,566.34 | 0.6% | 0.01 | `HexParse` | 1.60 | 626,149,544.07 | 0.3% | 0.01 | `HexParse` ``` After: ``` | ns/base16 | base16/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 0.68 | 1,465,555,976.27 | 0.8% | 0.01 | `HexParse` | 0.68 | 1,472,962,920.18 | 0.3% | 0.01 | `HexParse` | 0.68 | 1,476,159,423.00 | 0.3% | 0.01 | `HexParse` ``` ACKs for top commit: achow101: ACK a19235c hebasto: ACK a19235c. andrewtoth: Re-ACK a19235c Empact: Re-ACK bitcoin@a19235c Tree-SHA512: e09a59791104be3fd1026862ce98de9efafa1f949626fa01e3b7d58e6a2ef02a11f0de55ddba5c43230a53effd24e6d368c1e12848b17e8ce91d7908a59333f0 (cherry picked from commit 6dda050) Dash adaptation: the benchmark is registered in src/bench/CMakeLists.txt instead of Makefile.bench.include. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5bc2077 validation: allow to specify frequency for -checkblockindex (Martin Zumsande) d5a631b validation: improve performance of CheckBlockIndex (Martin Zumsande) 32c8041 bench: add benchmark for checkblockindex (Martin Zumsande) Pull request description: `CheckBlockIndex() ` are consistency checks that are currently enabled by default on regtest. The function is rather slow, which is annoying if you * attempt to run it on other networks, especially if not fully synced * want to generate a long chain on regtest and see block generation slow down because you forgot to disable `-checkblockindex` or don't know it existed. One reason why it's slow is that in order to be able to traverse the block tree depth-first from genesis, it inserts pointers to all block indices into a `std::multimap` - for which inserts and lookups become slow once there are hundred thousands of entries. However, typically the block index is mostly chain-like with just a few forks so a multimap isn't really needed for the most part. This PR suggests to store the block indices of the chain ending in the best header in a vector instead, and store only the rest of the indices in a multimap. This does not change the actual consistency checks that are being performed for each index, just the way the block index tree is stored and traversed. This adds a bit of complication to make sure each block is visited (note that there are asserts that check it), making sure that the two containers are traversed correctly, but it speeds up the function considerably: On master, a single invocation of `CheckBlockIndex` takes ~1.4s on mainnet for me (4.9s on testnet which has >2.4 million blocks). With this branch, the runtime goes down to ~0.27s (0.85s on testnet).This is a speedup by a factor ~5. ACKs for top commit: achow101: ACK 5bc2077 furszy: ACK 5bc2077 ryanofsky: Code review ACK 5bc2077. Just added suggested assert and simplification since last review Tree-SHA512: 6b9c3e3e5069d6152b45a09040f962380d114851ff0f9ff1771cf8cad7bb4fa0ba25cd787ceaa3dfa5241fb249748e2ee6987af0ccb24b786a5301b2836f8487 (cherry picked from commit 891e4bf) Dash adaptations: - Dash's CheckBlockIndex has no snapshot (assumeutxo) bookkeeping, so the snap_update_firsts() call next to the reworded comment is not added. - The -checkblockindex help text takes upstream's new wording, which also repairs Dash's garbled "for the block tree, and occasionally". - The benchmark is registered in src/bench/CMakeLists.txt instead of Makefile.bench.include. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncy from validation.cpp fa7d319 refactor: Remove unused circular include dependency from validation.cpp (MarcoFalke) Pull request description: Also, sort includes ACKs for top commit: ismaelsadeeq: ACK fa7d319 hebasto: ACK fa7d319 TheCharlatan: ACK fa7d319 Tree-SHA512: ea5e0001644d70ecfbccf87e27b393786a0eda79af4923ff68a0096d4d5b910cf6eeed8667ecbf55f3a164f500d3f5aeaf9d81bb190296c30ce0cc93c165717d (cherry picked from commit b2240f6) Dash adaptation: Dash's validation.cpp never included kernel/chain.h, so only kernel/coinstats.h moves into the sorted include list. Prerequisite for bitcoin#30344, which moves mempool_persist to node/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f1478c0 mempool: move LoadMempool/DumpMempool to node (Cory Fields) 6d242ff kernel: remove mempool_persist.cpp (Cory Fields) Pull request description: DumpMempool/LoadMempool are not necessary for the kernel. Noticed while working on instantiated logging. I suppose these could have been left in on purpose, but I'm assuming it was probably just an oversight. ACKs for top commit: TheCharlatan: Re-ACK f1478c0 glozow: ACK f1478c0 stickies-v: ACK f1478c0 Tree-SHA512: 5825da0cf2e67470524eb6ebe397eb90755a368469a25f184df99ab935b3eb6d89eb802b41a6c3661e869bba3bbfa8ba9d95281bc75ebbf790ec5d9d1f79c66f (cherry picked from commit d2c8d16) Dash adaptation: the Makefile.am hunk becomes the matching move in src/CMakeLists.txt (bitcoin_node) and the removal of mempool_persist.cpp from the chainstate kernel library in src/kernel/CMakeLists.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…20Poly1305 8607773 Add fuzz test for FSChaCha20Poly1305 (stratospher) c807f33 Add fuzz test for AEADChacha20Poly1305 (stratospher) Pull request description: This PR adds fuzz tests for `AEADChaCha20Poly1305` and `FSChaCha20Poly1305` introduced in bitcoin#28008. Run using: ``` $ FUZZ=crypto_aeadchacha20poly1305 src/test/fuzz/fuzz $ FUZZ=crypto_fschacha20poly1305 src/test/fuzz/fuzz ``` ACKs for top commit: dergoegge: tACK 8607773 marcofleon: Tested ACK 8607773. Ran both targets for ~200 CPU hours. Coverage of intended targets looks good to me. The simulation of damaged keys and checks that follow seem useful as well. Tree-SHA512: b6b85661d896e653caeed330f941fde665fc2bbd97ecd340808a3f365c469fe9134aa77316569a771dc36d1158cac1a5f76700bcfc45fff12aef07562e48feb9 (cherry picked from commit 1db0be8) Dash adaptations: - Dash has no InsecureRandomContext (bitcoin#29625 and its follow-ups are not backported), so both targets seed a FastRandomContext from a fuzzer-chosen uint256. - The target is registered in src/test/fuzz/CMakeLists.txt instead of Makefile.test.include. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
86b3852 qa: a fuzz target for the block index database (Antoine Poinsot) Pull request description: This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb. ACKs for top commit: achow101: ACK 86b3852 TheCharlatan: Re-ACK 86b3852 maflcko: review ACK 86b3852 🥒 brunoerg: utACK 86b3852 Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a (cherry picked from commit ee57737) Dash adaptations: - Dash still calls the block tree database CBlockTreeDB; it becomes kernel::BlockTreeDB with bitcoin#28191, which Dash has not backported. - Dash's CBlockTreeDB::LoadBlockIndexGuts() takes no interrupt argument (it checks ShutdownRequested(); the SignalInterrupt argument comes with bitcoin#27861), so the target calls it without m_interrupt. - The target is registered in src/test/fuzz/CMakeLists.txt instead of Makefile.test.include. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 30 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
✅ Final review complete — no blockers (commit 0e291d3) · triage: normal |
thepastaclaw
left a comment
There was a problem hiding this comment.
Final validation — Phase 1 + Phase 2
The backport stack faithfully restores the intended benchmark, IPC, and fuzz registrations and consistently adapts the upstream changes to Dash APIs. The only raised finding concerns the intentionally omitted raw-block benchmark; the PR explicitly documents that Dash lacks the required API, so no actionable in-scope defect remains. Successful lint, cppcheck, and fuzz-build checks are available; remaining build checks are pending.
Review provenance
Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: backport-reviewer); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 6: gpt-6.1-sol (agent: phase2-reviewer, role: backport-reviewer); reviewer 7: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
normalbygpt-6.1-sol(effort low) — This is a large, cross-cutting backport affecting validation, IPC, fuzzing, benchmarks, and build/test registration, but it does not directly change consensus rules, cryptographic behavior, funds handling, or storage migration semantics. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Second Phase-2 pass: on this first full review every Phase-2 reviewer also ran a second, independent time, and the verifier weighed both sets
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— backport-reviewer (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— dash-core-commit-history (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— backport-reviewer (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— dash-core-commit-history (completed, effort high); agentphase2-reviewer
Issue being fixed or feature implemented
The CMake import in #7481 (bitcoin#30454) had to leave out several test, bench and fuzz registrations because the upstream PRs that add those source files had not been backported. Most notably, the IPC test block in
src/test/CMakeLists.txtwas commented out, so-DWITH_MULTIPROCESS=ONbuilt no IPC test at all. This PR backports the small, self-contained upstream PRs behind those gaps, so the registrations now exist where upstream's CMake tree has them.What was done?
One commit per upstream PR (
cherry-pick -m1 -xof the upstream merge). Upstream'sMakefile.am/Makefile.*.includehunks become edits to the matchingCMakeLists.txt, in the place upstream's CMake tree has them.bench/readblock.cpp. It uses Dash's bundledblock813851and dropsTX_WITH_WITNESS.ReadRawBlockFromDiskTestis left out because Dash'sBlockManagerhas noReadRawBlockFromDisk.ipc/capnp/common-types.h) andipc_tests. The commented-out IPC test block insrc/test/CMakeLists.txtis restored as upstream build: Introduce CMake-based build system bitcoin/bitcoin#30454 has it. Adaptations:SpanReaderstill takes a type and version (refactor: Remove unused SER_DISK, SER_NETWORK, CDataStream bitcoin/bitcoin#28451 is not backported), so the hook passesSER_NETWORK, CLIENT_VERSION.Txid(util: Type-safe transaction identifiers bitcoin/bitcoin#28107), so the test builds theCOutPointfrom a plainuint256.m_rpc_system->bootstraphunk from Update libmultiprocess library bitcoin/bitcoin#31105, which Dash's partial Update libmultiprocess library bitcoin/bitcoin#31105 skipped because this file did not exist yet. Dash's libmultiprocess pin needs it.build_msvchunk is dropped.TryParseHexpreallocation andbench/parse_hex.cpp.CheckBlockIndex. The best header chain is walked through aCChaininstead of the multimap.-checkblockindexbecomes a frequency, andbench/checkblockindex.cppis added. Adaptations:snap_update_firsts()call is not added, because Dash'sCheckBlockIndexhas no assumeutxo bookkeeping.kernel/mempool_persist.hinclude fromvalidation.cpp, along with its expected circular dependency. It is the upstream prerequisite of kernel: remove mempool_persist bitcoin/bitcoin#30344; without it, the move would leave a newnode/mempool_persist -> validationcycle.mempool_persistfromkernel/tonode/. In CMake, the file moves tobitcoin_nodeand leaves the chainstate kernel library.crypto_aeadchacha20poly1305andcrypto_fschacha20poly1305. Dash has noInsecureRandomContext, so both are seeded with aFastRandomContextfrom a fuzzer-chosenuint256.block_index. It usesCBlockTreeDB(kernel::BlockTreeDBcomes with refactor: Remove unused MessageStartChars parameters from BlockManager methods bitcoin/bitcoin#28191).LoadBlockIndexGuts()is called without the interrupt argument, which arrives with kernel: Rm ShutdownRequested and AbortNode from validation code. bitcoin/bitcoin#27861.No new Dash-specific files are added, so
test/util/data/non-backported.txtis unchanged.How Has This Been Tested?
Two builds, each with the depends toolchain, Ninja,
-DWERROR=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON:NO_QT=1(GCC 15):ctest: 184/184 passed.bench_dash -sanity-checkpassed forReadBlockFromDiskTest,HexParseandCheckBlockIndex.validation_load_mempoolpassed against batches of random inputs. Forblock_index, a breakpoint confirmed that inputs reachedLoadBlockIndexGutsand every assert.mempool_persist,feature_reindex,feature_reindex_readonly,feature_loadblock,rpc_invalidateblock,rpc_getchaintips,rpc_blockchain,feature_block,p2p_unrequested_blocks,p2p_invalid_block,p2p_sendheaders,feature_assumevalid,feature_llmq_chainlocks,feature_llmq_is_cl_conflicts,feature_dip3_deterministicmns,mempool_reorg,mining_basic,feature_shutdown. On regtest these all run the reworkedCheckBlockIndexafter every block-tree change.NO_QT=1 MULTIPROCESS=1with clang-19, plus-DWITH_MULTIPROCESS=ON: the build passes,ipc_testspasses, andctestpassed 185/185.test/lint/lint-circular-dependencies.pyandlint-includes.pypass.Breaking Changes
-checkblockindex(a debug-only option) now takes a frequency, as upstream:-checkblockindex=<n>checks on average everynoperations,0disables it, and a bare-checkblockindexstill means1. Defaults are unchanged.Checklist:
🤖 Generated with Claude Code