Repository navigation
ci: reuse the built OpenVMM VMM tests while their key is unchanged - #810
Merged
Pedro Henrique Penna (ppenna) merged 3 commits intoOct 23, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Both producers still retain temporary PR-only save conditions that prevent dev pushes from populating the caches and contradict the workflow-contract assertions.
1 open issue
What changed in this PR
This PR adds cross-run caching for built OpenVMM VMM tests to reduce CI build time while preserving same-run artifact delivery to test jobs.
Changes:
- Adds cache-key, validation, and reuse commands with manifest hashes and explicit build targets.
- Updates Linux and Windows producers to reuse validated content or rebuild on a miss.
- Adds CLI and workflow-contract tests and documents cache behavior.
| File | Description |
|---|---|
scripts/test_nvx_tools.py |
Tests cache keys, content validation, reuse, and workflow conditions. |
scripts/nvx.py |
Exposes the three cache-related CLI commands. |
scripts/nvx_tools/ci.py |
Implements manifest hashing, cache keys, validation, and reuse. |
doc/usage.md |
Documents the new commands and manifest fields. |
doc/ci.md |
Explains cache lifecycle, invalidation, and full-run behavior. |
.github/workflows/ci.yml |
Adds cache restore, validation, conditional builds, and saves. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-692-logical-commits
branch
from
October 23, 2026 17:22
8be11e4 to
4b3e521
Compare
Contributor
Author
|
A note on the red The CI run for the final head, 39994590839, passed: 38 jobs succeeded, including |
The OpenVMM VMM tests that build-openvmm-tests builds are to be cached across runs and reused while nothing that determines them changes (#692). Their manifest records the OpenVMM revision, the Rust release, and the backends' filters, but neither the target nor the build options that built them, and only the size of each file, so a restored copy could not be checked against what a fresh build makes. - build-openvmm-tests passes vmm-tests-run its target, linux-x64 or windows-x64, instead of letting flowey take the host's. NVX supports only x86-64 hosts, and the WHP tests run only windows-x64 content, so the build makes what it made before, but its target becomes an input that it names. The build fails if the run script that flowey writes runs the tests of another target. - OPENVMM_TEST_BUILD_OPTIONS holds the vmm-tests-run options that build the content: a release build without optional features, whose run script uses the nextest CI profile and skips the VHD prompt. - The manifest moves to format 2, which adds the target, the build options, and the SHA-256 of each file. test-openvmm --content refuses other formats, as before; a producer and its consumers run in the same workflow run, so they always agree. A helper now builds content with the tests' fake flowey, which the build test and a new test of the target check share. doc/usage.md and doc/ci.md describe the new fields. OpenVMM is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI is to cache each operating system's build of the OpenVMM VMM tests and reuse it while nothing that determines it changes (#692). Add the three commands that the producers run around that cache: - openvmm-test-cache-key prints the key of the content that build-openvmm-tests builds for some backends from this checkout: openvmm-vmm-tests-vVERSION-TARGET-REVISION-DIGEST. REVISION is the OpenVMM revision that the index pins, and DIGEST the SHA-256 of every manifest field that describes what made a clean build of it: the manifest's format, the Rust release, the target, the build options, and each backend's filter and their union. VERSION, OPENVMM_TEST_CACHE_VERSION, covers what none of those captures, such as the environment in which the tests build. The command reads only the gitlink, so it runs before OpenVMM is checked out, and gives its step the key output. - check-openvmm-tests fails unless a directory holds exactly that content: a manifest that records those fields for a clean checkout, and the runner and options that the content's run script runs, read again from the script, and exactly the files that the manifest lists, with their sizes, SHA-256, and executable bits. CI is to check each build this way before it caches it, so that it caches only what a later reuse accepts. - reuse-openvmm-tests keeps a directory that a cache restored only if it passes the same check, and otherwise removes whatever it holds, with a warning annotation that gives the reason, so that a build can take its place after a refused entry or a partial restore. It fails only when it cannot compute the key or remove the directory, never because of the content, and gives its step the reused output. build-openvmm-tests now writes the fields that the key digests with the function that the key uses, and test-openvmm --content reads the manifest and the gitlink through the helpers that the checks share; neither changes its behavior. doc/usage.md describes the commands. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The VMM tests run on most dev merges, but what determines their build rarely changes: replaying the change classifier over the 42 dev merges through #659, the tests would have run on 33, and the OpenVMM pin changed in none. Each run still built them: in run 39954081539, the build took 330 s of the Linux producer's 6.2 minutes and 858 s of the Windows producer's 16.2, which bounds the WHP lane (#692). Cache each producer's test content under the key that openvmm-test-cache-key computes, and restore it instead of building it: - A producer computes the key from the gitlink before it checks out OpenVMM and, on pushes and pull requests, restores the newest entry of the key into the directory that it uploads. reuse-openvmm-tests keeps the content only if it is exactly what a fresh build of the key makes, and removes what it refuses, with a warning, or what a partial restore left. The restore continues on error, so that a failed restore falls back to the build as a miss does. - On a hit, the producer skips the OpenVMM checkout, the retrying curl, the OpenVMM input and Cargo caches, sccache, and the build, and uploads the restored content as the same-run artifact that the test jobs download as before. The test jobs never read the cache. - After a build, check-openvmm-tests fails the producer unless the build passes the same check, so that neither the cache nor the test jobs get content that a reuse would refuse. - Only dev pushes save the content, as they save the other OpenVMM caches, and only after a build: under the key followed by the run ID and attempt, so that each build is an entry of its own and the restore takes the newest. An entry that a later reuse refuses then cannot block a newer build, as it would under an exact key, which a save never replaces. The OpenVMM input and Cargo caches also save only after a build, since a hit restores neither. - Scheduled and manually dispatched full runs neither restore nor save the content, so the nightly run still builds the tests every day and catches a breakage of the build that the key does not cover, such as a toolchain change on the runners. A miss, an eviction, a failed restore, or a refused entry therefore only costs the build. A test checks each producer's steps, conditions, keys, paths, and order; the full-run test counts the two new cache saves and the two restores that name the push event; and the guest artifact test registers the new commands as booting no guest. doc/ci.md describes the cache, its key, and its effect on the WHP runners, and doc/usage.md links to it. OpenVMM is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/fix-issue-692-logical-commits
branch
from
October 23, 2026 18:12
4b3e521 to
323a37d
Compare
Pedro Henrique Penna (ppenna)
deleted the
agents/fix-issue-692-logical-commits
branch
October 23, 2026 18:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #692.
Since #660, the OpenVMM VMM tests run on most
devmerges, but what determines their build rarely changes. Replaying the change classifier over the 42devmerges through #659, the tests would have run on 33 of them, and the OpenVMM pin changed in none. Each run still builds them. In run 39954081539, the build took 330 s of the Linux producer's 6.2 minutes and 858 s of the Windows producer's 16.2, which bounds the WHP lane.This PR caches each producer's test content across runs and restores it instead of building it while nothing that determines it changes. On a hit, the producers took 20 s (Linux) and 36 s (Windows) instead of 6.5 and 14.2 minutes, and the test jobs ran exactly the tests that a fresh build gives them. Consumers are unchanged and still take only the same-run artifact. OpenVMM and the runners are unchanged.
Key
nvx.py openvmm-test-cache-keyprintsopenvmm-vmm-tests-vVERSION-TARGET-REVISION-DIGEST:linux-x64orwindows-x64.build-openvmm-testsnow passes it tovmm-tests-run --targetexplicitly instead of taking the host's, which builds what it built before.vmm-tests-runbuild options (--release --ci-profile --skip-vhd-prompt --build-only, with no optional features), and each backend's filter and their union.OPENVMM_TEST_CACHE_VERSION) covers what none of those captures, such as the build environment.Hit, miss, and save
continue-on-error, so a failed restore behaves like a miss.nvx.py reuse-openvmm-testskeeps the restored content only if it is exactly what a fresh build of the key makes. Its manifest must record every keyed field, for a clean checkout, and the runner and options that its run script runs, re-read from the script. The directory must hold exactly the listed files, with their sizes, SHA-256 (new in manifest format 2), and executable bits. Anything else is removed with a warning, so the build starts from an empty directory.nvx.py check-openvmm-testsfails the job unless the build passes the same check, so neither the cache nor the test jobs get content that a reuse would refuse.devpushes save, as for the other OpenVMM caches, and only after a build. The key is<key>-<run_id>-<run_attempt>, so each build is its own entry and the restore takes the newest. An entry that a later reuse refuses therefore can't block a newer build the way an immutable exact key would.A miss, an eviction, a failed restore, or a refused entry only costs the build. It never skips tests or fails a run.
Commits
scripts: describe the OpenVMM test content by its target and digests: manifest format 2, with the target, the build options, and per-file SHA-256, plus the explicit--target.scripts: key, check, and reuse cached OpenVMM VMM test content: the three commands, tests, anddoc/usage.md.ci: reuse the built OpenVMM VMM tests while their key is unchanged: both producers, tests of their steps and conditions, and the new OpenVMM VMM test cache section ofdoc/ci.md.Validation
ruff check,ruff format --check, andpyright(Linux and Windows platforms) are clean, andcheck_markdown_links.pypasses.test_nvx_tools.py,test_nightly_ci.py, andtest_markdown_links.pypass (715 tests), as does the new upstreamtest_nvx_rs_coverage.py.test_no_c_or_assembler_sources_remain_outside_openvmmandtest_linux_runners_terminate_openvmm_left_by_earlier_jobs, fail the same way on unmodifieddevthere.concurrency.queuefinding.devsave.dev(sandbox: deliver managed live shares as a host mapping table #794 changeddoc/usage.mdandscripts/test_nvx_tools.py), CI run 40001414148 passed on323a37d62. 38 jobs succeeded, includingRequired status checkand the performance regression gate, and 4 were skipped as expected for a PR. Both producers missed, built, passed the post-build check, and skipped the save. Locally, the CLI job's suites (1,260 tests) and the adversarial tests (77) passed on the rebased head. An earlier head showed a redmeasure / ubuntu-latestfrom the push-onlynvx-rs coverageworkflow (nvx-rs: measure line, branch, and mutation coverage on Linux and Windows #806), which fails the same way ondev; see the PR comment. It doesn't run for this head.Measurements
From run 39988095577. Times in seconds unless marked.
devpush's save adds about 10 s (Linux) and 4 s (Windows).7ff31ed7, about 2.4 GiB. A hit also stops restoring thevmm-testsOpenVMM input entries (~945 MiB per OS) and their Cargo entries (161 and 261 MiB), so those age out between pin promotions.artifactsand the 36 s producer, and the KVM lane about 11 min after it, so the WHP microVM tests bound such runs instead.Post-merge verification
devpush after merge misses and saves both entries; later pushes and PRs that leave the key unchanged should hit.devruns, against CI: maximize workflow parallelism and reduce turnaround time #430's 12-15-minute cache-hit target, with the producers' durations and queue times.