Repository navigation
nvx-rs: pin the wire formats with versioned conformance vectors - #811
Merged
Pedro Henrique Penna (ppenna) merged 7 commits intoOct 23, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The changes strengthen compatibility testing without altering production behavior, and no blocking issues were identified.
No open issues 🎉
What changed in this PR
This PR strengthens NVX wire-format compatibility checks by replacing scattered golden-byte literals with versioned conformance vectors, addressing #727 without changing production protocol behavior.
Changes:
- Adds documented NVXC and NVXL vectors, plus copies of OpenVMM’s nine records and 37 acceptance/rejection cases.
- Migrates existing tests to vector files and adds property tests for record round-trips and decoder robustness.
- Adds submodule-copy comparisons to both CLI CI jobs and the OpenVMM promotion checklist.
| File | Description |
|---|---|
scripts/test_nvx_tools.py |
Checks CI comparison commands and prerequisite ordering. |
nvx-rs/tests/vectors/README.md |
Documents vector formats, versioning, and provenance. |
nvx-rs/tests/vectors/openvmm/control_session_protocol_vectors.txt |
Copies OpenVMM’s nine record vectors. |
nvx-rs/tests/vectors/openvmm/control_session_protocol_cases.txt |
Copies OpenVMM’s 37 protocol cases. |
nvx-rs/tests/vectors/nvxl-v1.txt |
Defines listener-announcement vectors, including omitted listeners. |
nvx-rs/tests/vectors/nvxc-v1.txt |
Defines control-protocol vectors and invalid MAPS payloads. |
nvx-rs/tests/openvmm_vectors.rs |
Adds an ignored comparison against submodule files. |
nvx-rs/tests/guest_tools.rs |
Uses announcement vectors for guest-binary expectations. |
nvx-rs/src/tests.rs |
Adds shared vector readers and reader tests. |
nvx-rs/src/testing/fake_openvmm.rs |
Updates documentation terminology. |
nvx-rs/src/protocol/tests.rs |
Migrates codec assertions and adds decoder properties. |
nvx-rs/src/protocol/nvxs/tests/record.rs |
Checks OpenVMM cases and record-parser properties. |
nvx-rs/src/protocol/nvxs/tests/host.rs |
Uses the host-attachment vector. |
nvx-rs/src/protocol/nvxs/tests/guest.rs |
Uses OpenVMM’s guest-attachment vector. |
nvx-rs/src/protocol/nvxs/tests.rs |
Adds shared OpenVMM vector lookup helpers. |
nvx-rs/src/protocol/mod.rs |
Documents independent conformance checks. |
nvx-rs/src/lib.rs |
Registers test-only shared helpers. |
nvx-rs/src/guest/ingress/encode.rs |
Tests announcement encoding against vectors. |
nvx-rs/src/guest/ingress/collect.rs |
Replaces the inline announcement fixture. |
nvx-rs/README.md |
Documents vectors, properties, and comparison tests. |
nvx-rs/Cargo.toml |
Adds the property-testing development dependency. |
nvx-rs/Cargo.lock |
Locks property-testing dependencies. |
doc/project-structure.md |
Lists wire-format conformance vectors. |
doc/design/4.4-host-guest-control-interface.md |
Links control vectors and defines versioning rules. |
doc/design/3.5-network.md |
Links listener-announcement vectors. |
doc/contribute.md |
Documents vector maintenance and promotion checks. |
doc/ci.md |
Describes vector and property-test coverage. |
aci_edge_sandboxes/README.md |
Updates protocol-test terminology. |
.github/workflows/ci.yml |
Runs vector comparisons on Linux and Windows. |
.github/skills/nvx-openvmm-promote/SKILL.md |
Adds comparison and coordinated-update guidance. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/nvx-rs-conformance-vectors
branch
from
October 23, 2026 17:15
50fdcf5 to
71835af
Compare
The golden tests of nvx::protocol pinned NVXS records, NVXC frames, and EXEC, MAPS, and features payloads as hexadecimal literals, which Python's struct module once produced in the formats of the former Python client. The host, the guest agent, and the fake OpenVMM share one codec, so a change to it passes every round trip among them, yet breaks OpenVMM's broker and released guests, some of which run the C agent. Only bytes that no codec produced catch such a change, so they now live in a versioned file that the protocol owns rather than in the tests (#727). tests/vectors/nvxc-v1.txt holds them, one vector per line: its kind, its name, its bytes, and the values that they encode, in a line format that tests/vectors/README.md documents with the rules that the vectors follow: a vector changes only with a protocol version, which gets files of its own, and neither its bytes nor its values come from the code that it checks. Besides every byte that the golden tests pinned, the file holds the MAPS payloads that every decoder must refuse, which the tests built by changing bytes of a valid payload. A reader of the line format in src/tests.rs serves the unit tests of every module, and refuses malformed and repeated vectors and values that a test leaves unread. The tests check that the outer codec and the NVXS record codec both decode each record to its values and encode them to its bytes, and that the frame, EXEC, MAPS, and features codecs do the same, the host's and the guest's decoders alike. Without an explicit environment, the guest's inherit_default_env follows the layering flag alone, while the host's is true (#581). The tests that presented the vectors as the Python client's output are renamed for what they check, and the host's attachment takes its bytes from the file. The tests of the wire constants and of the C agent's feature sets stay. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The tests of the NVXL announcement, which nvx-ingress sends to the gateway of the portable network profile, pinned its bytes as literals of their own: the unit tests of its encoder and collector, among them the vector that #358 required, and the tests of the guest binary. OpenVMM's decoder, in net_backend_resources::ingress, has round-trip and rejection tests, but no vector file that both sides check. tests/vectors/nvxl-v1.txt holds the announcements in the line format of nvxc-v1.txt: no listener, a UDP listener, a TCP listener of both families, the two TCP listeners of #358, and the 64 lowest of more TCP listeners, with the flag that marks the omitted ones, which no test covered. The encoder's tests encode the listeners of each vector, added in reverse and, for the omitted ones, with one more above them, to its bytes, and the collector's scan and the nvx-ingress binary's test mode produce the bytes of the vectors that their snapshots announce. The tests that called the vector golden are renamed for what they check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The NVXS tests copied the 9 records of OpenVMM's control_session_protocol_vectors.txt into a table of their own, but not the 37 cases of control_session_protocol_cases.txt beside it: the bounds of DATA lengths, ACK windows, and CREDIT increments, malformed headers, the authentication code, and records that the broker refuses for their payload lengths, bootstrap identities, or error codes. tests/vectors/openvmm/ now holds both files unchanged, from the OpenVMM revision that NVX pins, 79a501b63fcd, which tests/vectors/README.md records. The record tests read the vectors as OpenVMM's tests do, and check that the NVXS record codec and the outer codec both decode each to its record and encode it to its bytes. They check each case as OpenVMM's tests do: the bounds against both the encoder and the parser, a malformed header against the parser before any payload byte arrives, and a refused record against the parser with the error that the case names. A case of an unknown kind fails, and the tests count all 37. The guest's attachment takes its bytes from the copy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The protocol tests read copies of OpenVMM's control-session protocol vectors: the nvx-rs job cannot check out the OpenVMM submodule, whose deploy key fork pull requests lack, and OpenVMM's workspace needs a later Rust release than nvx's minimum, so the tests cannot call its broker either. Nothing noticed an OpenVMM revision that changed the files that the copies come from. The ignored test tests/openvmm_vectors.rs compares the copies with the files of the submodule's checkout, byte for byte, and fails with how to update them. Both NVX CLI jobs, which check out the submodule at its pinned revision, run it once they validate the CLI, and a test of the CI configuration pins the step in each job, after the checkout and the installation of Rust. The nvx-openvmm-promote skill runs it before a promotion commits, and copies changed files together with the gitlink, as doc/contribute.md now asks of a pin promotion. doc/ci.md describes the step, and tests/vectors/README.md how to run it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The conformance vectors pin chosen bytes, while the NVXS record parser and the EXEC and MAPS decoders read bytes that a peer chooses: the broker's records, and the host's requests, which the guest agent decodes. Each must return an error rather than panic on any of them. proptest, a new development dependency without its forking and timeout features, generates the inputs of properties that check that: - every valid record of each type encodes, with the NVXS record codec and the outer codec alike, to bytes that the parser returns intact, whichever two pieces they arrive in; - the parser returns, never panics, on arbitrary bytes, on bytes after a valid magic and version, and on valid records with bytes overwritten, fails every later call with its error once it fails, and encodes each record that it accepts to exactly the bytes that it consumed for it; - the host's and the guest's EXEC and MAPS decoders return, never panic, on arbitrary bytes and on vectors with bytes overwritten, truncated, or extended, which reach past the checks of the headers; - the guest's EXEC decoder accepts every payload that the host's accepts, with the same values. proptest 1.12 needs Rust 1.88, below nvx's minimum of 1.89, and the lockfile holds the releases of its dependencies that 1.89 builds. A test of the CLI classifies every crate of the lockfile as one that ships in nvx-rs, whose notices it requires, or one that does not. proptest and the crates that it brings are the tests' own, and so is cpufeatures 0.3, which its chacha20 takes beside the cpufeatures 0.2 that sha2 ships. The test therefore finds each shipped crate's notices by the one version of it that ships, rather than by the last version of its name in the lockfile, and names the versions that only the tests use. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The design of the host-guest control interface gains 4.4.4, Conformance Vectors, which links nvxc-v1.txt and OpenVMM's copies and states the rule that a vector changes only with a protocol version, and the design of host-local access and ingress links nvxl-v1.txt where it describes an announcement. The nvx crate's README describes the vectors in its layout, in the row of nvx::protocol, and in its testing section, with the property tests and the ignored comparison with the submodule. doc/contribute.md places the vectors among the tests of the NVX mechanics, with their rule, and doc/ci.md, the project structure, and the README of aci_edge_sandboxes, which called them golden protocol fixtures, now call them conformance vectors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
The nvx-rs coverage workflow runs the crate's tests built with -C instrument-coverage, and has failed on Linux since #806 merged: managed_agent_helper_applies_inherited_exact_and_layered_environments fails in every selection of tests, as in dev's run 39990930082. The LLVM profiling runtime of an instrumented executable sets __LLVM_PROFILE_RT_INIT_ONCE in the environment of its own process when it starts, and the instrumented nvx-managed-agent hands that environment on to a workload that inherits the default environment, alone or beneath the request's entries. The test compared the environment that /usr/bin/env printed exactly, so it met an entry that no uninstrumented build has. The test now leaves that one variable out of the environments of the inherited and layered cases, the two that take the helper's own environment, and still compares the rest of them exactly. The cases that replace the environment are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff659e8f-6bed-4a2c-8c84-1835fc7d4834
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/nvx-rs-conformance-vectors
branch
from
October 23, 2026 18:00
71835af to
0bc5b9b
Compare
Pedro Henrique Penna (ppenna)
deleted the
agents/nvx-rs-conformance-vectors
branch
October 23, 2026 18:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #727.
The golden byte tests of the wire formats are the only check against implementations that share no code with
nvx: OpenVMM's broker and gateway, and released guests, some of which run the C agent. A change to the codec that the host, the guest agent, and the fake OpenVMM share passes every round trip among them, yet breaks those peers. The tests pinned bytes that Python'sstructmodule once produced, as literals spread over the tests. This change moves them into versioned conformance vectors that the protocols own, checks all 37 of OpenVMM's cases, and has CI compare the copies of OpenVMM's files with the submodule.Changes
nvx-rs/tests/vectors/, in one line format,KIND|NAME|BYTES|VALUE..., whichREADME.mddocuments with its rules: a vector changes only with a protocol version, and neither its bytes nor its values come from the code that it checks.nvxc-v1.txt:NVXSrecords,NVXCframes, andEXEC,MAPS, and features payloads, with theMAPSpayloads that every decoder must refuse.nvxl-v1.txt: listener announcements, including one that sets the omitted-listeners flag, which no test covered.openvmm/: OpenVMM'scontrol_session_protocol_vectors.txtandcontrol_session_protocol_cases.txt, unchanged from the pinned79a501b63fcd.guest_toolstests. Every vector decodes to its values and encodes to its bytes, with the host's and the guest's decoders alike, and each of OpenVMM's 37 cases is accepted or refused as it says. Tests named for the Python client are renamed; the wire-constant tests and the C agent's feature sets stay.tests/openvmm_vectors.rscompares the copies with the submodule's files. Both NVX CLI jobs run it after they check out OpenVMM, a configuration test pins the step, and thenvx-openvmm-promoteskill runs it before a promotion commits.proptest(a development dependency that builds on Rust 1.89): records round-trip, and theNVXSrecord parser and the host's and the guest'sEXECandMAPSdecoders return errors, never panic, on arbitrary bytes and on bytes near the vectors. The CLI test of third-party notices classifiesproptest's crates, and thecpufeatures0.3 that itschacha20takes beside the shipped 0.2, as the tests' own.nvx::protocolrow, the layout, and the Testing section of the crate's README,doc/contribute.md, anddoc/ci.md.The six commits are self-contained: each builds, lints, and passes its tests.
A seventh commit fixes a test that the
nvx-rs coverageworkflow has failed on since #806 merged (dev's run 39990930082), which a push of this branch ran: under-C instrument-coverage, the LLVM profiling runtime of the instrumentednvx-managed-agentsets__LLVM_PROFILE_RT_INIT_ONCEin its own environment, andmanaged_agent_helper_applies_inherited_exact_and_layered_environmentscompared the environment that its workloads inherit exactly. The test now leaves that one variable out of the inherited and layered cases.The issue's optional task, offering
nvxl-v1.txtto OpenVMM's decoder tests, needs an OpenVMM pull request and is not part of this change.Validation
-D warningsfor all targets and features and for the library without default features;cargo test --lockedwith all features and for the library without default features; rustdoc with-D warnings; andcargo +1.89 check --all-targets --all-features.guest_toolsingress tests, the NVXS integration tests, the Markdown link check, and, from the CI commit on, the ignored comparison against an OpenVMM checkout at the pinned revision. At the head: the full test suite with all features,nvx.py check-guest-binaries, the Rust 1.89 check, and theguest_toolstests built with-C instrument-coverage, which reproduce the coverage workflow's failure without the seventh commit. A tampered copy fails the comparison.validate-nvxaction, Ruff lint and format, and Pyright for Linux and Windows. On this machine,test_setup_azure.pyfails 18 subtests through its local Git Bash, as it does on unmodifieddev.ci.ymlschedules.In a heavily loaded local Linux run before the final rebase, two tests that this change does not touch flaked and then passed on retry:
guest::exec::binding::tests::a_write_that_fails_closes_the_program_s_inputhung with the writer of its socket still open, andguest::sys::signal::tests::a_thread_that_blocks_sigpipe_keeps_only_a_signal_that_was_pendingwrote 4 bytes where it expectedEPIPE.