Skip to content

Update Pydantic AI to CLAI 2 and decode canonical sessions - #69136

Merged
pelikhan merged 3 commits into
mainfrom
pelikhan-pydantic-ai-release-compatibility
Oct 16, 2026
Merged

pelikhan merged 3 commits into
mainfrom
pelikhan-pydantic-ai-release-compatibility

Conversation

@pelikhan

@pelikhan pelikhan commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Why

The Pydantic AI smoke and conformance workflows used harness 0.35.0 and the legacy pai runner. Upstream v2.56.0, released October 13, 2026, provides CLAI 2 and canonical session capture. The local Pydantic adapter needs to understand these events instead of treating the new output as legacy terminal text.

Not merge-ready: review confirmed an unresolved capture-integrity defect in the upstream definition. Its framing key and records enter an agent-readable, writable log. Matching-key framing therefore does not authenticate tool results, usage, terminal status or inference evidence. The prerequisite is now tracked in pydantic/pydantic-ai#10397; no fixed upstream revision was available in the inspected history/current main.

Approach

  • Pin both workflow imports to upstream commit 4f6799896fcb386cc5d0af7e84b4b17c17dd14c3, selecting matching harness and CLAI 2 packages at 0.56.0, and include the compiler-fetched definition and regenerated locks. The repository release tag itself still pins 0.55.0, so this uses upstream's subsequent version-bump commit.
  • Decode upstream keyed frames before terminal-control stripping. Preserve reported prompts, reasoning, tool arguments/results, native-tool events, identity and usage; redact declared masked values and ignore malformed, wrong-key and unsupported records. These records are not authenticated evidence while the upstream integrity defect remains.
  • Preserve legacy stdout capture as explicitly partial, without inventing tool outcomes or metrics. No shared session/schema or other engine source files are changed.
  • At the user's direction, merge origin/main at 2687cab90c in commit 5d0e56305c747cd6b671b668a4078c6395a89ac5. Regenerate both conflicting locks with firewall 0.28.59 and MCP gateway 0.4.33. Correct the unified-session usage assertion to canonical camelCase fields.
  • In review fix a604435b7b41fed14bc1fc7f4db68539da44e8cb, revalidate event envelopes after redaction. Discard records whose envelope keys/types were masked while preserving valid neighbors. Add three regression cases for embedded mask commands targeting data, type and an event type.

Validation and review status

  • Current-source build, JavaScript type-checking and focused formatting checks passed.
  • After the review fix, 74 focused tests passed across the Pydantic, custom-parser and engine-parser suites. The three new regressions reproduced the defect before the fix. The change-scoped final make agent-report-progress BASE_REF=HEAD gate passed, including 39 Pydantic tests, JavaScript lint and schema freshness; no Go changes required new Go tests. Two parser lint warnings remain visible and were not suppressed.
  • At the preceding latest-main merge, 155 impacted setup JavaScript tests and impacted Go tests in actions/setup, pkg/constants and pkg/workflow passed. Custom Go linters and schema freshness passed. Restoring pinned dependencies used lifecycle scripts disabled.
  • A shared golangci-lint lock initially blocked the merge gate; an isolated-TMPDIR non-test gate subsequently passed Go lint, custom Go lint, JavaScript lint, schema freshness and workflow drift. All 344 workflows compiled and every lock matched its source. No workflow/compiler source changed in the subsequent parser-only review fix.
  • Whitespace checks passed, the latest commit is pushed, and the working tree is clean.
  • The repeated capture-integrity review batch made no repository changes. Both comments were answered with the upstream issue and resolved as upstream-tracked at the user's request, not as repaired. No new commit or push was manufactured.

The earlier compile-only dry-run gate remains failed: shellcheck passed across 93 run-step resources and model inventory was available, but experimental-engine notices, the ignored native tool allow-list warning and the existing threat-detection fallback to Copilot were fatal to that gate. No warnings were suppressed. Normal locks were restored after diagnostic compilation. Passing normal compilation/drift checks do not replace that gate.

The initial security review's required same-reviewer continuation remains unavailable because the runtime does not support follow-up messages to the synchronous reviewer. Its incomplete coverage is not a security-review PASS. Subsequent review confirmed the framing-key exposure described above; fixing the local redaction crash does not repair capture integrity or establish live native inference/provider/MCP correctness.

Unresolved capture-integrity prerequisite

Comments 4289321180 and 4289419765 are accepted. Their discussions are resolved as upstream-tracked, but the defect remains unresolved in pydantic/pydantic-ai#10397. The header and every frame expose the same key in /tmp/gh-aw/agent-stdio.log, which both generated workflows expose read-write to agent tools. Rechecked upstream main 11807a4e1c44a21d03b483065cb42597fa032bac still has the same definition blob 511d7739e450de70cd4553ad750b620497b2e87b as the pin.

A trusted recorder transport must be inaccessible to agent tools, with matching parser/runtime support and adversarial checks against stdout/shared-log forgery. Then update both immutable import pins and regenerate locks. Removing only the header breaks recording and leaves keys in frames; patching only the local adapter does not secure Actions' imported custom parser. Upstream-managed source and generated locks were not manually edited to hide this defect.

Live testing

No Actions smoke was dispatched. Live testing and merge readiness remain blocked by the capture-integrity defect, failed dry-run gate, incomplete technical review and missing current-session human validation of the exact revision and effects. The normal smoke lock does not encode the proposed five-credit bound. Conformance requires a real queue Claim and must not run standalone.

The PR was published at the user's explicit request for code review; local validation does not authorize live execution.

Pin the smoke and conformance workflows to the upstream 0.56.0 engine definition and retain observed canonical events in the Pydantic session adapter, with legacy partial stdout support.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review October 15, 2026 23:29
Copilot AI max review requested due to automatic review settings October 15, 2026 23:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The runtime replacement has unresolved capture-integrity issues and incomplete integration validation, requiring human review.

3 open findings
What changed in this PR

Migrates gh-aw’s Pydantic AI workflows to CLAI 2 and adds canonical session decoding for local audits.

Changes:

  • Pin harness and CLAI 2 packages to 0.56.0 and regenerate workflows.
  • Decode keyed session events with redaction while retaining partial legacy capture.
  • Add regression coverage and patch release notes.
File Description
actions/​setup/​js/​parse_pydantic_log.test.cjs Adds canonical-frame and unified-session regression cases.
actions/​setup/​js/​parse_pydantic_log.cjs Decodes keyed canonical events before legacy parsing.
.github/​workflows/​smoke-pydantic.md Updates the upstream engine pin.
.github/​workflows/​smoke-pydantic.lock.yml Regenerates smoke execution for CLAI 2.
.github/​workflows/​engine-conformance-pydantic-ai.md Updates the upstream engine pin.
.github/​workflows/​engine-conformance-pydantic-ai.lock.yml Regenerates conformance execution for CLAI 2.
.github/​aw/​imports/​pydantic/​pydantic-ai/​4f6799896fcb386cc5d0af7e84b4b17c17dd14c3/​src_pydantic_ai_harness_gh-aw_pydantic.md Adds the pinned runtime and canonical recorder definition.
.changeset/​pydantic-ai-clai2-release.md Documents the migration and decoding support.

🧠 Review effort: Max

Comment thread actions/setup/js/parse_pydantic_log.cjs Outdated
Comment thread actions/setup/js/parse_pydantic_log.test.cjs Outdated
pelikhan and others added 2 commits October 15, 2026 16:55
Regenerate Pydantic workflow locks with the latest firewall and gateway pins. Correct the unified-session regression assertion to use canonical camelCase usage fields.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Discard events whose envelope keys or event types were redacted, preserving valid neighboring records and avoiding partial-capture crashes. Add regressions for embedded mask commands targeting envelope keys and types.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan merged commit 249a3a7 into main Oct 16, 2026
52 checks passed
@pelikhan
pelikhan deleted the pelikhan-pydantic-ai-release-compatibility branch October 16, 2026 00:11
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.92.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants