Skip to content

ci: avoid downloading full diagnostics for test summaries - #21018

Draft
Ankit Jain (radical) wants to merge 1 commit into
mainfrom
ankit-jain/compact-ci-test-results
Draft

Ankit Jain (radical) wants to merge 1 commit into
mainfrom
ankit-jain/compact-ci-test-results

Conversation

@radical

Copy link
Copy Markdown
Member

Description

Final test reporting downloads about 2.1 GB of full diagnostics just to
read TRX files, keeping CI open after every test has finished.
A measured
run spent 47 seconds downloading artifacts in a 94-second final job.

Upload compact TRX artifacts alongside the existing diagnostic archives
and consume those during final aggregation. Preserve repository-relative
result paths, the combined summary, and failure gates. Full diagnostic
archives remain available; transient upload failures keep the existing
retry classification.

Contributes to #21015.

Validation

The timing candidate
reduced artifact downloads from 47s to 5s and the final job from 94s to 52s
against the preceding iteration. The additional upload took a median of
1s per test job, p95 2s, maximum 3s.

An artifact-only replay
compared both representations from that same source run on one runner:

Check Result
Full diagnostic downloads 52s; 2,186 MiB compressed
Compact TRX downloads 9s; 26 MiB compressed
TRX inventory and bytes All 396 files identical across Linux, Windows, and macOS
Combined summaries Identical
Executed job identities vs preceding iteration All 319 retained, including all 44 extension E2E shards

The timing run remained red: Windows Hosting, YARP, and CLI jobs failed
during test execution, before the new upload. Its final gate correctly
propagated those failures. The successful artifact replay is not a claim
that the full test suite passed.

Those runs exercise the same artifact change on an experimental base.
This PR is independently based on main; exact-head PR CI is pending.

From this branch's repository root, with the existing local SDK:

PATH=/workspaces/aspire/.dotnet:$PATH MSBUILDTERMINALLOGGER=false \
dotnet test --project tests/Infrastructure.Tests/Infrastructure.Tests.csproj --no-launch-profile -- \
  --filter-class '*.AutoRerunTransientCiFailuresTests' \
  --filter-class '*.GenerateTestSummaryToolTests' \
  --filter-class '*.ExtensionWorkflowTests' \
  --filter-class '*.TestTriggerMapTests' \
  --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'

Passed: 296 tests, no failures or skips. New behavioral cases cover
transient compact-upload retries and preservation of the mixed test-failure
veto.

actionlint .github/workflows/run-tests.yml .github/workflows/tests.yml
git --no-pager diff HEAD --check

Passed with pinned actionlint 1.7.12 and no whitespace errors.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Final aggregation downloaded roughly 2.1 GB of diagnostic artifacts just
to read TRX files. This added 47-60 seconds after the last test finished.

Upload TRX-only artifacts alongside the existing diagnostic archives and
consume those in final aggregation. Keep the same search roots and result
paths, retain full diagnostics, and preserve the summary and failure gates.
Classify transient compact-upload failures like diagnostic-upload failures.

The timing run reduced the final job from 94 to 52 seconds; the added
upload took a median of one second per test job. An artifact-only replay
verified identical paths and bytes for all 396 TRX files and identical
combined summaries. Its full and compact downloads took 52 and 9 seconds.

Document the artifact boundary and validation requirements in
docs/ci/ci-pipeline-optimizations.md.

Refs #21015

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21018

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21018"

@github-actions github-actions Bot added the area-engineering-systems infrastructure helix infra engineering repo stuff label Oct 15, 2026
@aspire-repo-bot
aspire-repo-bot Bot requested a balanced review from Copilot October 15, 2026 04:09
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

Selects the full PR test matrix + all PR-gated jobs (ALL) — a rule matching '.github/workflows/run-tests.yml' selects ALL


Selection computed for commit 3034210.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Exact-head CI validation remains pending in the supplied evidence, while artifact parity was validated on an experimental base.

0 open findings

What changed in this PR

Reduces Aspire CI reporting overhead by transferring compact TRX artifacts instead of full diagnostics for final aggregation.

Changes:

  • Adds TRX-only uploads and switches final aggregation downloads to them.
  • Preserves diagnostic archives and extends transient-failure classification to compact uploads.
  • Adds retry regression tests and documents the transfer strategy.
File Description
tests/​Infrastructure.Tests/​WorkflowScripts/​AutoRerunTransientCiFailuresTests.cs Tests upload retries and concurrent test-failure handling.
docs/​ci/​ci-pipeline-optimizations.md Documents compact artifacts and validation criteria.
docs/​ci/​auto-rerun-transient-ci-failures.md Documents compact-upload retry behavior.
.github/​workflows/​tests.yml Downloads compact TRX artifacts for aggregation.
.github/​workflows/​run-tests.yml Uploads TRX files alongside full diagnostics.
.github/​workflows/​auto-rerun-transient-ci-failures.js Includes compact uploads in existing failure classifications.

🧠 Review effort: Balanced


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-engineering-systems infrastructure helix infra engineering repo stuff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants