Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/auto-rerun-transient-ci-failures.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ const retryableWithAnnotationStepPatterns = [
/^Set up \.NET Core$/i,
/^Install sdk for nuget based testing$/i,
/^Upload logs, and test results$/i,
/^Upload aggregation test results$/i,
];

const ignoredFailureStepPatterns = [
Expand Down Expand Up @@ -91,6 +92,7 @@ const ignoredFailureStepOverridePatterns = [

const postTestCleanupFailureStepPatterns = [
/^Upload logs, and test results$/i,
/^Upload aggregation test results$/i,
/^Copy CLI E2E recordings for upload$/i,
/^Upload CLI E2E recordings$/i,
/^Generate test results summary$/i,
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -742,6 +742,19 @@ jobs:
artifacts/bin/Aspire.Templates.Tests/Debug/net10.0/logs/**
artifacts/log/test-logs/**

# Final aggregation needs only TRX files, not the binlogs, dumps, and
# recordings retained in the original diagnostic artifact above.
# Keep the same search roots to preserve repository-relative result paths.
- name: Upload aggregation test results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: trx-${{ inputs.testShortName }}-${{ inputs.os }}
path: |
testresults/**/*.trx
artifacts/bin/Aspire.Templates.Tests/Debug/net10.0/logs/**/*.trx
artifacts/log/test-logs/**/*.trx

- name: Copy CLI E2E recordings for upload
if: always()
shell: bash
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -775,19 +775,19 @@ jobs:

- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: logs-*-ubuntu-latest
pattern: trx-*-ubuntu-latest
merge-multiple: true
path: ${{ github.workspace }}/testresults/ubuntu-latest

- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: logs-*-windows-latest
pattern: trx-*-windows-latest
merge-multiple: true
path: ${{ github.workspace }}/testresults/windows-latest

- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: logs-*-macos-latest
pattern: trx-*-macos-latest
merge-multiple: true
path: testresults/macos-latest

Expand Down
4 changes: 4 additions & 0 deletions docs/ci/auto-rerun-transient-ci-failures.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ The full analysis runs in four passes:

Passes 1–2 are hardcoded because they target well-known infrastructure signatures that rarely change. Passes 3–4 are configurable because transient test failure patterns evolve as integrations are added or CI environments change.

`Upload aggregation test results` uses the same infrastructure and Windows
post-test cleanup classifications as the full diagnostic upload. An upload
failure does not override a concurrent test-execution failure.

## When does it trigger?

| Trigger | Behavior |
Expand Down
20 changes: 20 additions & 0 deletions docs/ci/ci-pipeline-optimizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,26 @@ The main test workflow in `.github/workflows/tests.yml` is organized into these
5. `results`
- Aggregates the final workflow status after all required lanes finish

### Compact final-results transfer

`run-tests.yml` retains the full `logs-{testShortName}-{os}` diagnostic
artifact and additionally uploads `trx-{testShortName}-{os}`, containing
only TRX files from the same result and diagnostic directories. The final
`results` job downloads the compact
artifacts rather than downloading binlogs, dumps, and recordings again.

Downloads retain the existing `{os}/testresults/` hierarchy in
`All-TestResults`. The combined summary, original diagnostic artifacts,
test-result existence checks, and dependency failure gate are unchanged.
An early build or test-host failure can legitimately leave no TRX file;
the compact upload warns in that case, while the original failure still
blocks CI.

When validating this boundary, compare every TRX path and byte against the
full diagnostic artifacts, then compare combined summaries. Measure the
extra per-test upload time as well as the final job's download time and
total workflow wall time before treating the smaller transfer as a win.

## Why the jobs are split this way

### 1. Package build and CLI archive build run in parallel
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,30 @@ public async Task IgnoresConfiguredAggregatorJobsEntirely(string jobName)
Assert.Empty(result.SkippedJobs);
}

[Fact]
[RequiresTools(["node"])]
public async Task RetriesCompactResultUploadForArtifactNetworkFailure()
{
WorkflowJob job = CreateJob(failedSteps: ["Upload aggregation test results"]);

AnalyzeFailedJobsResult result = await AnalyzeSingleJobAsync(job, "Failed to CreateArtifact: Unable to make request: ENOTFOUND");

Assert.Single(result.RetryableJobs);
Assert.Empty(result.SkippedJobs);
}

[Fact]
[RequiresTools(["node"])]
public async Task DoesNotRetryTestFailureAlongsideCompactResultUploadFailure()
{
WorkflowJob job = CreateJob(failedSteps: ["Run tests (Windows)", "Upload aggregation test results"]);

AnalyzeFailedJobsResult result = await AnalyzeSingleJobAsync(job, "Failed to CreateArtifact: Unable to make request: ENOTFOUND");

Assert.Empty(result.RetryableJobs);
Assert.Single(result.SkippedJobs);
}

[Fact]
[RequiresTools(["node"])]
public async Task KeepsMixedFailureVetoWhenIgnoredTestStepsFailAlongsideRetrySafeSteps()
Expand Down Expand Up @@ -888,6 +912,7 @@ public async Task RepresentativeWorkflowFixturesStayAlignedWithCurrentWorkflowDe
"- name: Build test project",
"- name: Run tests (Windows)",
"- name: Upload logs, and test results",
"- name: Upload aggregation test results",
"- name: Copy CLI E2E recordings for upload",
"- name: Upload CLI E2E recordings",
"- name: Generate test results summary",
Expand Down
Loading