Skip to content

fix(topology): await removed disk sinks on shutdown - #26238

Closed
Jansen-w wants to merge 7 commits into
vectordotdev:masterfrom
Jansen-w:fix/await-removed-disk-sinks-on-shutdown
Closed

Jansen-w wants to merge 7 commits into
vectordotdev:masterfrom
Jansen-w:fix/await-removed-disk-sinks-on-shutdown

Conversation

@Jansen-w

@Jansen-w Jansen-w commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Retain task handles for removed durable disk-buffered sinks that continue draining after configuration reload, and include those tasks in the existing graceful-shutdown wait and deadline handling.

  • Reload remains non-blocking and never waits for active retired sinks.
  • Completed handles are synchronously reaped on later reloads without cancellation points; failures remain sticky so final shutdown reports false.
  • Active retired disk sinks remain tracked until final shutdown or its configured deadline.
  • Removed memory-buffered sinks keep their existing best-effort reload behavior; this change is intentionally limited to durable buffers.

References

Related: #26050

Vector configuration

Covered with topology regressions using an active sink and a removed sink with a blocking disk buffer.

How did you test this PR?

  • Deterministic reload/shutdown drain regression, repeated 10 times
  • Graceful-shutdown deadline expiry regression
  • Reload-does-not-wait regression
  • Completed-handle cleanup and failure-preservation regression, repeated 10 times
  • Serial topology reload suite: 13 passed
  • Transient-state and existing ordinary sink-removal tests
  • cargo clippy --lib --tests -- -D warnings
  • cargo fmt --all -- --check
  • cargo vdev check changelog-fragments

The parallel reload suite retains an unrelated existing enrichment-state timing flake; the test passes serially and in isolation.

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. A changelog fragment is included.
  • No. A maintainer will apply the no-changelog label to this PR.

Contributor Guidelines

@github-actions github-actions Bot added the domain:topology Anything related to Vector's topology code label Aug 26, 2026
@Jansen-w
Jansen-w marked this pull request as ready for review August 27, 2026 22:24
@Jansen-w
Jansen-w requested a review from a team as a code owner August 27, 2026 22:24
@Jansen-w
Jansen-w force-pushed the fix/await-removed-disk-sinks-on-shutdown branch from 31c4d7a to a7eb390 Compare August 27, 2026 22:28
Comment thread src/topology/running.rs Outdated
Comment thread src/topology/running.rs Outdated
Comment thread src/topology/running.rs Outdated

@bruceg bruceg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of optional nits below; I wouldn't block on them.

Comment thread src/topology/test/mod.rs
Comment thread src/topology/running.rs
Comment thread src/topology/test/mod.rs
@Jansen-w Jansen-w closed this Oct 14, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 14, 2026
@Jansen-w

Copy link
Copy Markdown
Contributor Author

Closing this. As discussed with Rob, this likely needs an RFC to discuss what the expected behavior would look like. Some users may not expect/want the sink running continuously in the background.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain:topology Anything related to Vector's topology code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants