Skip to content

fix(sinks): report cancelled in-flight requests as errored - #26836

Draft
thomasqueirozb wants to merge 1 commit into
reload-sink-drain-limitfrom
driver-cancelled-requests-errored
Draft

thomasqueirozb wants to merge 1 commit into
reload-sink-drain-limitfrom
driver-cancelled-requests-errored

Conversation

@thomasqueirozb

Copy link
Copy Markdown
Member

Summary

When a Driver-based sink is cancelled, for example when --reload-sink-drain-limit-secs expires, its in-flight requests are dropped. Their finalizers reported the events as Delivered, so sources with end-to-end acknowledgements acknowledged data that was not sent. These events are now reported as Errored. Requests that complete keep the status from their response.

Sink buffer Before After
Memory (default) Cancelled in-flight events report Delivered. The source acknowledges them and they are lost Cancelled in-flight events report Errored. The source can retry them
Disk The source is acknowledged when the record is written to disk. Cancelled in-flight records are acknowledged by the buffer and deleted No change. The disk buffer acknowledges records whatever their status
Trade-off Before After
Request reached the remote before it was cancelled Source acknowledges. No duplicate Source retries. The remote can receive the event twice
Fan-out to several sinks, one sink cancelled Source acknowledges the batch Source retries the batch. Sinks that already delivered it receive it again
Source without acknowledgements Events are lost No change. Events are lost
Events not yet submitted (batcher, request builder), sinks that do not use Driver, and sinks whose requests complete before the remote confirms them (for example mqtt) Report Delivered No change. Report Delivered

What a retry means depends on the source. Keeping cancelled records in disk buffers needs a separate status and a reader rewind on buffer reuse, which is out of scope here.

References

Depends on: #26834
Related: #26793

Vector configuration

NA

How did you test this PR?

Unit tests in vector-stream driver::tests: a cancelled request reports Errored, and a completed request keeps the status from its response.

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant