Skip to content

Stop monitor_tasks polling after the timeout expires - #306

Open
arpitjain099 wants to merge 1 commit into
CDCgov:mainfrom
arpitjain099:fix/monitor-tasks-timeout-break
Open

arpitjain099 wants to merge 1 commit into
CDCgov:mainfrom
arpitjain099:fix/monitor-tasks-timeout-break

Conversation

@arpitjain099

Copy link
Copy Markdown
Contributor

Please describe the bug this fixes or the feature this adds.

Fixes #297. The timeout branch in monitor_tasks logs a warning and then drops straight into the next iteration of the while loop. Nothing on that path changes job.state or completed, so the next pass lands in the same branch and the call never returns. There is no sleep on that path either, so it spins. Everything after the loop was already written for this case, down to logging the timeout and returning the dict with completed=False, so all that was missing was the break.

The issue also flags the docstring promising a RuntimeError on timeout. Nothing in the function raises, so I replaced that line with what actually comes back. Happy to raise instead if you would rather keep the documented contract.

I left _cloudclient.py alone because #296 is already reworking monitor_job.

Please describe how you tested this change. Include unit tests whenever possible.

Added test_monitor_tasks_returns_when_timeout_expires. It feeds a fixed clock so the first poll falls inside the deadline and the second is past it, with one task still running, then checks the returned counts. Against the unpatched function the clock runs out and you can see the timeout warning logged twice. Full suite goes from 274 passed to 275. The single failure, test_download_job_stats, already fails on main.

Did you create or modify any associated documentation with this change? If documentation is not included in PR, please link to related documentation.

Only the monitor_tasks docstring.

If you added or modified HTML, did you check that it was 508 compliant?

No HTML.

Please tag any specific reviewers you would like to review this PR

Whoever owns batch_helpers.py.

Please include the following checks for open source contributing?

  • Did you check for sensitive data, and remove any?
  • Are additional approvals needed for this change?
  • Are there potential vulnerabilities or licensing issues with any new dependencies introduced? (none added)

The timeout branch logged a warning but never left the while loop, so the next iteration re-entered it immediately and the call never returned. The code after the loop already reports the timeout and returns completed=False, so break out of the loop and let it run.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: monitor_tasks loops indefinitely after timeout

1 participant