check_test_reachability counts a step disabled with if: false as running its test #652

scripts/check_test_reachability.sh counts a workflow step as running a test even when the step is switched off. Its parser looks only at run (scripts/check_test_reachability.sh:41-44):

run = step.get("run")
if isinstance(run, str):
    match = command.fullmatch(run.strip())

It never reads the step's if, the job's if, or continue-on-error on either. At 90cc24e, on a copy of the tree, I added if: ${{ false }} to the step that runs systemd-directory-modes.test.sh in ci.yml:

pristine   rc=0  test-reachability: every release and E2E test is invoked by a gate.
if: false  rc=0  test-reachability: every release and E2E test is invoked by a gate.

tests/release/npm-audit-off-pr-path.test.sh has the same blind spot: it decides whether the scheduled npm audit step is enforced from step.get("run") and continue-on-error (:86, :97), and does not read if.

Why it matters

The usual way to quiet a flaky test in CI is if: false or continue-on-error: true. Either one leaves the test unrun or non-blocking while the gate whose job is to notice still prints "every release and E2E test is invoked by a gate".

Scope

• Count a step only when neither the step nor its job has an if that is a literal false (false, ${{ false }}), and neither has continue-on-error: true. For any other if expression, either count it and print which expression it depends on, or refuse; say which you chose in the PR.
• Apply the same rule in npm-audit-off-pr-path.test.sh.
• Trap: YAML parses a bare false as a boolean and ${{ false }} as a string. Handle both.

Tests first

• tests/release/test-reachability.test.sh already builds workflow fixtures. Add three: step if: ${{ false }}, step continue-on-error: true, job if: false. Each must exit non-zero and name the test file. All three pass today.
• The same three fixtures for the npm-audit guard.
• Remove the new check and the fixtures must go green again, which proves they bite.

Difficulty

easy: two small Python blocks and fixtures in tests that already have the harness.

Getting started

CONTRIBUTING.md has the build and test commands. No CLA and no copyright waiver. The project is MIT; sign off each commit with git commit -s (the required dco check enforces it since #732).