Repository navigation
fix(ci): stop all-acceptance-tests passing when no tests ran - #13469
Open
oc-tmueller wants to merge 2 commits into
Open
oc-tmueller wants to merge 2 commits into
oc-tmueller wants to merge 2 commits into
Conversation
The required all-acceptance-tests gate did not list build-and-test in its needs and only failed on a failed or cancelled need. When build-and-test failed or was cancelled, every test job was skipped, and the gate saw only skipped results and reported success: the required check went green with no acceptance tests run. Add build-and-test to the gate's needs, so its failure or cancellation fails the gate. Docs-only changes still pass, because the gate exits early for them before looking at the results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: oc-tmueller <323649642+oc-tmueller@users.noreply.github.com>
Outside docs-only, no acceptance job has a condition of its own: each one runs whenever build-and-test succeeded. A skipped need therefore always means tests did not run. Fail on it instead of only on failed or cancelled needs, so a future condition or dependency that skips the test jobs cannot turn the required check green again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: oc-tmueller <323649642+oc-tmueller@users.noreply.github.com>
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes. |
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The required
all-acceptance-testscheck can go green when no acceptance test ran.The gate didn't list
build-and-testin itsneeds, and it only failed on afailureorcancelledneed.When
build-and-testfails or is cancelled, every test job is skipped. The gate then sees only skippedresults and exits 0.
This has already happened. Three PRs were merged on 2026-11-11 with a green required check, but the only
Acceptance Tests run on their head commit had
build-and-test= cancelled and every test job skipped:Those merges never had acceptance coverage, so it may be worth a manual run on master and stable-9.0.
Fix
build-and-testto the gate'sneeds, so its failure or cancellation fails the gate.if:; each runswhenever
build-and-testsucceeded. So in a code run, a skipped need always means tests didn't run. Thisstops a future condition or dependency from reopening the same hole.
Docs-only changes still pass: the gate exits early for them.
workflow_dispatchand[full-ci]force a fullrun, so nothing there is skipped.
Backports to stable-9.0/8.2/8.0 will follow once this is merged.
🤖 Generated with Claude Code