feat(review): run every added or changed E2E spec as PR proof - #5175
Merged
Merged
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 4 Skipped Deployments
|
Contributor
|
Selected evidence: Passed · 1/1 tests · 2/2 assertions · 0 images Commit |
benjaminshafii
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Replace the "must add a new spec" gate with a simpler rule: every E2E spec a PR adds or changes is run and published as that PR's proof. Nothing blocks. A PR touching no spec publishes nothing and says so.
Why
The gate correctly refused #5170/#5171/#5172, but #5172 rewrites three existing E2E specs that are its real proof, and the runner never executed them. Counting only newly added files was a poor proxy.
Change
evals/specs/**/*.e2e.test.ts. Removed specs and non-E2E files are ignored.proof-contractjob and itsopenwork-tests-requiredwiring removed; the structural validator is deleted. No branch-protection impact.xvfb-runin its own bounded job.(den)); traversal, absolute paths, backslashes and control chars still refused.Proof on this PR
evals/specs/pr-proof-git-paths.e2e.test.ts(new) is selected and runs here. Local: 1 passed, 0 skipped.Verification
node --test .github/scripts/pr-proof.test.mjs evals/scripts/publish-review.test.mjs evals/packages/test-artifacts/test/*.test.ts— 26 passed.node --test scripts/ci/workflow-authoring-gate.test.mjs .github/scripts/required-verification.test.mjs— passed.actionlinton the three workflows — passed.No new environment variables or secrets.
After merge: update #5172, #5170, #5171, #5168 branches; their changed specs will run and publish.