ci: open drift issues on job-level sensor failures - #170
Open
fullstackjam wants to merge 1 commit into
Open
fullstackjam wants to merge 1 commit into
fullstackjam wants to merge 1 commit into
Conversation
Every Harness job sets continue-on-error: true, so the Harness run concludes success even when a sensor fails. drift-to-issue gated its step on workflow_run.conclusion == 'failure', so it never ran and no drift issue was ever opened. Drop that gate; the script already filters on per-job conclusions from the jobs API, where a failed sensor reports failure. Exclude PR-triggered Harness runs with an explicit event check, since the branches filter also matches a fork PR whose head branch is main. Create the harness-drift label before opening an issue: it does not exist in the repo, and gh issue create --label fails on a missing label.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Makes
drift-to-issue.ymlopen or update tracking issues when individual Harness drift jobs fail. Before this, it only acted when the whole Harness run failed, which never happens.Why?
The sensor-to-issue loop has never fired:
continue-on-error: true, so the Harness run concludessuccesseven when a sensor fails.drift-to-issue.ymlgated its only step ongithub.event.workflow_run.conclusion == 'failure', so the step was always skipped. It did nothing, and its runs still showedsuccess.harness-driftissues, even thoughgovulncheck (drift)has failed on every recent nightly run.Evidence from the Actions API (scheduled runs on
main):conclusiongovulncheck (drift)jobconclusionsuccessfailuresuccessfailuresuccessfailuresuccessfailureI got these values with
gh api repos/openbootdotdev/openboot/actions/runs/<id>/jobs. Undercontinue-on-error, the failed job reportsconclusion: failurein the jobs API. Only the run-level conclusion is masked. The script already fetched that jobs JSON and filtered onjob["conclusion"] != "failure", so the only thing needed was to stop skipping it.Changes
conclusion == 'failure'gate. The Python loop's per-job filter is now the real gate. Jobs with any other conclusion (success,cancelled,skipped) are ignored, so a run with no failed jobs is a no-op.if: github.event.workflow_run.event != 'pull_request'. Thebranches: [main, master]filter matches the triggering run'shead_branch, and a fork PR from<fork>:mainalso hashead_branch == main. So the branch filter alone doesn't keep PRs out.push,scheduleandworkflow_dispatchruns on main still go through. This is the permissive form: if Harness ever adds apull_request_targettrigger, those runs would get through too, so that change would need this gate updated.harness-driftlabel before opening an issue. This is a second root cause: the label doesn't exist in the repo, andgh issue create --label harness-driftfails on a missing label. Incli/cli,LabelsToIDsreturns'harness-drift' not found, whichissue createwraps ascould not add label: …. Dropping the gate alone would have turned a silent no-op into a failing step, and still no issue. The fix isgh label create harness-drift --force, an idempotent create-or-update that only runs when a new issue is being opened.issues: writealready covers labels.One issue per sensor is unchanged. It's still the
[harness-drift] <job name>title, looked up among open issues with the exact-title check, and a repeat failure adds a comment instead of opening a new issue.Testing
{"event":"schedule","head_branch":"main","conclusion":"success"}for run 35860790347{"event":"pull_request","head_branch":"claude/fix-review-bot-allowlist","conclusion":"success"}for PR run 30735002314run:block out of the workflow YAML and ran it withbash -eo pipefail. It used the real jobs JSON from run 35860790347 and a stubghonPATHthat logs its arguments:gh label create harness-drift --force …thengh issue create --title "[harness-drift] govulncheck (drift)" …. That's exactly one issue, and nothing for the four passing sensors.gh issue comment 99 --body "Sensor fired again. Run: …".success: only thegh api …/jobscall, no issue calls.go run github.com/rhysd/actionlint/cmd/actionlint@latest(v1.7.12): exit 0 fordrift-to-issue.ymland for all workflows. actionlint doesn't type-checkgithub.event.*, so theworkflow_run.eventfield is backed by the API output above, not by actionlint.git merge-treeagainstci/bump-actions(ci: bump GitHub Actions off the deprecated Node 20 runtime #167, checkout v4→v7 in this file): merges cleanly. This diff leaves theactions/checkoutline alone, so the two PRs can land in either order.go vet, Go tests, andopenboot install --dry-rundon't apply; this is a workflow-only change.Not verified live:
if:is backed by the API field values, not by an end-to-end PR-triggered run.workflow_runworkflows always run from the default branch's copy of the file, so none of this takes effect until merge.Cross-repo checklist
openboot.dev? No.Notes for reviewer
mainshould open[harness-drift] govulncheck (drift)and create theharness-driftlabel. If chore: bump Go toolchain to 1.26.8 #166 (Go 1.26.8) lands first and clears govulncheck, no issue opens. To see it sooner, run Harness viaworkflow_dispatchon main.docs/MERGE_POLICY.md:36lists four drift sensors and leaves outrequired-checks alignment.