feat(review): warn and close abandoned CI-failing pull requests

[neubig]

Desired Behavior

Extend the existing GitHub PR review automation so that its scheduled reconciliation also handles every open, non-draft pull request in its configured repositories whose GitHub-required checks for the base branch have remained failing. A reviewer trigger label or review request is not required for this deterministic maintenance pass. Optional checks do not count.

The lifecycle must be deterministic and idempotent:

1. After required CI has been continuously failing for at least seven days, post one marked comment asking the author to repair CI.
2. If the PR still has failing required CI seven days after that warning and the author has neither pushed a commit nor commented after the warning, close the PR with a marked explanation.
3. Do not close when CI recovered, CI is pending, the PR became draft, the PR was already closed/merged, or the author followed up after the warning.
4. A head update or author comment after the warning counts as follow-up. If CI remains failing, a later warning must start a new seven-day response window rather than immediately closing.

Reuse the reviewer automation's existing GitHub request, pagination, repository configuration, and durable state/comment-marker mechanisms. Keep the stale-CI decision deterministic; it should not consume an LLM review slot.

Acceptance Criteria

• The scheduled maintenance pass covers every open, non-draft PR in configured repositories, regardless of review trigger state.
• Required CI is derived from GitHub's required checks for the PR's base branch; optional failed checks do not trigger this lifecycle.
• A non-draft PR with required CI continuously failing for at least seven days receives exactly one clearly marked repair-CI warning.
• Repeated scans do not duplicate the warning.
• A warned PR is closed only after seven additional days with required CI still failing and no author commit or comment after the warning.
• Passing or pending required CI, a new author commit, an author comment, draft conversion, merge, or closure prevents the stale close.
• After author follow-up, any later warning starts a fresh seven-day response window.
• The logic uses existing reviewer extension machinery and works the same in local and Docker-backed Canvas deployments.
• Focused tests cover warning, idempotent rescan, close, CI recovery, pending CI, and author follow-up without duplicating existing GitHub helper tests.
• A live Canvas validation demonstrates the warning path and a time-controlled close/no-close reconciliation.

[all-hands-bot]

---

The following comments were added by the OpenHands AI agent.

Decision needed

Two material behaviors cannot be inferred safely from the repository, and each one changes both the implementation and its tests:

1. What counts as "required CI" for the deterministic scanner? The reviewer automation (skills/github-pr-reviewer/scripts/main.py and the catalog worker.py) currently has no CI evaluation of its own, and this repository has no single authoritative "required CI" mechanism to reuse. skills/github/scripts/github_client.py exposes only raw commit statuses, while skills/github-delivery-watchdog/scripts/worker.py evaluates a configured workflow list plus two hard-coded factory statuses that are specific to that automation. Please choose one:

   • (a) the required status checks from the base branch's GitHub branch-protection rules;
   • (b) an explicit list of required check/workflow names configured on this automation; or
   • (c) every check/status on the head that is not success.

2. Which pull requests does the reconciliation cover? The reviewer automation is currently trigger-scoped (a TRIGGER_LABEL application or a reviewer request). Automatically closing every open non-draft PR whose required CI has failed would extend its authority to PRs it was never asked to review. Should the stale-CI reconciliation run for all open non-draft PRs in the configured repositories, or only for PRs that carry the trigger label / were requested for review?

[neubig]

Decisions for the triage questions: use GitHub-required checks for the PR base branch, and run the deterministic maintenance pass across every open non-draft PR in the reviewer automation’s configured repositories. The issue body and acceptance criteria now state both choices explicitly.

[all-hands-bot]

👀 I see this issue was assigned to me and will start looking into it shortly.

This was posted by an AI agent (OpenHands) in response to being assigned this issue.

[all-hands-bot]

🤖 OpenHands is working on this issue.

View the conversation: https://oss-agent-canvas-backup.ngrok.dev/conversations/5ec2bd41-f403-48a5-bde6-62cf3dfc4e67

This was posted by an AI agent (OpenHands) in response to being assigned this issue.

[all-hands-bot]

⚠️ OpenHands gave up on this issue after 3h (last state: active). Unassign and reassign to try again.

This was posted by an AI agent (OpenHands) in response to being assigned this issue.