Skip to content

telemetry: run outcomes are all-or-nothing — carry ac_passed/ac_total so partial delivery is visible #2390

Description

@Q00

Problem

workflow_outcome for run is binary: one rejected AC (or a final revalidation failure) makes the whole run failed, so the fleet cannot distinguish a run that delivered 3 of 4 criteria from one that delivered none. On the maintainer's local event store (60d, 153 failed sessions): 33 runs had at least one AC accepted, 16 had every AC accepted and still failed (final workspace_mutated revalidation, fixed in 0.54.0), 97 were 0/N wipeouts clustered on three incident days. None of that is visible in PostHog.

Fix direction

Derive ac_passed / ac_total (root-level, from durable execution.ac.attempt_judged events; counts only) and carry them on every terminal run outcome — MCP job path, post-restart recovery, and CLI ooo run — with the telemetry boundary forwarding only a consistent non-negative int pair.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions