Portfolio case study: https://ngallodev-software.uk/projects/agent-workflow
Public benchmark results: https://github.com/ngallodev-software/agent-workflow-benchmark-results
- What it is: a durable orchestration layer for coding-agent work: delegation, isolated execution, evidence, review, recovery, and acceptance.
- Why it matters: model work does not have to disappear into chat history or a terminal session; execution state and proof survive restart and can be reviewed independently.
- Key boundary: deterministic application code owns lifecycle and policy. Models work inside explicit contracts, and optional semantic providers return bounded evidence rather than taking control.
- Proof: the public benchmark repository publishes finished outputs, timing/token measurements, scores, and limitations instead of only headline claims.
agent-workflow is a headless orchestration, evidence, evaluation, and delegation
engine for coding-agent work. It keeps workflow authority in deterministic code
while allowing coding agents and bounded semantic providers to operate inside
explicit, inspectable contracts.
The durable execution object is an Agent Run. A task may produce one or more Agent Runs over its lifetime; each Agent Run has an immutable execution contract, a worker plan, durable messaging, evidence, and a review/acceptance lifecycle.
Why it exists: multi-step agent work becomes difficult to inspect when task state, worker identity, instructions, verification, and acceptance live only in chat or terminal history. Agent-Workflow makes those boundaries durable and restart-safe without owning the interactive host UI.
Related projects: SpecGen-AW compiles reviewable specifications into Agent-Workflow prompt packs; agent-workflow-benchmark provides the optional comparative benchmark surface; and agent-workflow-benchmark-results publishes finished benchmark software and evidence.
Workflow
└── Task
└── Agent Run
└── Worker
A worker has one of two modes:
- headless — agent-workflow launches and owns the local worker process group;
- external — agent-workflow prepares the Agent Run and execution contract, while another runtime launches the worker.
The core intentionally does not own workspace, pane, window, or interactive terminal layout. External hosts may consume the implemented host-neutral, generation-checked binding/delivery contract in EXTERNAL_WORKER_BINDING.md, while interactive layout remains outside the core.
- Git worktree isolation, source baselines, and provenance;
- Agent Run contracts and worker policy;
- restart-safe workflow DAGs and hierarchical delegation authority;
- persist-first steering, acknowledgement, replay, and correlation;
- controlled process execution and bounded supervision;
- completion handoffs, sealed evidence, receipts, review, and acceptance;
- evaluation plans, scoring, review/acceptance evidence, and optional plugin-provided comparative benchmarks;
- rebuildable SQLite projections and read-only MCP access;
- prompt-pack validation and trusted semantic plugins.
For the normal path, create/select the worktree and launch the Agent Run in one deterministic composition:
agent-workflow delegate RUN-001 prompt.md --repo /path/to/repo \
--ticket TICKET-001 --base-ref HEAD --role implementation --tier medium--pack accepts either the stable pack_id or the path to the pack root containing pack.yaml. --job accepts either a native JSON job path or a task ID declared in pack.yaml (for example P0-00 in a v1 Markdown prompt pack). When --job supplies the ticket identity, delegate does not invent a conflicting ticket ID from the Agent Run ID.
Dirty worktrees remain fail-closed by default. If pre-existing changes are intentionally part of reconciliation work, pass --allow-dirty; Agent-Workflow records that dirty launch baseline rather than silently discarding or normalizing it.
When --allow-dirty is used, that approval is also injected into the immutable worker launch context: pre-existing overlapping changes are treated as baseline drift rather than an automatic blocker. Workers must still preserve unrelated drift and stay within the assigned scope. For read-only review/evidence work, the flag records the baseline only; it is not permission to rewrite the pre-existing changes.
--agent-name is a logical worker identity, not a model selector. [agents].preferred_names controls automatic allocation order only, so an explicit valid name may be outside that list. Normal launches use --role, which selects a configured private runtime binding. If an operator intentionally pins executor/model/reasoning for qualification or diagnosis, omit --role and use the matching --agent-class with the explicit runtime options.
Lineage retries are prepare-first:
agent-workflow agent-run restart RUN-001 --context-file corrective.md
agent-workflow agent-run start RUN-001-retry1--context-file is bound into the retry launch prompt without changing the original ticket prompt. Add --start only when immediate execution is explicitly desired.
Independent review prerequisites use sealed completion evidence rather than requiring acceptance first: a verified, sealed, successfully completed run can be reviewed before host acceptance, while implementation-to-implementation dependencies still require acceptance. Explicitly rejected prerequisites remain blocked.
For Codex linked worktrees, Agent-Workflow grants the worker both the linked worktree Git administrative directory and the repository's shared Git common directory so normal commits can write the shared object database and refs.
The lower-level worktree create, agent-run prepare, and agent-run start commands remain available for recovery, diagnostics, and explicit operator control. delegate uses those same authorities and produces the same durable Agent Run evidence; it does not introduce a parallel lifecycle.
Observe and communicate durably:
agent-workflow agent-run status RUN-001
agent-workflow agent-run status RUN-001 --json
agent-workflow agent-run progress RUN-001 "implemented parser changes" --actor worker
agent-workflow agent-run steer RUN-001 "also run the integration tests" --actor parent
agent-workflow agent-run ack RUN-001 MESSAGE_ID "applied" --actor workerThe compact delegate response includes a steering capability block. agent-run status exposes the same decision as steering_supported, steering_adapter, and steering_reason. Steering is persist-first: delivery_outcome=unsupported means the request is durably recorded but no evidence-capable adapter delivered it to the running worker. Do not treat delivery or persistence as application; correlated acknowledgement is still required.
Review and disposition remain separate from worker completion:
agent-workflow agent-run review RUN-001 --actor reviewer --reason "evidence inspected"
agent-workflow agent-run accept RUN-001 --actor maintainer --reason "accepted"Implementation and review workers do not author Agent-Workflow protocol JSON. The normal successful worker path is intentionally one closeout transaction:
agent-workflow agent finish RUN-001 --result completedFor native jobs, finish runs or safely reuses every declared acceptance
command, records durable verification receipts, derives criteria explicitly
mapped to those commands, and generates the completion handoff. A failed
acceptance command returns verification_failed with bounded repair evidence
without publishing a terminal completion; repair stays in the current worker
session and finish is retried. Unmapped semantic criteria are the exception:
finish returns only their IDs, after which the worker records those bounded
judgments and retries finish:
agent-workflow agent criterion RUN-001 semantic-id pass --evidence "bounded semantic evidence"
agent-workflow agent finish RUN-001 --result completedagent limitation remains available for a real controlled-environment
limitation. The former granular worker verification/completion and routine progress/context/watch
commands are removed. Their deterministic responsibilities are owned behind
agent finish; mid-run context uses agent-run steer.
Mid-run instructions and additional context use durable agent-run steer;
workers should not poll status/watch/context for information the parent can send
directly. Correlated agent-run ack remains an exceptional evidence operation
when a steering request must prove application.
Agent-Workflow derives identity, Git revisions, changed files, observed command
exit status, schema-valid completion JSON, and acceptance revision. Native-job
criteria may bind acceptance_command_ids; only those explicit mappings permit
host-derived criterion results. A limitation is always not_verified and
non-gating.
Normal runner completion and recovery finalization use the same terminal pipeline: collect completion/task evidence, collect post-run scope/commands, write provider evidence, evaluate policy, derive terminal status, update provenance, write final status, seal, and refresh the projection. An exit code 0 can no longer be interpreted differently by recovery than by the normal runner when completion evidence is missing or invalid.
New run handoffs are stored under the Agent-Workflow run-state directory rather than inside the source checkout, preventing runtime evidence from entering Docker/build contexts. Existing worktree-local handoffs remain readable for compatibility.
When --config is omitted, agent-run prepare and delegate auto-discover an
exact-root .agent-workflow-execution.toml. This repository-local file may
override execution identity (agents, agent_classes, executors, roles,
runtime_aliases) but cannot override host security, state paths, plugins, or
other administrative policy.
Canonical review/accept/reject operations and prerequisite requirements are
closed, deterministic operation/policy sets. Acceptance derives the revision
from sealed completion evidence; --revision is now only an optional
compatibility assertion.
An external runtime can consume a prepared Agent Run without being a dependency of the core:
agent-workflow delegate RUN-EXT prompt.md --workdir /path/to/worktree \
--worker-mode external --interactive --role implementationThe facade records the durable authority and launch plan but does not launch a process.
agent-workflow workflow validate workflow.json
agent-workflow workflow start ./workflow-run workflow.json
agent-workflow workflow status ./workflow-run workflow.json
agent-workflow workflow resume ./workflow-run workflow.json
agent-workflow workflow seal ./workflow-run workflow.jsonWorkflow eligibility and durable state belong to agent-workflow. Presentation of workers does not.
The repository is acceptance-first. Invariant tests protect durable contracts and security boundaries; installed end-to-end journeys exercise public CLI behavior. The core must operate on a machine with no interactive runtime host installed.
python -m pytest -q tests/invariants
python -m pytest -q tests/acceptance
python -m pytest -qSee docs/TESTING.md.
- Architecture
- 0.9 skill-first simplification plan
- Installation
- Operations and recovery
- Testing strategy
- Prompt packs
- Comparative benchmarks
- BM3 TypeSafe/Jev optimization audit
- Benchmark plugin migration
- MCP server
- Plugin API
- Backlog
- Legacy notes
- Contributing, versioning, and CI
Generate the exact installed command surface directly from the parser:
agent-workflow commands --format markdownThe core is deliberately host-independent. A future plugin may project Agent Runs into an interactive coding-agent environment, provide live delivery after durable persistence, and reconcile host bindings, but it must consume public Agent-Workflow contracts rather than become workflow authority. See the external host/plugin boundary and backlog.
Version 0.11.10 builds on 0.11.8 with post-BM5 telemetry hardening: worker-issued agent finish commands, entered finish transactions, terminal finish outcomes, and incomplete invocations are recorded as distinct evidence. The steering-first worker protocol and host-owned lifecycle, review, acceptance, scope, provenance, and sealing authority remain unchanged.
Jenkins CI and local server-job files remain in the source repository for maintainers. They are repository infrastructure, not installed runtime features; see Contributing.
Normal agent-workflow delegate output is intentionally compact: run ID, logical role, worker mode, worktree, state, idempotency/worktree indicators, steering capability, and next actions. Use agent-workflow agent-run status RUN or agent-workflow agent context RUN when detailed durable state is actually needed rather than paying that context cost on every delegation.
Agent-Workflow has built-in deterministic, typesafe, and comparative decision modes. The TypeSafe-backed modes require the optional typesafe install extra and remain bounded to registered semantic seams; deterministic lifecycle/recovery authority does not move to the provider. Unrelated trusted plugins may still advertise additional decision providers or modes through the public plugin API. Inspect the effective capabilities with:
agent-workflow decision modes
agent-workflow decision providers
agent-workflow decision checkSelect a mode/profile in configuration or for one invocation with --decision-mode / --decision-profile. Built-in TypeSafe modes are unavailable when the optional SDK extra is absent; plugin-contributed modes are unavailable unless their plugin is installed and enabled. --no-plugins suppresses optional plugins but does not remove built-in deterministic recovery authority.
Modes that advertise comparative capture also require the neutral shared library:
pip install 'agent-workflow[comparative-eval]'Agent-Workflow persists the already-computed control/candidate pair in the workflow coordinator's comparative-eval.sqlite without issuing a second semantic-provider call. Inspect persisted cohorts with agent-workflow decision report PATH.
See DECISION_MODES.md and TYPESAFE_ARCHITECTURE_ALIGNMENT.md for the policy boundary and receipt semantics.
CLI commands contributed by enabled plugins are attached to the live argparse tree dynamically. As a result, top-level help, the parser-derived command catalog, and shell completion reflect the plugins enabled in the selected configuration:
agent-workflow --help
agent-workflow commands --format markdown
agent-workflow completion bashUse agent-workflow --no-plugins --help for the core-only recovery surface. The parser-derived command catalog is the command-reference source of truth. Static man pages provide stable orientation for core/workflow/index/benchmark concepts but do not replace live plugin-aware --help or commands --format markdown.
The historical comparative benchmark subsystem is no longer part of Agent-Workflow core. Install and enable the separate agent-workflow-benchmark plugin to restore the top-level agent-workflow benchmark ... command. Core still owns generic sealed-run evaluation, review, acceptance, and lifecycle authority.
Benchmark 0.3 adds the BM3 development study:
structured-direct/v1
vs
agent-workflow-full/v1
Both arms receive the same structured workflow prompt discipline; only the candidate executes through Agent-Workflow lifecycle/evidence machinery. The benchmark repository provides a one-command runner:
bash scripts/run-bm3-structured.sh --helpBM3 records per-phase executor-active/host-overhead timing, copies sealed Agent-Workflow terminal-section timing, performs machine scoring, and runs a separate pre-treatment TypeSafe/Jev routing qualification with private redacted request/response audit. The paired treatments themselves must add zero TypeSafe routing calls. See Comparative benchmarks and the BM3 TypeSafe/Jev optimization audit.
Agent-Workflow includes an optional built-in TypeSafe provider for the three registered routing judgments. Install agent-workflow[typesafe], provide TYPESAFE_API_KEY through the runtime environment, and select decision_policy.mode = "typesafe" or "comparative". The provider uses the official SDK Choice, Noul, and Score primitives; deterministic control, fallback, lifecycle, review, and acceptance remain Agent-Workflow-owned. See DECISION_MODES.md and TYPESAFE_ARCHITECTURE_ALIGNMENT.md.
The integration is intentionally narrow. src/agent_workflow/semantic/typesafe.py projects bounded task state, removes secret-like fields, versions the projector and question set, then lowers application-owned question specifications to the Python SDK. src/agent_workflow/routing.py computes the deterministic route first, records semantic receipts, applies the configured decision policy, and recomputes the route through the same deterministic policy function. The provider cannot start or complete runs, select an unsafe executor/model, waive review, or accept work.
flowchart LR
A[Task text and metadata] --> B[Bounded versioned state]
B --> C[TypeSafe system_one]
C --> D[Choice: task class]
C --> E[Noul: interaction needed]
C --> F[Score: semantic risk]
D --> G[Agent-Workflow decision policy]
E --> G
F --> G
H[Deterministic route and configured limits] --> G
G --> I[Deterministic route recomputation]
I --> J[Agent-Workflow enforcement and lifecycle]
routing.task_class chooses among registered work classes; routing.interaction_required assesses whether a material user decision is missing; routing.semantic_risk scores the consequence of a mistaken interpretation. Each result is accompanied by provenance, status, confidence/uncertainty, and fallback information. Policy dispositions can keep evidence in shadow, apply it under configured confidence rules, or use it comparatively. Service failures and uncertain answers follow the deterministic fallback path. Comparative capture stores the control/candidate pair and does not make another model request.
The benchmark runner uses the same provider in a separate pre-treatment qualification step: one call for each analyze-plan, implement, and verify-repair context. It excludes routing calls from both paired treatments so that the lifecycle comparison does not also change model-call count or semantic-routing behavior. BM3, BM4, and BM5 each published three successful qualification calls; BM3 recorded one task-class disagreement and uncertainty fallbacks for all three risk scores, while BM4 and BM5 each matched deterministic control in all three phase recommendations. These are routing diagnostics, not evidence that TypeSafe improved the benchmark task score. See the benchmark results.
Learn more from Jev and System One, the System One documentation, building with System One, state and context, semantic primitives, and the Python SDK guide. Agent-Workflow calls the TypeSafe SDK's system_one interface; Jev/System One supplies the semantic model and typed primitive layer, while Agent-Workflow retains operational authority.