An experiment in durable coding-agent execution, evidence and acceptance, with an ongoing review of whether a second orchestration layer earns its complexity.
Agent work is hard to audit when status, evidence and acceptance live only in a chat or terminal. Agent-Workflow explored durable boundaries for these concerns. The current question is which boundaries need a separate harness when the coding agent already has one.
Agent-Workflow currently runs, records and evaluates coding-agent work through a separate workflow engine. Each Agent Run holds instructions, permissions, progress, saved evidence and review state. The architecture is now being audited against simpler agent-harness execution: the goal is to keep the useful evidence and recovery features without defending every existing layer.
I designed the architecture, the rules for who may approve what, the evidence and review process, the limits on model decisions, the evaluations and the release criteria. Coding agents write the code and help review it, under my direction.
Each diagram links to its Mermaid source (a text format for diagrams) and has a static SVG version. They show how control and evidence move through the system. They illustrate the design and do not record real runs.
Two tasks each run on their own, save a checked patch, and are reviewed and accepted separately. Agent-Workflow then combines both in a separate worktree and writes a new receipt. Merging and pushing are separate decisions.
Drawn from merged Agent-Workflow pull requests #66, #73 and #75, plus the patch-collection fix in #77 (commit e93ccff3). This is a static SVG illustration. It is not Mermaid CLI output, a release sign-off or a merge receipt. The tested path combines exactly two single-commit tasks. A release-readiness check on a clean machine is still blocked.
These diagrams show the architecture, who is responsible for what, and where evidence comes from, based on the project record. They describe the implementation and its stated limits. They are not independent test results.
01 / Execution path
Workflow state is saved. Model workers act only within the instructions and permissions they are given.
02 / Execution path
Saved runs, messages and evidence let work resume after an interruption. Review still happens separately.
03 / Who controls what
Rules handle facts and permissions. Models give structured advice on narrow judgment calls and do the open-ended generation.
04 / Execution path
OpenSpec 1.13.2 plans are imported into a fixed job format (native-job/v2). Each task is accepted separately, and the combined result gets its own check. Agent-Workflow then writes a receipt before any separate merge or push. The older native-job/v1 format is still supported.
05 / Feedback loop
Benchmarks show where the workflow adds overhead or gets worse. Changes follow, and the same claims are tested again.
↻ The evidence feeds the next iteration.
06 / Execution path
The planned BM6 study keeps task runs separate from the evaluator, which sees the outputs only after they are locked. This describes the method. No BM6 results are published.
Both setups get the same task and run separately, so each setup's results stay apart and clearly labeled.
Task, scoring and evaluator definitions are versioned. They are not changed after results are seen.
Run outputs can be hashed and locked before any scorer sees them. If an output changes afterward, later scoring is invalid.
Published studies include the finished software and cleaned-up evidence, so readers can inspect what was produced.
-185.003 s run time · -235,308 tokens used · -2.5 automated-score points
One development run comparing two setups on the same task, a direct baseline and a slimmed-down workflow, using the published benchmark harness.
One pair of runs, and no pair has finished human review. It does not show a general effect or that either setup wins.
Each study runs two setups on the same task and publishes the finished software, scores, run time, token use and the study's limits. The results guide engineering work. They do not show that one setup wins in general.
A side-by-side development study run after the optimization work. Both setups built the same Priority Picker app with the published benchmark harness. In the slimmed-down workflow, steering messages (instructions sent while the worker runs) guide the worker, and structured commands are used only in exceptional cases. Structured Direct is the baseline: a coding agent (Codex) run directly, without Agent-Workflow.
Observed: In this one comparison, Agent-Workflow finished 185.003 seconds sooner and used 235,308 fewer tokens. Its automated score was 2.5 points lower.
One development run (n=1), and no pair has finished human review. It does not show a general effect or that either setup wins.
inspect study evidenceA side-by-side development study of the first major Agent-Workflow optimization pass. It pointed the next round of work at verification and review overhead. Structured Direct is the baseline: a coding agent (Codex) run directly, without Agent-Workflow.
Observed: In this one comparison, optimized Agent-Workflow scored 6.5 points higher but took 428.667 more seconds and used 2,630,657 more tokens. The result pointed the next optimization round at verification and review overhead.
One development run (n=1), and human visual review is incomplete. The published study does not claim that either setup wins in general.
inspect study evidenceThe benchmark repository publishes the finished applications and screenshots at each screen size for both setups, so readers can see how the two builds differ beyond their scores.
BM5
Both setups built the same Priority Picker app, and each finished app can be inspected on its own. Desktop screenshots are shown here. Tablet and mobile screenshots are linked for each setup.
Structured Direct
Agent-Workflow Slimmed
Development benchmark evidence: one pair of runs, none with finished human review, and human visual review may still be pending. The screenshots are for inspection. They do not show which one looks better.
BM4
BM4 also publishes desktop, tablet and mobile screenshots of both finished apps.
Structured Direct
Agent-Workflow Optimized
Development benchmark evidence. The screenshot setup has known limits, which the study records, and human visual review may still be pending. Read the screenshots alongside the study's scores and its notes on which runs qualified.
Visitor analytics: page interactions, source category, browser information, and network IP metadata are recorded in private server logs. No analytics cookies are used. Opt out of future engagement beacons in this browser.