Agentic Workflow Patterns
Agentic workflows replace single-pass prompting with iterative loops: reflection, tool use, planning, and multi-agent collaboration. We review the measured gains behind each pattern — from Reflexion's HumanEval results to Devin's SWE-bench score — state what the techniques do not deliver, and argue for starting with the simplest loop that an evaluation can verify.
Why One-Shot Prompting Hits a Ceiling
Most production use of large language models in March 2024 is single-pass. A prompt goes in, tokens stream out, and the first draft is the final answer. The model never runs its code, never checks a claim against a source, never revises. That is the equivalent of asking a developer to write a feature top to bottom without ever pressing backspace — and then shipping it.
An agentic workflow replaces the single pass with a loop: the model drafts, examines the draft, gathers missing information, and revises before anything reaches the user. The published numbers for this shift are large. The engineering question is not whether the loop helps but which loop is worth its added latency, cost, and failure modes. This article works through the four patterns that recur in the literature.
Four Patterns for Agentic Workflows
Between October 2022 and today a consistent structure has emerged from the research: reflection (the model critiques its own output), tool use (the model calls external functions), planning (the model decomposes a goal into steps), and multi-agent collaboration (several role-prompted instances divide the work). Andrew Ng's letter of 6 March 2024 in The Batch describes the same landscape: rapid month-over-month progress, writer-critic pairs, and frameworks such as Microsoft's AutoGen, CrewAI, and LangGraph that make these loops programmable.
The four patterns are not equally mature. In our projects, reflection and tool use behave predictably enough for production. Planning and multi-agent collaboration deliver occasional strong results but not yet reliable ones. The sections below give one measured result per pattern — and one documented limitation.
Reflection: The Cheapest Gain
Reflection prompts the model to criticize its own output and then revise it. Self-Refine (Madaan et al., March 2023) reported roughly 20 percentage points of absolute improvement averaged over seven tasks, using the same model for drafting, feedback, and revision. Reflexion (Shinn et al., 2023) added memory of past failures and an external signal — unit tests — and reached 91% pass@1 on HumanEval, against 80% for its GPT-4 baseline. For scale: OpenAI's zero-shot evaluation of March 2023 put GPT-3.5 at 48.1% and GPT-4 at 67.0% on the same benchmark — baseline protocols differ, the direction does not.
The limitation is documented just as clearly. Huang et al. (October 2023) showed that intrinsic self-correction — reflection without any external feedback — can reduce reasoning accuracy rather than improve it. The pattern earns its cost when an objective check exists: a failing test, a schema validator, a retrieved source. Reflection against nothing is often just paraphrase.
Tool Use: Grounding the Model
Tool use gives the model functions to call: search, code execution, database queries. ReAct (Yao et al., October 2022) interleaved reasoning traces with actions and beat imitation- and reinforcement-learning baselines by 34 percentage points absolute on ALFWorld and 10 on WebShop, while reducing hallucination on knowledge tasks. OpenAI's function-calling API of June 2023 turned the idea into a mainstream interface, and every serious agent framework now builds on some variant of it.
Tool use does not make a model reliable at deciding when to call a tool, and every tool is also an attack surface: inputs the model constructs from untrusted text reach your systems. Tool schemas, timeouts, retries, and permission boundaries are ordinary engineering work that the pattern does not remove — it creates them.
Planning: Powerful but Brittle
Planning has the model decompose a goal into steps before executing them. HuggingGPT (Shen et al., March 2023) demonstrated the pattern by having an LLM plan task graphs across specialized models. On 12 March 2024 Cognition announced Devin, an agent with its own shell, editor, and browser, reporting 13.86% of real GitHub issues resolved unassisted on a random 25% subset of SWE-bench — against 1.96% for the previous best.
Both readings of that number are correct. It is a sevenfold improvement, and it leaves roughly 86% of issues unresolved. Long plans drift, errors compound across steps, and a forty-minute trajectory is far harder to debug than a single completion. Planning pays off when steps are checkable in isolation; it degrades when only the final state can be judged.
Multi-Agent Collaboration
Multi-agent systems assign roles to several model instances — often the same base model with different prompts — and let them converse. ChatDev and MetaGPT (both 2023) staffed a virtual software company this way; AutoGen (Wu et al., 2023) generalized the idea into a framework for programmable agent conversations. The simplest useful configuration remains a writer and a critic, which is reflection implemented as two agents.
The costs multiply visibly. Every agent added multiplies token spend and latency, conversation state must be managed, and agents can converge confidently on a shared wrong answer. Splitting one model into five prompts adds structure, not intelligence. We treat multi-agent designs as an optimization to reach for, not a starting point.
Measured Gains Versus Added Complexity
The evidence available in March 2024 shows the same trade in every row of the table below: accuracy is bought with tokens, latency, and new failure modes. None of the gains arrive without an evaluation harness that can detect them — which is why we build the harness before the loop.
Our recommendation is to start with the simplest loop: one model, one generate-critique-revise pass, one objective check such as a unit test or validator. Add tools when the task needs facts or actions the model does not have. Add planning or additional agents only when measurement shows the simple loop has plateaued. Each escalation should be justified by the same evaluation that justified the previous one.
| Pattern | Measured gain (example) | Added complexity |
|---|---|---|
| Reflection | HumanEval pass@1 80% → 91% with GPT-4 (Reflexion) | 2–3× token cost; can degrade without external feedback |
| Tool use | +34 points absolute on ALFWorld (ReAct) | Schemas, error handling, security surface |
| Planning | SWE-bench 1.96% → 13.86% (Devin) | Long trajectories; compounding errors; hard to debug |
| Multi-agent | Small apps end to end at under $1 average cost (ChatDev) | Cost per agent; orchestration state; shared wrong answers |
Outlook: Where Agents Go From Here
Writing on 18 March 2024, we expect workflow design to rival model upgrades as a source of capability gains this year. The 11 points Reflexion measured between GPT-4 alone and GPT-4 in a loop are the scale of a model-generation jump. We expect benchmarks like SWE-bench to become the standard yardstick for agents, and falling inference prices to make iteration loops economical where they are marginal today.
Open problems remain open: evaluating agents is harder than evaluating completions, cost control for long trajectories is unsolved, and tool-wielding agents raise security questions the industry has barely started to answer. We expect the frameworks to consolidate and the patterns to stay. Teams that learn to measure loops now will be able to adopt each new model the day it ships — by dropping it into a workflow that already works.
Sources
- ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 6 Oct 2022)
- GPT-4 Technical Report (OpenAI, 15 Mar 2023)
- Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 20 Mar 2023)
- Large Language Models Cannot Self-Correct Reasoning Yet (Huang et al., 3 Oct 2023)
- The Dawning Age of Agents — The Batch (Andrew Ng, 6 Mar 2024)
- Introducing Devin the first AI software engineer (Cognition, 12 Mar 2024)