- Author
- Lilian Weng
- Published
- July 4, 2026
- Kind
- Long survey post on a research area, with a benchmark appendix
- Original
- lilianweng.github.io/posts/2026-07-04-harness/
- Subject
- Harnesses and agent loops
- Themes
- Feedback loops
What it is
Lilian Weng reviews recent work on harness engineering as a route to recursive self-improvement. The post organizes the papers by what the harness optimizes.
What it says
- Weng defines a harness as the system around a base model that orchestrates its work.
- The post names three patterns: workflow automation, files as memory, and sub-agents with backend jobs.
- What gets optimized progresses from prompts to structured context, workflow, harness code and optimizer code.
- In the STOP work, recursive improvement helped GPT-4 but degraded results for GPT-3.5 and Mixtral.
- Lin et al. report flat harness-editing skill across model sizes. Benefit peaks in middle-tier models.
- Darwin Gödel Machine agents matched or beat handcrafted ones, citing 20% to 50% on SWE-bench.
- Weng lists seven open challenges. The last is moving humans up the stack, not out of the loop.
What evidence it gives
The post summarizes many papers and reports their results. Weng did not run the experiments. The post adds critiques, such as calling one study's evidence provisional. An appendix lists benchmarks with their reported scores.
Interests and limits
Results are secondhand and many cited papers are preprints. The post checks none of them independently. The post names no employer for the author. The forecasts about self-improvement are the author's own predictions.