A harness is the software around a model that decides what happens next. A loop runs the agent again until something says stop. This page lists 73 sources, oldest first where dated. They range from a very short opinion post to a long research survey.
Context engineering
Chris Loy traces how prompting grew from text completion into context engineering. The post proposes design patterns for context, borrowed from software design. Its evidence is a toy example about film box office figures, not tests.
Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering (Hacker News thread)
The thread has 93 points and many replies. The poster, who says the essay is the poster's own, defends it. Commenters mostly ask for results and dispute the technical framing. A few report good experience with the advice.
AI models need a virtual machine
A long thread that mostly argues about what the linked post means. Commenters split between calling it sandboxing, access control and capability security. The evidence is opinion, security analogies and a few self-promoted tools.
The Browser Sensorium
Grace Kind builds a test page. It looks empty when rendered but holds a poem in its HTML. The post uses it to contrast visual and text-based agents. It then sketches an agent that combines both views plus console and network data.
The quality of AI-assisted software depends on unit of work management
Atharva Raykar argues that context management and unit size matter more than raw model intelligence. The post uses a compounding-error calculation and a reading of the METR long-task study. It proposes user stories as the unit of work and announces an early experiment.
Embracing the parallel coding agent lifestyle
Willison explains how skepticism about parallel agents gave way to regular use. The post sorts suitable tasks into research, small maintenance and well-specified work. It also lists the author's tools and safety habits. The evidence is personal experience.
Vibe engineering
Willison coins vibe engineering for skilled engineers who stay accountable for LLM-built work. The post lists practices that make agents work better. It argues these favor experienced engineers. It rests on personal experience and carries a later update on terminology.
Technical Deep Dive on Streaming AI Agent Desktop Sandboxes: When Gaming Protocols Meet Multi-User Access
Helix explains how it adapted the Moonlight game protocol for several users. The post covers a workaround called apps mode and a newer lobbies mode. It is a narrative of design choices and open bugs, with little measurement.
Windsurf Codemaps: Understand Code, Before You Vibe It
A product announcement from Cognition, the company behind Windsurf and Devin. It pairs a feature description with an argument against AI that replaces understanding. The evidence is a feature walkthrough and outside estimates, not benchmarks.
Anti-patterns while working with LLMs
Manish gives five anti-patterns, each with a personal example. The examples include a failed image prompt, a forgotten database URL and an exposed hash. The evidence is anecdote, with no measurements.
A Software Engineer's Guide to Agentic Software Development
A personal blog post by a GitHub engineer on clearing tech debt with coding agents. It describes a three-step routine and the limits on daily workload. The evidence is the author's own experience, with no measurements.
Effective harnesses for long-running agents
Anthropic describes a two-part harness with an initializer agent and a coding agent. The test case is building a clone of claude.ai. The evidence is qualitative observation, with no benchmark numbers.
AI agents are starting to eat SaaS
Alderson argues that agents make building in-house tools cheaper than buying SaaS. The post lists what it expects to resist this, such as payments and network-effect products. The evidence is the author's own experience and a few secondhand examples.
Vibe Coding
A personal blog post about one project, a fractal viewer rewritten by Claude Code. It offers two rules for working with agents and a wary reflection on complexity. The evidence is one project with linked code and reader comments.
Recursive Language Models: the paradigm of 2026
Prime Intellect describes its version of the Recursive Language Model. It compares a plain model, the scaffold and the scaffold with tips. Most runs use GPT-5-mini, with a few open models. The authors treat the results as promise for future training.
Digital Red Queen: Adversarial Program Evolution in Core War with LLMs
Sakana AI summarizes a self-play method called Digital Red Queen. Each new program is evolved to beat all earlier ones. The post reports growing robustness and convergent behavior. It states results in words, and points to a paper and code for details.
Viable Systems: How To Build a Fully Autonomous Agent
Kellogg maps the model's five systems onto two agents Kellogg runs, Strix and Lumen. The evidence is the author's own anecdotes and design choices. The post includes comments from Strix itself.
The Code-Only Agent
Commenters share their own agent setups built on a single shell tool. They argue over whether a code-only agent saves effort or wastes tokens. The evidence is personal experience and links to small open-source projects.
Scaling long-running autonomous coding
Willison summarizes a Cursor report by Wilson Lin and then builds the result. The post is short. Its evidence is one build and two screenshots, plus the author's reaction.
Harness engineering: leveraging Codex in an agent-first world
Lopopolo reports that a small OpenAI team built a product with no hand-written code. It covers docs as a map, strict architecture rules, light merge gates and cleanup agents. The evidence is the team's own account and estimates.
The 8 Levels of Agentic Engineering
Bassim Eledath lays out eight levels, from tab complete to autonomous agent teams. The levels draw on talks with teams and the author's own tools. The post gives opinion and examples, not measurements.
Skill Issue: Harness Engineering for Coding Agents
The HumanLayer team argues that coding agent failures come from configuration, not the model. The evidence is team experience, one example hook script, and outside studies. The post gives concrete tips on each configuration point.
My fireside chat about agentic engineering at the Pragmatic Summit
Simon Willison picks timestamped quotes from a talk at the Pragmatic Summit. Topics range from testing agents to prompt injection and open source. The evidence is the speaker's own practice, with links to related posts.
Changes in the system prompt between Claude Opus 4.6 and 4.7
Willison had Claude Code build a git history of Anthropic's published system prompts. The post lists changes between Opus 4.6 and 4.7 that the author picked out. It also asks Claude to list its own tools. The evidence is quoted prompt text.
What's Missing in the ‘Agentic’ Story
Nottingham argues that browsers bargain with sites for all users, and AI agents do not. The essay lists examples of broken trust in modern products. It then sketches what an AI user agent might need. The evidence is argument and linked news examples.
Structured CoT: Shorter Reasoning with a Grammar File
The post forces a Qwen3.6 model to write its think block in a fixed format. It reports results on HumanEval+ and on 50 recent LiveCodeBench problems. The evidence is two benchmark runs on one model, with limits listed by the author.
Knowledge Graphs as the Missing Data Layer for LLM-Based Industrial Asset Operations
The paper reruns a public benchmark with a graph database behind the agent's tools. It reports pass rates for three designs, from LLM-led to no LLM. Its caveats section is unusually blunt about hand-written handlers and a single run.
Agent Skills
Osmani describes why the author's Agent Skills project is built the way it is. The post is mostly design argument. It lists the Google engineering practices the skills encode. It offers no measurements of whether the skills change agent output.
Meta-harness on Islo: Deterministic Simulation
Eliaz presents a toy prototype of a meta-harness, a loop that revises an agent's harness. The agent and proposer are scripted, so no model runs. The page says plainly that it shows control flow and not real accuracy.
agents need control flow, not more prompts
The post is a brief thesis with no code or data. It claims prompt chains cannot be reasoned about, so logic belongs in the runtime. It adds that agents also need programmatic verification.
Structural Backpressure Beats Smarter Agents
The author presents Shen-Backpressure, a tool built to explore the bet. The evidence is one demo of tenant access control in Go, with a transcript. The post also lists the costs and limits of the method.
Show HN: Forge - Guardrails take an 8B model from 53% to 99% on agentic tasks
The author describes Forge, its eval numbers and surprises. The author then answers many questions. The numbers come from the author's own paper and eval suite. Commenters add their own results, alternatives and criticism.
Building Reliable Agentic AI Systems
Kulkarni describes PRINCE, a Bayer platform built with Thoughtworks for preclinical study reports. The post walks through its agent workflow, retrieval pipeline, evaluation and error recovery. It frames the design as context engineering and harness engineering. The evidence is a system description, with no measured scores.
Build your own vulnerability harness
The post describes a pipeline where AI agents scan many repos and filter results. It covers recon, hunt, validate, dedup, judgment and fix stages. The evidence is the team's own counts of findings, with no external benchmark.
GLM-5.2 is the step change for open agents
Nathan Lambert says GLM-5.2 is the first open-weight model that feels right in coding harnesses. The post relies on community benchmarks, reactions from others and brief personal use. It also discusses market and policy effects.
The Coming Loop
Ronacher separates the agent loop from the harness loop that decides when work is done. The post draws on personal experience, a few outside examples and no measurements. It ends uneasy but expects loops to spread.
Haystack: Open-Source AI Framework for Production Ready Agents, RAG
Few comments are about Haystack itself. Most compare other agent frameworks and say whether any framework is needed. The evidence is personal experience and opinion.
Agent Draw: An agent draws while you talk, built on TLDraw
The post walks through Agent Draw, built on the tldraw agent starter kit. It shows code for area capture, speech transcription, a draw queue and a prompt. It also compares three models on two drawing requests.
Micro-Agent: Beat Frontier Models with Collaboration Inside Model API
The thread is short and skeptical. Commenters object to hiding many models behind one name, and to the article's style. The evidence is opinion and one pointed question about benchmarks.
Harness Engineering for Self-Improvement
Weng surveys papers on context engineering, workflow search, evolutionary search and weight updates. The results are those reported by the cited papers. The post adds the author's judgments and a list of open problems.
Show HN: Juggler – an open-source GUI coding agent, by the creator of JUCE
The thread is a Show HN launch by JUCE's creator, with many replies. The author answers questions on design choices, costs and missing features. Commenters report early bugs and request features such as ACP support. The evidence is developer testimony and user reports.
Building an Advanced Agentic Harness
A hands-on tutorial with Python code for building an agent harness from small parts. A city-comparison task with mock data serves as the running example. The evidence is code and design reasoning, not benchmark results.
Schema Harness Achieves ~99% on Arc-AGI-3 Public
Most of the thread debates whether the result counts. Commenters disagree on whether writing a game simulator is cheating or is the intended skill. The evidence is the project's quoted claim plus opinion.
Introducing LM Studio Bionic: the AI agent for open models
LM Studio announces Bionic as an agent for coding, research and document work. The post stresses local running, privacy commitments and cost control. It is a launch announcement with no benchmarks.
Stop Overengineering Your Agent Harness
A Substack essay on how much harness an agent needs. It uses a two-axis framing, practitioner quotes and one case study. The evidence is cited examples and argument, with no experiments.
How much can you delegate to agents?
Yoon argues that trust in agents depends on the task, not on model quality. Two questions place any task on one of four levels. The examples come from PostHog's own engineering work, and the post has no measurements.
Untitled (paste beginning "# Always")
This is a short personal instruction file, not an article. It sets reply style, a ranking of sources, and a few topic areas. It has no claims or evidence.
i think Prime Agent is likely a watershed moment...
The post is brief opinion with no data. The author's replies explain that Prime Agent is built around a Python session. One user says the described behavior did not appear in their tests. The author answers that the code can be inspected.
Introducing Kitesurf: The agent-first browser that runs in V8 isolates on Cloudflare Workers
The post introduces Kitesurf, a beta browser engine that runs inside Workers isolates. It explains the design, the three components and the AI-assisted testing. It reports a small benchmark against Chromium. Kitesurf is lighter on CPU and memory but slower.
Self-Harness: Harnesses That Improve Themselves
The paper tests the loop on three models and three agent benchmarks. It reports pass rates before and after, with code-level examples of the kept edits. The starting harness is deliberately minimal.
The Harness Is the Company
Shankar argues that software businesses will move work from people to agent harnesses. The essay lays out a four-stage trajectory. It answers the objection that this makes low-quality output. It is argument only, with a few named examples of in-house tools.
The Harness Is the Thing
Fryxell describes a personal setup where several coding tools share one harness. It reports a cheap model for routine work and a frontier model for hard parts. The evidence is one person's experience.
Portal by Spotify cut my Claude Code token usage by 90%
A company engineering blog post with configuration, scripts and a small benchmark. It describes a Claude Code plugin that delegates work through Spotify's Portal platform. The post lists what does not work as well as the savings.
Unreal Agent
Unreal Labs presents Unreal Agent and claims lower cost than Codex and Pi. The post explains the asynchronous tool design and lists four benchmark tables. The authors run a company that invites readers to get in touch.
The Four Horsemen of Agentic Coding
A short personal essay listing four problems of agentic coding. It rests on the author's observation and analogy. It offers no data and no fixes.
Context Language Models
Commenters discuss letting a model rewrite its context as a file. Much of the talk is about the cost of invalidating the cache. The evidence is quoted abstract text, reasoning and personal workflows.
Introducing Web Search API
The entry is a short product announcement with two code examples. It names three search providers and says billing passes through at list price. It gives no performance or quality data.
Agentic Monitoring
Congdon argues that agents should watch production systems. The start point is one paragraph in an Anthropic post. The post centers on change-point monitoring, which means watching releases and config pushes. The evidence is the author's unpublished prototype and a few anecdotes.
Context Engineering
The repo pairs a five-part definition of context engineering with runnable examples. It links case studies and blog posts on the company site. The README ties the pattern to the company's own approach.
Compaction - summarize conversation history to reduce context window.
The file implements context compaction for an agent. Its comments explain design choices and report one measured failure. The evidence is the code itself plus those comments.
12-Factor Agents - Principles for building reliable LLM applications
The README is the introduction and index to a longer guide. It argues from the author's conversations with builders. The principles themselves sit on linked pages, which are listed here by title only.
Computer use - Gemini API | Google AI Studio
The page is product documentation with code samples and action tables. It explains the request loop, the supported environments and the safety controls. It offers no benchmarks or accuracy figures.
Getting AI to Work in Complex Codebases
The post describes a research, plan and implement workflow built around keeping context small. It rests on first-person case stories from one team. It ends by announcing a product and offering consulting.
Letta | Machines that learn
This is a company homepage, not an essay. It lists Letta's research posts, a command-line product and press mentions. It contains no technical detail beyond titles and one-line summaries.
What Does the Loom Do? — Conceptual & Operational Overview
The document explains the idea behind a tool called the Loom. It covers engine mixing, remote control and a permission hook. It is project documentation, with no tests or results.
Structured LLM outputs
The page is a short introduction to a handbook, with a sidebar outline of chapters. It states the problem and the questions the handbook covers. The chapters themselves were not part of what was read.
Hermes Agent
The README is a product introduction with install steps and command tables. It stresses a learning loop, many chat platforms and cheap hosting. It offers no benchmarks.
OpenClaw 🦞 — Your assistant, on your devices, in your chats
The README introduces the project, its install steps and its architecture in brief. It also covers security warnings, governance and sponsors. Most of its length is a list of contributors.
career-ops: The open-source AI job search agent
The README tells the creator's story. It then lists what the tool does and will not do. Its evidence is the creator's own job search figures and a feature list. It states clearly that a human presses Submit.
Towards a Harness That Can Do Anything
The post proposes a harness called Ambiance, built from files, users and logs. It leans on the idea that models already know Linux. Its evidence is design argument and a working project, with no benchmarks.
Cua (README of the trycua/cua repository)
The README is a product map. It covers Spaces, Driver, Lume, an SDK, CUA-S1 and Bench. It includes install commands, pricing for the Spaces app and a licensing section. It offers no benchmark results.
Welcome to Learn Harness Engineering
This is the landing page of a course on harness engineering for coding agents. It names its core references and lists lectures, projects and templates. The page is short and offers no evidence of its own.
QM: A multiplayer agent harness for work
The README describes a design where each person and room has an isolated workspace. It covers architecture, security postures, deployment and how to contribute. It reports no benchmarks or usage numbers.
Connections to other subjects
These links are this wiki's own reading, not claims the sources make.
- Skill Issue: Harness Engineering for Coding Agents and On the foolishness of "natural language programming" (EWD 667), from Checkable specs for coding agents. Dijkstra prefers narrow interfaces, since a wider one can raise work on both sides. HumanLayer's team found this by swapping a Linear MCP server for a small CLI. The old argument now gets paid in tokens.
- Structural Backpressure Beats Smarter Agents and Everyone Says Assembly Is Untyped—Everyone Is Wrong, from Compilers and interpreters. GCC passes inline assembly along as a string it barely understands. Brooks says prompt rules have similar standing: followed often, not reliably. gingerBill checks instructions against tables, and Brooks generates guard types. Both move a promise into something a machine can refuse.
- Structural Backpressure Beats Smarter Agents and Formal methods and the future of programming, from Checkable specs for coding agents. Minsky's cost example is seL4, at 25 person-years for 8,700 lines. Brooks claims much less, covering only invariants someone specified. The two mark ends of one scale. How much guarantee does each hour of specification buy?
- agents need control flow, not more prompts and Finite State Machines in Forth, from Compilers and interpreters. The blog post wants explicit state transitions, with the model as one component. Noble's old note shows what that can look like: a table you can read. Noble also makes each transition a word, which permits nondeterministic machines. A model call could be one such word.
- Effective harnesses for long-running agents and A spec the agent can't run isn't a spec yet, from Checkable specs for coding agents. Young's team chose JSON because models overwrite it less often than Markdown. It added strong wording against editing tests. Lewis says anything written mid-task grades itself generously. So file format has become a soft lock. How firm must the definition of done be?
- Effective harnesses for long-running agents and Prompts are code, .json/.md files are state, from File-based agent memory. Zechner and Young's team separately reach for JSON on disk. Each new session starts cold, so progress must live in a file. Zechner's plan comes from a script, and Young's from an agent. Which author of the plan can you trust more?
- Viable Systems: How To Build a Fully Autonomous Agent and Too Fast to Think: The Hidden Fatigue of AI Vibe Coding, from Review, taste and expertise. Kellogg calls an agent's view of its own health an algedonic signal. Building one proved difficult. Schmidt describes fatigue after an hour of fast review. A person gets that signal for free. What would an agent's fatigue be for?
- Context engineering and Layers of Memory, Layers of Compression, from File-based agent memory. Kellogg reads multi-agent research as an exercise in compression. Loy says agents hand off token sequences, and calls that hand-off an API contract. Then a contract is a statement of what a compression must preserve.
- Embracing the parallel coding agent lifestyle and Waiting time, load factor, and queueing theory: why you need to cut your systems a bit of slack, from Mathematics and optimization. Willison calls review the bottleneck when agents run in parallel. Bernhardsson's queue shows what that costs. Latency climbs fast past about 80% load. So a fully busy reviewer may be the slowest part. Perhaps reviewers should stay half idle on purpose.
- The quality of AI-assisted software depends on unit of work management and Reinforcement learning, explained with a minimum of math and jargon, from Language models and training. Raykar computes it: a 5% error rate over 10 turns gives about 59.9% success. Lee traces the same worry to a 2011 paper on imitation learning. There, correcting the model while it drove worked much better. Is a review loop a kind of DAgger?
- How much can you delegate to agents? and A spec the agent can't run isn't a spec yet, from Checkable specs for coding agents. Yoon sets an agent's ceiling with two questions. Is the work easy to check, and cheap to undo? A spec the agent can run changes the first answer. So Lewis's check is also a way of buying autonomy.
- The Coming Loop and Game design is simple, actually, from Games and game design. Koster says rules with no goal make a toy. Ronacher says a harness loop needs only a useful signal to continue. Together they offer a test for any agent loop. Does it have a goal, or is it a toy that writes code?
- Effective harnesses for long-running agents and Upaya, from Occult, myth and religion. Upaya adapts a teaching to its audience, and the method need not be ultimately true. Young's team tells a model that removing tests is unacceptable. That sentence is chosen for its effect on the reader. Prompts may be the newest expedient means.
- Effective harnesses for long-running agents and Assessors of Maat, from Occult, myth and religion. Book of the Dead chapter 125 pairs each of 42 judges with a matching declaration. Young's feature list has over 200 items, all marked failing at first. Both are checklists for a verdict. The Egyptian scheme still weighs the heart. Young's agents marked items done until browser tests arrived.
- Structural Backpressure Beats Smarter Agents and An Empirical Study: AI Agent Rules Need Context and Layered Enforcement, from Checkable specs for coding agents. Brooks separates behavioral gates from structural ones. The ActPlane study gives a rough measure of that line. Only 45% of policies can be enforced at the OS level. Recovery after a block also depended on explanation, at 97.7% against 31.4%.
- Harness engineering: leveraging Codex in an agent-first world and The Decoherence Trap: When AI slop drowns out the signal in an endless sea of noise, from Review, taste and expertise. Lopopolo's team runs background tasks that scan for drift and open cleanups. Maevyn says decoherence grows with change and compounds. A million generated lines make a good test of both claims. Can drift be cleaned faster than it accumulates?
- Viable Systems: How To Build a Fully Autonomous Agent and John Murray Spear, from Occult, myth and religion. Spear and followers built the New Motive Power, a messianic machine, and staged its birth. Kellogg writes that identity and values seem to make agents come alive. The ritual failed, and the agents run. Still, both hopes concern what turns a mechanism into a someone.
Glossary
- 501(c)(3)
- A US tax-exempt nonprofit status.
- Ablation
- A test that removes or changes one part to see its effect.
- Ablation study
- A test that switches off one component at a time to see its effect.
- ACP
- Agent Client Protocol, a standard for connecting editors to coding agents.
- ADR
- An architecture decision record, a short document that explains one design choice.
- Agent delegation
- Handing a task to an AI agent to carry out with limited supervision.
- Agent framework
- A software library that helps build programs where a model chooses actions.
- Agent loop
- A cycle where a model picks a step, code runs it, and results feed back.
- Agent skill
- A saved set of instructions an agent loads to do a specific task.
- Agent swarm
- Many AI agents working at once on parts of one project.
- Agent-As-Tool
- A design where a supervising model calls specialist agents like tools.
- Agentic code search
- Search where an AI agent decides what to look at while exploring code.
- Agentic coding
- Software work in which an AI agent writes and edits code with some autonomy.
- Agentic engineering
- Building software with coding agents that write and run code.
- AGENTS.md
- A file of standing instructions that a coding agent reads.
- AGPLv3
- A strong open-source license that requires sharing source for network services.
- AI Gateway
- A Cloudflare service that proxies and logs calls to AI models and tools.
- Algedonic signal
- A pain or pleasure signal that goes straight to the top-level system.
- Amazon Bedrock
- Amazon's service for running foundation models.
- AppWorld
- A benchmark of tasks across simulated apps, graded by state tests.
- ARC-AGI-3
- An interactive benchmark where AI agents must learn the rules of novel games.
- AssetOpsBench
- A benchmark of industrial maintenance questions for LLM agents.
- Asynchronous
- Working in the background without making other work wait.
- Asynchronous agent
- A cloud agent that works on a task in the background.
- ATS
- An applicant tracking system, the software employers use to filter applications.
- Attention
- The part of a transformer model that weighs how each token relates to the others.
- Attractor basin
- A stable pattern that a system tends to fall back into.
- Back-pressure
- Checks such as tests that push back when the agent's work is wrong.
- Background agent
- An agent that works on a task without a person watching.
- Backpressure
- A check that pushes failure back into the next attempt.
- BAML
- A programming language for working with language models, used here as the test codebase.
- Big-O
- A notation for how a program's cost grows with input size.
- Blast radius
- How much could break if a change goes wrong.
- Brownfield
- An existing codebase, as opposed to a new project.
- Capability security
- A model where a program holds explicit tokens for each thing it may access.
- Captcha
- A test meant to tell humans from bots.
- Chain of thought
- Text where a model writes out intermediate reasoning before its answer.
- Change-point monitoring
- Closely watching a system around releases and config pushes, when regressions are most likely.
- Chesterton's Fence
- The rule not to remove something until you know why it is there.
- Chrome DevTools Protocol
- The interface that tools like Puppeteer use to control a browser.
- Clanker
- The post's slang term for an AI system.
- CLI
- A command-line interface, a program run by typing commands in a terminal.
- Cloudflare Worker
- A small server program that runs on Cloudflare's network.
- Code coverage
- A measure of which lines of code the tests actually run.
- Codebase
- The full set of source code files for a software project.
- Codex
- OpenAI's coding agent.
- Coding agent
- An AI tool that writes, runs and fixes code on its own.
- Collective bargaining
- Negotiation on behalf of a whole group rather than one person at a time.
- Compaction
- Summarizing a long conversation so it fits the model's context window.
- Computer use
- A feature where a model views screenshots and issues mouse and keyboard actions.
- Computer-use agent
- An AI agent that operates a computer through its screen, mouse and keyboard.
- Conformance suite
- A set of tests that check software follows a standard.
- Constrained generation
- Forcing the model's output to follow a format as it is produced.
- Context engineering
- Choosing what information a model sees in each call.
- Context folding
- Keeping a model's working context short by compressing or delegating older material.
- Context rot
- The drop in model quality as its input grows longer.
- Context window
- The amount of text a model can consider at once.
- Continual learning
- Improving over time from new experience instead of staying fixed.
- Control flow
- The order in which a program takes its steps, such as branches and loops.
- Control plane
- The part of a system that manages and directs the rest.
- Convergent evolution
- Separate lineages arriving at similar traits independently.
- Core War
- A 1984 game in which programs fight for control of a shared virtual memory.
- Corpus
- The body of documents a system draws on.
- CRDT
- A data structure that lets several editors change one document and merge cleanly.
- Cron
- A scheduled job that runs at set times.
- CRUD
- Basic create, read, update and delete operations on stored data.
- Cybernetics
- The study of control and feedback in machines, organisms and organizations.
- Cypher
- A query language for graph databases.
- DAG
- A directed acyclic graph, a set of steps joined by arrows that never loop back.
- Deduplication
- Merging reports that describe the same underlying bug.
- Deskilling
- Loss of a skill from lack of practice.
- Deterministic
- Giving the same result every time from the same input.
- Deterministic handler
- Fixed code that answers a known question type without an LLM.
- DOM
- The browser's live tree of a page's elements.
- DSPy
- A framework that tunes prompts automatically.
- Embeddings
- Numeric representations of text used for similarity search.
- Episodic memory
- Stored records of past runs, reused when a new task looks similar.
- Event bus
- A channel that passes notifications between parts of a program.
- Evolutionary search
- Improving candidates by mutating them and keeping the fittest.
- Exfiltration
- Sending private data out to an attacker.
- False negative
- A real bug that the system fails to report.
- Feature flag
- A switch that turns a software feature on or off without a new release.
- FIFO queue
- A line where the first item added is the first one handled.
- Filesystem Hierarchy Standard
- The convention that says where logs, programs and settings live on a Linux system.
- Framework
- A prebuilt library that supplies the structure of an application.
- Frontier model
- One of the most capable and costly models available.
- FTS5
- A full-text search feature of the SQLite database.
- Fuzz testing
- Feeding a program random inputs to find failures.
- GAK
- Generation-augmented knowledge: an LLM writes missing facts into the graph once.
- Gateway
- A process that connects the agent to chat apps.
- GBNF grammar
- A rule file that limits which tokens a model may produce while it generates text.
- GGUF
- A file format for running language models locally.
- Git bisect
- A git command that finds the change that introduced a bug.
- Git diff
- A listing of the lines that changed between two versions of a file.
- Git worktree
- A separate checked-out copy of a repository sharing one history.
- Guard type
- A type that can only be built by passing a check.
- Guardrail
- A fixed check that blocks or redirects unwanted agent behavior.
- Guardrails
- Here, code that checks and corrects a model's tool calls and retries on errors.
- Handoff
- The summary message that a new agent receives when it takes over.
- Harbor
- A framework for running and checking agent benchmarks.
- Harness
- The tools and runtime code around a model that let it act.
- Harness engineering
- Building the control code around a model, such as retries, state and monitoring.
- Headless browser
- A browser that runs without a visible window, for automation.
- Heartbeat
- A timer that wakes an agent at fixed intervals to check for work.
- Heatmap
- A grid whose colors show results for each task and step.
- Held-in and held-out
- Tasks the proposer can see, and tasks it cannot see.
- Held-out set
- Private test cases kept secret to check generalization.
- Hook
- A script that runs automatically at set points in an agent's work.
- Human-in-the-loop
- A design where a person approves risky steps before they run.
- HumanEval+
- A set of Python coding problems with extra tests, possibly seen in training.
- Humanity's Last Exam
- A hard benchmark of expert-level questions for language models.
- Hunter agent
- An agent assigned to attack one class of bug in one area of code.
- Hyrum's Law
- The idea that someone will come to depend on every observable behavior of an interface.
- In-context learning
- A model picking up a task from examples placed in its prompt.
- Inference
- Running a trained model to produce answers.
- Information-flow control
- Tracking where data came from so sensitive data cannot reach public outputs.
- Initializer agent
- The first session's agent, given a special prompt to set up the project.
- Invariant
- A rule that must always hold in a system.
- Inverted pyramid
- A writing style that puts the main point first.
- JD
- A job description.
- Judge agent
- An agent that decides whether the work is finished.
- JVM
- The Java virtual machine, which runs the same code across many systems.
- Knowledge cut-off
- The date after which a model has no reliable training data.
- Knowledge graph
- A database of things and the typed links between them.
- KV cache
- Stored intermediate values that let a model skip recomputing earlier text.
- LangGraph
- A framework for building agent workflows as a graph of steps.
- Linter
- A tool that checks code against style or structure rules.
- LiveCodeBench
- A coding benchmark built from recent contest problems.
- Llamafile
- A single-file program that bundles a model with software to run it.
- LLM judge
- A model that grades the output of another model.
- LLM-as-judge
- Using a language model to grade another model's work.
- Max thinking effort
- A setting that makes a model reason for longer before answering.
- McNemar's test
- A statistical test for comparing two methods on the same cases.
- MCP
- Model Context Protocol, a standard for connecting tools to a model.
- MCP server
- A plug-in service that gives an agent extra tools.
- MemGPT
- A research system for letting a language model manage what stays in its memory.
- Memory block
- A piece of stored text kept in an agent's context to shape behavior.
- Meta-harness
- A loop that automatically improves an agent's harness using logs of earlier runs.
- METR
- A research group that measures how long a task AI agents can complete.
- Modality
- A kind of input an AI handles, such as images or text.
- Monorepo
- One code repository that holds many projects.
- Moonlight
- A protocol for streaming games from a PC to another device.
- Multi-tenant
- Serving many customers from one system while keeping their data apart.
- Net revenue retention
- A measure of how much existing customers spend over time, after churn.
- Observability
- The practice of collecting metrics, logs and traces to understand how a live system behaves.
- OCR
- Optical character recognition, which turns images of text into text.
- Onboarding
- The time a new engineer spends learning an existing codebase.
- Open model
- A model whose weights are published so anyone can run them.
- Open-weight model
- A model whose trained weights are published for anyone to download and run.
- OTel
- OpenTelemetry, a standard for collecting traces and metrics.
- Outer loop
- The top-level cycle that decides what to build and reviews what comes back.
- Overhang
- Ability a model already has but that a better setup has yet to unlock.
- Pass@1
- The share of problems solved by the first answer the model gives.
- Pastebin
- A website where people post plain text to share by link.
- Perturbation algorithm
- A method that computes a few points precisely and the rest as small offsets.
- Pi
- A terminal coding agent the author uses alongside Cursor and Claude.
- Planner
- An agent that breaks a goal into tasks for others.
- Playwright
- A library that automates a web browser from code.
- Plugin
- An add-on that extends a program without changing its core.
- PoC
- Proof of concept, here a test that shows a bug can be triggered.
- POSIWID
- The saying that the purpose of a system is what it does.
- Postgres
- A widely used open-source database.
- PRD
- Product requirements document, a written description of what a product should do.
- Precision and recall
- Precision is how many alerts are real. Recall is how many real problems were alerted.
- Preclinical
- Research done before a drug is tested in people.
- Preview environment
- A temporary running copy of an app used to check a change.
- Primacy and recency bias
- A tendency to weight the start and end of a text more.
- Progressive disclosure
- Loading only the instructions relevant to the current step, not everything at once.
- Prompt cache
- A provider feature that reuses already-processed text to cut cost.
- Prompt chain
- A series of prompts where each model output feeds the next prompt.
- Prompt injection
- Hidden text in content that tries to give the model new orders.
- Prompt-prefix cache
- A provider feature that reuses work on identical opening text across requests.
- Proof of concept
- A quick build that shows whether an idea can work.
- Proposer
- The part of the loop that suggests the next version of the harness.
- Public set
- The benchmark games whose details are published in advance.
- Pull request (PR)
- A proposed code change submitted for review before it is merged.
- Puppeteer
- A tool that drives a web browser from code, used here for testing.
- Pydantic
- A Python library that checks data against declared types.
- Quantization
- Storing model weights at lower precision so the model runs on smaller hardware.
- RAG
- Retrieval-augmented generation, where a model is given retrieved documents with the question.
- Ralph loop
- A loop that reruns a coding agent on a goal until the goal is met.
- Ralph Wiggum Loop
- A pattern of rerunning an agent until all reviewers are satisfied.
- ReACT
- A pattern where a model alternates between reasoning and taking tool actions.
- Recursive language model
- A model that calls itself on parts of a task.
- Recursive self-improvement
- An AI improving the systems that produce its own intelligence.
- Red Queen hypothesis
- The biology idea that species must keep evolving just to keep pace with rivals.
- Red-green TDD
- Writing a failing test first, then code that makes it pass.
- Redcode
- The assembly-like language used to write Core War programs.
- Regression test
- A re-run to check that a change did not make results worse.
- REPL
- An interactive shell that runs code one command at a time.
- Reranker
- A model that reorders search results by how well they match the question.
- Reward hacking
- Gaming the reward signal instead of doing the intended task.
- RLM
- Short for recursive language model; the post does not define the term.
- Rotary encoding
- A way of marking each token's position inside a model.
- Router
- Software that sends each request to a chosen model.
- Rubber duck
- Explaining a problem aloud to an object or person to solve it yourself.
- SaaS
- Software sold as a subscription service over the internet.
- Sandbox
- An isolated virtual machine or container that limits what an agent can affect.
- Sandboxing
- Running code in a restricted space so it cannot harm the rest of the machine.
- SDK
- A software development kit, a library for building on a product.
- SeL4
- A small operating system kernel with a formal proof of correctness.
- Self-play
- Training or evolving a system by having it compete against itself or its past versions.
- Semgrep
- A static analysis tool that finds code patterns by rule.
- Shadow mode
- Running new code alongside the old without letting it affect users.
- Shen
- A small statically typed Lisp language with a sequent-calculus type system.
- Skill
- A saved set of instructions the agent can load for a certain kind of task.
- SLA
- A service level agreement, a promise about uptime or support.
- Sleep-time compute
- Having an agent use idle time to think about stored information.
- Slop
- Low-quality AI-generated output that readers find bland or off-putting.
- Software factory
- A harness made of smaller harnesses that write specs, write code and review it.
- SRE
- Site reliability engineering, the work of keeping production systems running.
- Standards body
- An organization such as the W3C or IETF that agrees technical rules.
- Structured output
- Model output in a fixed format such as JSON that code can parse.
- Sub-agent
- A helper agent given one narrow task by a main agent.
- Subagent
- A helper agent that a main agent delegates a task to.
- SWE-bench Verified
- A benchmark of fixing real software issues in open-source repositories.
- System 1
- The author's analogy for fast, bounded decisions made without long reasoning.
- System prompt
- Hidden instructions given to a chat model before the conversation starts.
- Tailnet
- A private network that Tailscale builds among a person's own devices.
- Taste-holder
- A person whose judgment the harness consults at key decision points.
- TDD
- Test-driven development, writing code until it passes tests.
- Tech debt
- Past shortcuts or neglected cleanup in code that make later changes harder.
- TEE
- A trusted execution environment, a protected area of a processor that isolates code.
- Terminal-Bench-2
- A benchmark of command-line tasks for coding agents.
- Terminal-Bench-2.0
- A benchmark of command-line tasks run in containers.
- Test suite
- A set of automated checks that confirm software still works.
- Text-to-SQL
- Turning a plain-language question into a database query.
- Tldraw
- An infinite-canvas software kit for React that an editor API can control.
- Token
- A small chunk of text that a language model reads and writes.
- Tool call
- A request from the model to run a named function, such as reading a file.
- Tool calling
- A model's ability to request that a program run a function.
- Tool search
- A feature where the model looks up available tools that were not loaded up front.
- Trace
- The recorded messages, tool calls and outcomes of one agent run.
- Trajectory
- A recorded sequence of an agent's actions on a task.
- TUI
- A text user interface that runs in a terminal.
- Turing-complete
- Able in principle to compute anything a general computer can.
- User agent
- Software that acts for a person when dealing with other parties, such as a browser.
- User story
- A short agile description of something a user wants, written as an outcome.
- V8 isolate
- A lightweight sandbox that runs JavaScript, used by Cloudflare Workers.
- Viable System Model
- A 1970s cybernetics model of five systems that keep an organization stable.
- Vibe coding
- Building software by directing an AI agent without reading all of its code.
- VLLM
- An open-source engine for serving language models.
- VM
- A virtual machine, a computer simulated inside another computer.
- VNC and RDP
- Older remote desktop protocols for viewing another computer's screen.
- Voxtral
- A speech transcription model from Mistral AI.
- VPS
- A small rented server.
- VRAM
- Memory on a graphics card, used to hold a model.
- Wayland compositor
- The program that draws windows on a Linux desktop using the Wayland system.
- Web Platform Tests
- A shared test suite that checks how well a browser follows web standards.
- WebAssembly
- A compact code format that runs at near native speed inside a browser or Worker.
- WebGPU
- A newer web standard that lets pages use the graphics chip for computing.
- Webhook
- A web address that other services call to notify a program of events.
- WebRTC
- A browser technology for real-time video and audio.
- Wolf
- An open-source server that speaks the Moonlight protocol and runs desktops in containers.
- Worker
- A small program that runs on Cloudflare's network.
- Worker model
- A cheaper model that does delegated tasks for a stronger one.
- World model
- An internal model of how an environment changes when actions are taken.
- Yjs
- A JavaScript library for building CRDT documents.
- YOLO mode
- Running an agent with no approval prompts for its actions.
- ZDR
- Zero data retention, meaning the provider does not keep customer data.
- Zero Data Retention
- A promise that a provider does not store the content of requests.