- Author
- Pavel Samsonov
- Published
- 8 March 2026
- Kind
- Newsletter essay, argument with quoted sources
- Original
- productpicnic.beehiiv.com/p/checking-an-llm-s-work-is-a-systemic-not-an-individual-problem
- Subject
- Coding with agents
What it is
Pavel Samsonov writes The Product Picnic, a newsletter on UX and product management. This issue responds to the idea that people can simply check what an LLM produces. It argues that the real gap is in defining the problem.
What it says
- The post says Claude Code is popular because it fits existing speed values, not novelty.
- The author says code cannot be the source of truth; a spec must define intent.
- The author argues vibe-coded prototypes should not ship professionally, and no automated test checks requirements.
- The post says an LLM can only output a statistically average response.
- The author argues checking AI output causes burnout, so people skip the checking.
- The post says AI requirements should be checked by design critique, not individual taste.
What evidence it gives
The essay offers argument only. It quotes writers such as Scott Berkun, Anthony Moser, Jennifer Moore and Rachel Been. It links to a claim that higher oversight of AI outputs correlates with mental fatigue. It runs no tests of its own.
Interests and limits
The author writes on UX and product management. The stance is critical of AI tools. Some claims, such as AI output always falling short, are stated without a test. The essay does not compare critique against other ways of checking. The linked sources were not followed here.