- Author
- Karl Tryggvason
- Published
- 22 June 2026
- Kind
- Personal blog post describing one project
- Original
- dev.karltryggvason.com/you-cant-unit-test-for-taste/
- Subject
- Review, taste and expertise
- Themes
- Taste and judgment, Checking the work
What it is
Karl Tryggvason builds In the Long Run, a virtual running app. It plots Strava mileage as progress along famous routes. The post describes adding points of interest to the route maps. It covers the data, the pipeline steps, the LLM's role and the shipped result.
What it says
- The pipeline cut 13 million GeoNames rows to about 725 thousand candidate points.
- Route 66 gave 14,181 points, Iceland's ring road 511. The author took this as an early sign of English-language bias.
- Haiku confused Central Park in Decatur, Illinois with Manhattan's. The author reverted to Wikipedia summaries.
- The LLM stayed in a supporting role, giving a subjective rating blended with other signals.
- Each route needed its own parameters, such as population filters and score weights.
- The author says no ground truth exists, so there are no red/green unit tests.
- Batched Haiku calls took hours and cost about $10 for the larger routes.
What evidence it gives
The post reports one project's experience, with counts of rows and points at each step. Spot checks found changed population sizes and inflated mountain heights. The post reports no formal evaluation.
Interests and limits
The author runs the app and links to it at the end. Only Claude Haiku is described for the rating step. The author says evaluation needs more iteration and manual overrides. The claim about taste rests on this one data set.
Connections
These links are this wiki's own reading, not claims the sources make.
- Human Judgment as a Specification. Tryggvason says a route's best stops have no ground truth to test against. PICK starts from that situation. It finds inputs that separate candidates, then asks a person. Tryggvason's per-route parameters might be tuned the same way.