Tech reports
Benchmarks I ran end to end and products I took apart — longer and more measured than a blog post, with the method, the caveats, and the raw numbers attached.
Caveman vs Headroom: Which Tokens Actually Get Saved?
56 isolated Opus runs on real SWE-bench and SpreadsheetBench tasks, with every token classified — which Claude Code token saver actually works, where, and the verdict a harness bug almost stole.
A tie at the top: caveman and the stack both cut cost per solved task −30%, headroom −18% — at identical quality, 10/14 each. About 460k tokens are read per run against ~6k written; caveman shrinks the 6k, headroom shrinks the 460k.
Devin vs Cursor: A Technical Comparison
Two opposite bets on how an AI should understand your codebase — Cursor's trained embedding index against Devin's generated wiki plus RL-trained agentic search, with Claude Code as the control that pre-computes nothing.
Neither product is open source, and as of August 2026 no rigorous head-to-head exists — published comparisons are single-run anecdotes or roundups that misattribute foundation-model scores to the products. This one documents what is actually known and pre-registers the evaluation.
What these are
The notes blog is where I think out loud. This is where the work that needed a harness ends up — each report is one self-contained page, and the repo behind it carries the runner, the task definitions, and the results the page charts.
- Numbers over vibes. Where there is a measurement, the method and the raw table ship next to the claim.
- Caveats are load-bearing. Small N, derived splits, and known harness bugs get their own section rather than a footnote.
- Status is stated up front. A report with results in says so; one that only pre-registers an evaluation says that too.