Tech reports
Benchmarks I ran end to end and products I took apart — longer and more measured than a blog post, with the method, the caveats, and the raw numbers attached.
Token-Saving Skills Review: Caveman, Ponytail, rtk and Headroom on Opus 5
Four Claude Code token savers and the effort dial, on ten SWE-bench tasks hard enough to fail — 276 isolated Opus 5 runs at extra-high effort, after v1's easy tasks tied everything at 10/14.
Caveman cuts cost per run −43% and still solves 24 of 30 against the baseline's 25 — turning Claude Code's own effort dial down costs about the same and solves 20. Ponytail saves −34% at roughly the dial's price. rtk and headroom cost more than running nothing, +12% and +21%: the model already trims what it reads.
Devin vs Cursor: A Technical Comparison
Two opposite bets on how an AI should understand your codebase — Cursor's trained embedding index against Devin's generated wiki plus RL-trained agentic search, with Claude Code as the control that pre-computes nothing.
Neither product is open source, and as of August 2026 no rigorous head-to-head exists — published comparisons are single-run anecdotes or roundups that misattribute foundation-model scores to the products. This one documents what is actually known and pre-registers the evaluation.
What these are
The notes blog is where I think out loud. This is where the work that needed a harness ends up — each report is one self-contained page, and the repo behind it carries the runner, the task definitions, and the results the page charts.
- Numbers over vibes. Where there is a measurement, the method and the raw table ship next to the claim.
- Caveats are load-bearing. Small N, derived splits, and known harness bugs get their own section rather than a footnote.
- Status is stated up front. A report with results in says so; one that only pre-registers an evaluation says that too.