things I think about, written down
How LLMs went from GPT-3's 2,048 tokens to 1M+ — positional encoding, attention rebuilt around sparse and linear layers, training data, optimizers, long-context RL, and the evals that separate advertised from effective context. With figures.
Two Claude Code token savers, 14 real SWE-bench and SpreadsheetBench tasks, every token classified. Caveman and the stack both cut cost per solved task 30%, headroom 18% — at identical quality. A run reads ~460k tokens and writes ~6k; the two tools attack opposite ends of that.
Two opposite bets on how an agent should find the right code — Cursor's trained embedding index against Devin's generated wiki plus RL-trained agentic search, with Claude Code as the control that pre-computes nothing. What's actually documented, and the evaluation I've pre-registered.
Can you see loop training in a reasoning chain? 18 runs of Kimi K3 vs K2.6 across one-shot, bare continuation, and a Ralph-style stop-hook loop: a hidden ~86-token harness template, a hallucinated file workspace, opposite reasoning-budget curves — and a Busted section for the three findings my own audit killed. Five figures.
Put five frontier models in a bare shell — one bash command per turn — and they split into dreamers, workers, pragmatists, and grinders. Early data from a benchmark I'm building: confabulated completions, verification habits, long-range recall, with interactive charts.
A step-by-step walk through what a coding agent actually does between your prompt and its reply — context assembly, streaming, mid-flight tool execution, compaction, and the decision to loop again. Adapted from an interactive demo.