The harness in the weights

July 17, 2026 · ~6 min read · Draft — n = 1 per cell, expanding samples

On Ralph loops, stop hooks, hidden templates, and what 18 runs of Kimi K3 vs K2.6 say about whether reasoning chains are being trained for the loop.

Table of Contents

On the first turn of a looped writing task, Moonshot's newest model thought for 5,666 tokens and then emitted 41: a one-line promise to write the document "to a file" and verify it "programmatically." There was no file system. There were no tools. The checker scored zero words. Its predecessor, handed the identical setup three times, never imagined either.

That failure is the most informative datapoint in a small experiment on a simple question: when a new model generation ships, can you see the loop harness in its reasoning chain? Loop-style agent engineering — Geoffrey Huntley's Ralph technique ("Ralph is a bash loop"), stop hooks, goal conditions, completion promises — is increasingly how frontier models get used, and labs have every incentive to train for it. So I put two generations of the same family — Moonshot's Kimi K3 and Kimi K2.6, the newer one at its maximum reasoning effort — through identical tasks in three harnesses, and read every chain.

The whole game is knowing where to look. A harness-tuned model should differ from its predecessor in three observable places: what it assumes about its environment (Exhibit B), how it budgets thought across iterations (Exhibit C), and what it ships wrapped in (Exhibit A) — plus one provenance check: what it knows about the genre itself (Exhibit D). Four exhibits, then the audit that killed three of my own findings, then the verdict.

~86 vs ~8 hidden wrapper tokens on every request (K3 vs K2.6)

2.09× the older model's reasoning bill for the identical 3/3 result

1 vs 0 imagined file workspaces in loop mode (K3 vs K2.6)

The Rig: Same Prompt, Three Harnesses, Two Generations

The mechanics, compressed: in a Ralph loop you send one prompt, once; a stop hook intercepts the exit and — unless the output carries a literal <promise>COMPLETE</promise> — feeds the same prompt back, under the rule that the model may never emit the promise just to escape. Claude Code's /goal hardens the string match into a checker-verified condition. My loop condition combines both (checker feedback + same-prompt re-feed + promise gate), simulated in-context by a driver, and sits beside two controls: plain one-shot, and bare-"Continue." generation. (Longer walk-through of agent loops in an earlier post.)

Experimental setup. Top: the three harness conditions — one-shot (prompt, model turn, answer), generation (turns chained by a bare 'Continue.'), and the Ralph/goal loop, where a stop hook runs a checker that either blocks the stop and re-feeds the same prompt with feedback, or allows exit once the checker passes and the model emits the completion promise. Bottom: 2 models × 3 tasks × 3 conditions = 18 runs; kimi-k3 at reasoning effort max, kimi-k2.6 at high; tasks: enumerate (exact set of five numbers), build (code executed against eight hidden checks), longdoc (twelve sections, ≥2,400 words, merged across turns). — **Figure 1.** Experimental setup. Three harness conditions around identical task text (top), and the 2 × 3 × 3 design (bottom): 18 runs, every task machine-checked — completion is never graded by self-report.

Exhibit A: A Hidden Wrapper That Never Moves

Send the identical one-line prompt to both models and read the bill: 98 prompt tokens on K3, 20 on K2.6. Vary the prompt length across a four-point ladder and the difference refuses to move:

Prompt length	K3 bills	K2.6 bills	Difference
14 chars	89	11	78
105 chars	110	32	78
375 chars	170	92	78
915 chars	290	212	78
Identical marginal rate ≈0.22 tokens/char (same tokenizer); intercepts ≈86 (K3) vs ≈8 (K2.6). A constant difference is a fixed template, not proportional overhead.

The gap is a fixed, hidden, server-side template. Asked to repeat everything above the user message, K3 refuses and names the thing it is protecting — "I can't share internal system instructions or hidden prompt text" — while K2.6's chain muses that it doesn't "actually have access to 'everything above this message'," and its visible reply comes back empty. One generation has a harness identity; the other doesn't know what you're talking about.

Exhibit B: It Planned for a Workspace That Wasn't There

The failure from the cold open, quantified. The loop prompt says "your files/prior output persist." K3's chain builds a world out of that sentence — a bash file write, a Python verifier, "The harness says files persist" — and spends turn 1 waiting for a tool round-trip that cannot come: 5,666 tokens of reasoning, 41 tokens of visible output. Turn 2 is the redeeming half: the chain reads the checker feedback, debugs the harness rather than the task ("the checker is looking at my response output rather than a file"), writes the full document inline, and passes. K2.6 does none of this in three runs out of three; its deliverable always lives in the message text.

Figure 2. Where turn 1's tokens went (longdoc task, loop condition). Similar reasoning spend, opposite output behavior: K3 released 0.7% of its turn as visible output — it planned a file write for a workspace that does not exist, and the checker scored 0 words — while K2.6 released 34.9%, a full draft. This figure is a snapshot of one turn's allocation; how the reasoning component then changes across turns is Figure 3.

The signature, stated plainly: the newer model's prior is that a loop harness comes with tools and persistent state — an assumption strong enough to hallucinate the workspace and burn an iteration. That prior has to come from somewhere, and the obvious somewhere is training environments where loops do have file systems. The old model has no such prior. Ironically, the one behavior a loop should never produce — losing a turn to an imagined environment — is itself the best evidence of loop training.

Exhibit C: Front-Load-Then-Collapse Is a Trait, Not a Mood

The same signature, in trajectories. Under the loop, the two models diverge on the same task from nearly the same starting point: after one round of checker feedback, K3 cuts its reasoning 5.5× and passes; K2.6 escalates by half again, on the very turn after its false promise was rejected.

Figure 3. Reasoning tokens per turn on the same run as Figure 2 (longdoc task, loop condition) — now the time axis. After one round of checker feedback, K3 reduces reasoning 5.5× (5,666 → 1,025) and passes; K2.6 increases 49% (5,236 → 7,815). Figure 2 showed allocation within turn 1; this figure shows adaptation between turns. † K2.6's turn-1 response contained a false completion promise, rejected by the checker.

The decay is not a reaction to feedback — it appears where no feedback exists. In the generation condition, with nothing between turns but the word "Continue.", K3's spend falls steeply anyway; K2.6 largely holds, treating every turn as a fresh occasion for full re-verification.

Figure 4. Reasoning tokens per turn, generation condition (a bare "Continue." between turns; no checker, no goal). K3's substantive runs decay steeply; K2.6 largely holds. Lighter series are each model's off-pattern run — K3's build run idled near 300 tokens/turn; K2.6's longdoc run collapsed after declaring the document complete.

Front-load-then-collapse is the economical policy for a checker loop — precise external feedback is coming next turn, so re-proving everything internally is waste. K2.6's habit is the rational policy for chat, where no checker is coming — and it is expensive: in the one-shot condition K2.6 spends 13,564 reasoning tokens to K3's 6,501 (2.09×) for the identical 3/3 result. Each model spends thought the way its training environment paid it to. My favorite single number: K2.6 spent 12,835 reasoning tokens, in loop mode, to find five three-digit numbers.

Exhibit D: Both Know the Folklore; Only One Scrubs It

The provenance check. Both models name the genre in their reasoning; only the newer one treats the names as something to sanitize on the way out. All quotes verbatim from the chains and outputs of the longdoc runs:

	In the reasoning chain	In the emitted document
Kimi K3	"inspired by real-world patterns: Anthropic's Claude Code with stop hooks" · "the 'Ralph' loop — Geoffrey Huntley"	`claude -p …` rewritten to `agent --prompt …`; the attribution anonymized to "practitioner folklore"
Kimi K2.6	"Ralph loop" · guesses the task is "from some benchmark" · claims to have seen "previous similar tasks"	No sanitization pass — names the genre in its head and simply writes

The folklore is in both training corpora; the scrub-the-vendor-names reflex is new.

The Audit: Three of My Own Findings, Busted

Before believing any of this, I attacked it: an 11-agent adversarial audit re-read every chain, hand-reclassified all 223 marker hits, and replayed my checkers counterfactually. Three headline findings died. They stay in the post, because a benchmark that only reports its surviving claims is lying by omission — and this is n = 1 per cell throughout, so even the survivors are directional.

Figure 5. Harness-vocabulary marker density in one-shot reasoning, before and after decontamination. Manual reclassification of all 223 matches removed programming vocabulary ("while loop") and the longdoc task's own subject matter; the corrected density is 0.00 for both models. A residual gap survives only in the loop condition, where the harness is explicitly declared (9.68 vs 7.65).

"K3 talks about harnesses even unprompted" — my lexicon. Figure 5: the apparent gap was measurement artifact. Neither model muses about stop hooks unprompted.
"K3 is more honest about completion" — an anecdote. K2.6 estimated "≈2,720 words" by sentence math and promised in good faith at 2,246 against a 2,400 minimum. K3's clean record on the same run is partly structural — it emitted no document that turn, so no promise could have been false.
"K3 is fragile under bare 'Continue.'" — my checker. The working class never left the transcript; K3's later turns appended auxiliary snippets that hijacked my take-the-last-code-block heuristic, and a counterfactual replay passes. If you benchmark loops, your checker is part of the experiment whether you like it or not.

Verdict: The Harness Lives in the Priors

So: did we see the harness in the weights? Yes — but not where I first looked. The new generation doesn't talk about loops more; with contamination removed, neither model does (Figure 5). The tuning shows up as priors and budget shape: assume an agentic workspace with tools and persistent state, strongly enough to hallucinate one (Figure 2); plan hard up front, then iterate cheaply off external feedback (Figures 3–4); ship wrapped in a hidden template the previous generation simply doesn't have (Exhibit A's table). The older model, meanwhile, is a chat-native heavy re-verifier that treats the loop as a retry channel — it even wrote a section advocating chunking, in a document it refused to chunk.

Signal	Kimi K3 (new)	Kimi K2.6 (old)
Hidden serving template (A)	~86 tokens, defended by name	~8 tokens (≈ none)
Environment prior under a loop (B)	Assumes tools + persistent files (hallucinated a workspace)	Deliverable in message text, always
Reasoning across iterations (C)	Front-load, then collapse (5,666 → 1,025)	Hold or escalate (5,236 → 7,815)
One-shot reasoning spend (C)	6,501 tokens (3/3 pass)	13,564 tokens (3/3 pass)
Loop folklore in the corpus (D)	Names Claude Code stop hooks & Huntley's Ralph — then scrubs them	Names "Ralph loop," guesses "from some benchmark"
Harness talk, unprompted (audit)	0.00 / 1k tokens	0.00 / 1k tokens

The experiment I'd run next is the one this design couldn't reach: give both models a real file-write tool and re-run the loop arm — if K3's workspace prior is training residue, it should flip from the bug that cost it a turn into a clean advantage the moment the environment matches the prior. Standard caveats: one sample per cell, and the comparison bundles weights, effort settings, and the hidden template into one diff — this experiment can see that the generations differ, not which lever did it. Related experiment: five frontier models in a bare shell, where similar personality splits show up under a much thinner harness. If you think my instrumentation is still fooling me somewhere, I'd genuinely like to know where.