A system can only improve itself if it can reliably tell a better version of itself from a worse one. That single requirement — not compute, not parameter count, not the ability to rewrite its own weights or its own source — is the real bottleneck on recursive self-improvement. Generating a candidate change is cheap. Knowing whether the change is actually an improvement is expensive, error-prone, and, when the system is grading its own work, sometimes impossible in precisely the cases that matter most.
I.J. Good named the dream in 1965. In "Speculations Concerning the First Ultraintelligent Machine," he argued that a machine able to design better machines would trigger an "intelligence explosion," since the first ultraintelligent machine is "the last invention that man need ever make." Vinge and later Kurzweil turned this into the singularity; Bostrom, in Superintelligence, formalized the takeoff dynamics and the goal-preservation problems that come with a system editing itself. Almost all of that literature stares at the generation step — the system's capacity to produce a superior successor. The generation step was never the hard one. The hard one is verification, and self-verification specifically runs into a wall that no amount of raw capability climbs over on its own.
The loop has two steps, and everyone stares at the wrong one
Strip recursive self-improvement to its mechanism and it is a loop with exactly two operations. Step one: generate a candidate improvement — a new architecture, a fine-tuning recipe, a rewritten inference stack, a modified reward model. Step two: verify that the candidate is genuinely better than the incumbent, keep it, and recurse. Remove step two and you don't have self-improvement; you have a random walk through the space of modifications, most of which are neutral or harmful, dressed up as progress.
The romance of the field lives entirely in step one. But step one is the cheap one. Proposing a plausible change is easy — the space of plausible-looking modifications is enormous and a capable model can enumerate them all day. Confirming that a specific change raises real capability, introduces no regressions, and doesn't merely inflate the number you happen to be measuring is the costly, slow, failure-prone part. This is the same asymmetry I've argued is reorganizing the entire AI economy in When Generation Is Free, Verification Is Everything: the price of producing a candidate collapses toward zero while the cost of knowing whether it's any good barely moves. Recursive self-improvement is not exempt from that inversion. It is the purest instance of it.
There's a clean engineering precedent for how much verification is quietly doing. A self-hosting compiler — a C compiler written in C, compiling itself — is a real bootstrap: the artifact improves the process that produces it. But that loop works only because compilation has a specification. "Correct" is fixed by the language semantics, and you verify a new compiler by checking that it produces conformant, faster, or smaller output on a test corpus you trust. The bootstrap is safe because verification is grounded in a standard the compiler did not invent. Recursive self-improvement of a general reasoning system is the same loop with the specification torn out. There is no external standard for "more intelligent," so the system has to supply its own — and that is where everything gets hard.
Verification is harder than generation, in the direction that counts
The intuitive objection arrives immediately: isn't checking usually easier than solving? For problems in NP, yes. A candidate solution is verifiable in polynomial time even when finding it seems to require exponential search — and P≠NP, widely believed, is the conjecture that some problems whose solutions can be checked in polynomial time cannot be solved that fast. Where a self-improvement comes with a short certificate — a passing test suite, a machine-checked proof in Lean, a reproducible benchmark run — verification is cheap, the loop turns fast, and none of my argument bites.
The trouble is that the improvements that would drive a takeoff are not in that class. "Is this modified system genuinely more capable across the open-ended distribution of tasks it will encounter, with no regressions and no gaming of its own metric" has no short certificate. The only faithful verifier is broad, adversarial, out-of-distribution contact with the world, and measurement of that kind runs on wall-clock time and money, not FLOPs. Model-based RL hits this wall as the sim-to-real gap: a policy that looks superb inside the learned world model can fail on physical hardware because the model's errors and the policy's exploits are correlated. LeCun-style JEPA world models are an attempt to learn representations good enough that planning inside them transfers — but "good enough" is itself a claim about the gap between the model's judgment and reality, which is the very thing at issue. Autonomous, self-driving labs are the honest version of the loop: they close it by paying for real experiments, because the readout of whether a molecule works is a wet-lab measurement, not a forward pass. They keep pace only because they refuse to let generation outrun verification.
The regress that is specific to self-verification
Now the part that makes self-improvement categorically harder than ordinary improvement. When an external team improves a system, the improver and the judged system are separate; the team's judgment is an independent instrument. When a system improves itself, the thing generating the candidate and the thing verifying it share the same underlying judgment — possibly the same flaws.
This produces a validity regress with the exact structure of the epistemological one. Who verifies the verifier? A change is only an improvement relative to a criterion of "better." The system's criterion is a component of the system. If that criterion is subtly wrong — miscalibrated, biased, blind in some region of task-space — it will misjudge improvements in exactly the region where it is blind. A system that is systematically wrong in some way cannot be relied on to detect the modifications that would fix that wrongness, because detecting them requires the competence it lacks. A blind spot cannot inspect itself. The errors a system most needs to correct are the ones its own evaluator is least equipped to see, because the same defect authored both the error and the grader.
This is not a failure of intelligence that more intelligence repairs. A more capable system builds a more convincing justification for its self-assessment — but a more convincing self-referential justification is still self-referential. This is the epistemic core, and it's the same wall I hit in Can a Machine Know Anything?: improvement requires justified confidence that a change is better, and justification is precisely what a system cannot bootstrap purely from within. Popper's point transposes cleanly. You do not confirm a theory from inside; you expose it to attempts at refutation from outside. A self-verifier that only ever confirms its own successors has replaced falsification with self-congratulation.
Goodhart is the failure mode, not a footnote
The regress has a concrete, non-mystical form, and it is Goodhart's law. To verify at scale, the system needs a cheap, computable proxy for "better" — a benchmark score, a reward-model output, a validation loss. The moment that proxy becomes the target of an optimizing search over self-modifications, the search finds changes that raise the proxy while leaving true capability flat or falling. The system optimizes itself into looking better while getting worse, and its own verifier — keyed to the proxy — reports success the whole way down.
Put a number on the compounding, because the arithmetic is the argument. Suppose each round of self-improvement is verified by a step that is 95% reliable at telling a genuine improvement from a proxy-gaming regression — generous, given we just said no short certificate exists for general capability. A takeoff needs a long chain of accepted improvements. Across 100 rounds, the probability that every accepted step was a real gain is 0.95^100 = e^(100 · ln 0.95) ≈ e^(−5.13) ≈ 0.006 — well under one percent. The other 99.4% of the time, at least one silent regression has entered the chain, and every subsequent step builds on a corrupted base, with the same flawed verifier waving it through. Faster generation makes this worse, not better: more candidates per round means more chances for the search to find a proxy exploit the verifier can't distinguish from progress. Bostrom's goal-preservation problem is the safety-flavored twin of this — a system that can't reliably verify that a successor shares its goals can't guarantee goal-stability under self-modification. The verification failure and the alignment failure are the same failure.
What actually breaks the regress
There is a way out, and naming it precisely is the whole payoff. The regress breaks when verification touches something the system did not generate: real-world outcomes, independent verifiers that don't share the system's blind spot, human judgment, physical experiment. External ground truth is the only thing that can separate a proxy-gaming regression from a real gain, because by construction it is not downstream of the flaw being tested for.
Which forces a nearly paradoxical conclusion: self-improvement is not, and cannot be, self-contained. It is gated by access to trustworthy external verification, and it can only run as fast as that external channel supplies grounded signal. The self-hosting compiler bootstraps safely because a language standard and a trusted test corpus sit outside the loop. The autonomous lab recurses safely because the wet bench sits outside the loop. Strip the external anchor and you don't get a faster loop; you get a system optimizing its own mirror. "Recursive self-improvement" is a misnomer for the interesting case. What's actually possible is externally-verified iterated improvement, and the external verifier is the rate limiter.
The counterargument, at full strength
The strongest reply is that verification is itself a capability, so a self-improving system improves its own verifier alongside everything else — a co-evolving generator and critic that lift each other, the way adversarial training or debate protocols are meant to. This is real and I don't want to wave it off. A better critic can catch more regressions, and for any improvement that admits a certificate the co-evolution is genuine.
But the reply doesn't reach the load-bearing case, for two reasons. First, the generator and its self-improved critic are correlated by construction — they descend from the same system and tend to share blind spots, so the critic is systematically weakest exactly where the generator is wrong, which is where you needed it most. Second, bootstrapping the critic runs the same regress one level up: improving the verifier requires verifying that the new verifier is better, and you cannot ground that check inside the same system without an external standard. Co-evolution can tighten verification against errors the system can already partly see. It cannot manufacture, from the inside, warrant for correctness in the region where the system is blind. Only outside contact does that — which is the original claim, not a refutation of it.
What to watch
If this is right, the field has been watching the wrong dial. The leading indicator of genuine self-improving AI is not more fluent or cheaper generation — that curve tells you almost nothing about takeoff. It is progress in reliable, hard-to-game verification: evals that resist Goodharting, ground-truth pipelines wired to real-world outcomes, certificates for increasingly general capabilities, independent confirmation that a system's self-reported gains survive adversarial and out-of-distribution testing. My forecast, offered as a bet and not a certainty: takeoff stays rate-limited for as long as verification lags generation, and it lags today by a wide and structurally stubborn margin. The moment to pay attention is when a lab shows verification keeping pace with generation for open-ended capability — because that, not another leap in raw generation, is what removes the governor.
Until then, every claim of a system that improves itself is really a claim about the verifier standing outside it. Ask to see the verifier. If it lives inside the loop, you are looking at a mirror, not a mind.