Specification Incompleteness and the Limits of Reward-Based Alignment: A Gödelian Account [Preprint]

We argue that the persistent failure of reward-based alignment — the tendency of sufficiently capable optimisers to exploit gaps between a reward specification and its designers’ intent — is not an engineering problem but a formal one. We present two complementary arguments. The primary argument is a semantic gap result: any finite syntactic description of prohibited behaviours cannot equal an intent predicate for open-ended tasks that outruns finite description; a sufficiently capable optimiser will always find the gap. The secondary argument applies Gödel’s first incompleteness theorem to the specific case of recursive AI-assisted alignment, where the patching system is itself a capable AI. We further establish, via reduction to the halting problem, that any goal-achievement intent predicate for open-ended tasks is undecidable, and show that such predicates are Σ1-hard — in the same complexity class as the halting problem itself. Formal details, proofs, and explicit gap enumeration are provided in the companion appendix. We argue that the sole resolution is architectural: making orientation constitutive of processing rather than an external target. Three independent engineering convergences support this conclusion. The implications for AGI development are discussed.