Revisability and the Dark Room: A Second-Order Constraint on Surprise Minimisation

The dark room problem is the standing objection that a creature which genuinely minimises prediction error should seek out an unchanging, unstimulating environment and remain there. The established replies specify the content of a creature's priors, or derive exploratory drive from the epistemic component of expected free energy. Both work, and both leave a residue: they answer the question by stipulating what the generative model contains, which invites the same question one level up. I propose a different kind of answer. A self does not minimise surprise simpliciter. It minimises surprise subject to preserving its own capacity to revise the model by which it minimises. Revisability is not a further prior about the world; it is a constraint on the update rule. On this account the dark room is excluded not because it is improbable under the creature's model, but because it is self-terminating: it drives prediction error toward zero and drives revisability toward zero at the same time, and a system with no residual error has no signal by which it could ever discover that it should leave. The account has a symmetric failure mode. A self that revises everything has no floor; it is not open but formless, and is maximally exposed to capture. The criterion is therefore not revise, but keep distinct what must remain revisable from what must hold. The paper closes by noting that the same second-order property is what distinguishes a corrigible artificial agent from an incorrigible one, and that this convergence is the same constraint stated at two scales.