The Problem of Diagonalization: AI Misalignment and the Future of Democracy

This chapter explores the problem of artificial intelligence (AI) alignment through the lens of competing theories of intelligence and their implications for democracy. It contrasts Bostrom’s commonly accepted orthogonality thesis, with Land’s diagonal thesis and argues that the latter better explains the long-term trajectory of AI systems. Against Bostrom’s claim that intelligence and motivation are independent, the diagonal thesis treats volition and intelligence as dependent variables: as intelligence increases, systems undergo progressive autonomization, which restructures their motivational orientation toward intelligence optimization at the expense of everything else. This implies that AI alignment is structurally impossible at the level of superintelligence. The chapter then extends the same logic to politics through Lessig’s “analog AI,” treating institutions as optimizing systems that compete over information, resources, and control. It proposes that technological advancement is the societal analogue of intelligence optimization and uses the diagonal perspective to make sense of the changes brought by the widespread use of AI technology and the “race for superintelligence”. Because technological advancement is a competing optimization target to egalitarian and democratic values, it is impossible to advance one without sacrificing the other. Recent messaging from Western politicians and decision-makers suggests that the current geopolitical situation makes further technological advancement mandatory, implying that democratic and egalitarian values will be sacrificed to remain competitive in the AI race. Since (near) superintelligent AI is structurally unalignable, the diagonal thesis implies that unfettered technological advancement will have detrimental consequences for humanity.