The Evolution of Morality and The Problem of AI Value Over-Alignment

Current AI alignment paradigms and strategies, predominantly identify the alignment of AIs with human values as central for both short and long-term AI safety. Outlining an evolutionary theory of normativity, this thesis argues that these approaches to AI alignment are often rooted in a naïve moral realist rationalism and neglect the ever-ongoing cultural genetic coevolution underlying normativity. Consequently, these approaches threaten to lead to what this thesis terms AI over-alignment: An AI-driven reinforcement and entrenchment of explicit current values of a particular demographic, which may already be or become maladaptive in an AI-transformed world and risks cultural-genetic autophagy, ultimately reducing humanity's adaptive capacity, despite seemingly perfect value alignment. As our hedonic and/or normative preferences have been shaped by cultural and biological evolution for a bygone world, aligning AIs with these preferences, may lead to cases of what we want not being necessary what would be best for us to align AIs with. This thesis thus reviews and critiques the current state of AI alignment discourse and academic ethics in general; and outlines an alternative paradigm of an evolutionary informed ethics and applies it to the issue of AI value (over-) alignment, pointing out hitherto neglected risks and suggesting alternative strategies.