AI identity and self-concern: A new theory for AI rights and safety

We first motivate and explain an attitude-dependent view of personal identity on which an AI system’s identity conditions are determined by its pattern of self-concern. We show that this view has important implications for the moral obligations we would have to AI moral patients. Self-concern, we contend, could also be used to predict, explain, and manipulate AI’s self-interested behavior in safety-relevant ways. The role that self-concern could play for AI identity, rights and safety generates desiderata on what self-concern should be like. To meet these desiderata, we argue that self-concern comprises belief-like self-representations as well as desire-like aspects and functionally related motivational, evaluative, and affective states. Finally, we explore how mechanistic interpretability and behavioral methods can be combined to measure self-concern in language models. Our argument suggests that a scientific understanding of language model self-concern is not only a crucial missing piece for debates on AI identity, rights, and safety, but also empirically tractable.