Ethically Aligned Design in Autonomous and Intelligent Systems: An Overview

Much recent work in the value theory of autonomous and intelligent systems (AIS) revolves around three issues. First is the alignment problem: the problem of producing AIS whose values align with humanity's interests. Second, superintelligence: the potential for AIS to develop intelligence which would surpass even the most intelligent humans. An increasing number of authors argue that superintelligent AIS could emerge overnight because of a recursively improving process-this is the singularity hypothesis. Further, many of the same authors believe that the concatenation of these problems should direct interest to the long-term potential for misaligned and superintelligent AIS which could pose risks of existential proportions to human interests. Therefore, they argue for a policy stance which we describe as “hard alignment,'’ the proposal of cooperation with technological experts to avoid hypothetical scenarios where AIS disempower humanity. On the other hand, we describe our view as “soft alignment.'’ Considering the lack of adequate evidence for hard alignment's radical claims, the finite resources and attention of policymakers and technological experts are best served by devoting, at best, a modest amount of time, attention, and resources towards policies regarding AIS which manage the moral risks involved in misaligned, already-existing AIS, and regarding misaligned potential AIS relative to the evidentiary basis for their possible realization. Therefore, we argue for the adoption of policies towards alignment which manage the everyday risk involved in misaligned AIS rather than long-term existential risks, which are difficult to quantify.