Trust and Responsibility in Human-AI Interaction

Two topics at the center of Ethics of AI and HRI regard trust in AI agents as well as the adjudication of moral responsibility in situations where AI causes harm. In this paper we aim to advance the state of the art concerning these topics in several regards: First, we propose and evaluate a new empirical paradigm for measuring appropriate or calibrated trust in AI, that is, attitudes which are neither too trusting nor too cautious. The best way to measure calibrated trust, we argue, is by contrasting trust vested in AI agents when their relevant capacities equal those of a human expert in the domain. A second shortcoming of extant work regards generalizability: Trust in, and reliance on, AI are standardly explored with respect to a single context or domain. To investigate context-sensitivity, we ran experiments (total N=1276) across five key areas of AI application. Finally, we explored perceived moral responsibility for harm caused in human-AI interaction, with a particular focus on recent philosophical debates on the topic. Our findings suggest that approximately half of the participants vest equal trust in AI and human agents when their capacities are the same. However, there is considerable variation in trust calibration across domains, suggesting that context-sensitivity needs more attention. Human agents are attributed more moral responsibility than AI agents, whereas their supervisors are blamed less than those of AI agents. This suggests that, at least according to folk morality, there are no perceived "responsibility gaps" (Matthias 2004; Sparrow 2007) and that "retribution gaps" are a genuine possibility (Danaher 2016).