Ethical Challenges of Deceptive Chatbots

The central claim of this paper is that chatbots are, in an important sense, deceptive technologies; this deception, however, is not always morally problematic. It becomes so when it is about the relationship the user has with the bot. Since our relationship with chatbots is built solely on dialogue, the ethics of dialogue can offer a privileged lens for discerning harmless and harmful deception. It emerges that the ethical issue with chatbot interactions does not lie in users being deceived about the system’s nature, i.e., thinking it is human, but rather in being misled about the quality of the dialogue they are engaging in. In particular, it is morally bad when chatbots deceive the user about the mutual openness of their interaction, as such systems are inherently incapable of reciprocating. Not all chatbots engage in harmful deception: some remain within the boundaries of what is known as banal deception. Broadly speaking, it is possible to distinguish between two categories of bots: instrumental chatbots, which are task-driven and engage only in banal deception, and simulated-behaviour chatbots, which may engage in harmful deception. In this latter category, AI companions are always morally problematic as they intentionally deceive users about the mutuality of the relationship. Based on this diagnosis, some normative prescriptions are advanced to regulate AI companions and ensure they foster only banal deception. The paper contributes both conceptual clarity to the ethics of AI deception and practical guidance for the responsible design and governance of chatbot technologies.