Explainable AI Models as Mediators

Recent work in epistemology and philosophy of science conceptualizes explainable AI (XAI) tools as ‘models of models’, that is representations of opaque machine learning models whose value is assessed by their representation fidelity, and by the type of understanding they convey (e.g., explanatory, or objectual). In this work, we argue that this representation-centred view rests on a mistaken way of framing what XAI is. The problem XAI addresses, we claim, is not related to a lack of understanding, but of epistemic incapacitation. Stated otherwise, XAI tackles the inability to use opaque models for all the scientific purposes their predictive accuracy should make possible in scientific contexts. Drawing on the literature on ‘models for’ and on models as epistemic artefacts, we propose an alternative account we name XAI-as-mediators. In doing so, we shift the question from “How faithful does the tool represent a model?” to “What does the tool enable the user to do with the model?”. We then distinguish between two forms of mediation: access mediation, which restores epistemic access to a model’s inner structure and behaviour, and semantic mediation, which aligns the model’s reasoning space with a given domain’s conceptual and inferential practices, and also specify the epistemic virtues proper to each type of mediation. Finally, we situate the proposed framework against Morgan and Morrison’s ‘models-as-mediators’ account, and show how it can answer Watson's challenge against XAI.