1
Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligence
arXiv:2605.03847v1 [cs.AI] 5 May 2026
Munkhdegerekh Batzorig, Purevbaatar Ganbold, Kyungbin Park, Pilkong Jeong, and Kangbin Yim
training across heterogeneous participants without centralizing raw data, making privacy preservation possible while rendering normative divergence across participants nearly inevitable. Transfer learning [5] allows agents trained in one domain to be rapidly deployed in another, amplifying capability while also transplanting behaviors that may be contextually inappropriate or normatively misaligned in the target environment. Swarm and swarm-inspired learning [6] coordinates large populations of simple agents through local interaction rules, producing collective intelligence whose global behavior cannot be predicted from any individual agent’s local policy. What these paradigms share is a structural property that makes safety analysis fundamentally harder: emergent risk. In a DCI environment, each agent may behave correctly with respect to its own local objectives and constraints, yet the composition of many such locally correct decisions—across time and across interacting agents—can produce globally unacceptable outcomes. Emergent risk is not caused by individual failures; it arises from the trajectory-level accumulation of small normative deviations under uncertainty. Current technical solutions address individual facets of this problem but do not resolve it. Constrained optimal control enforces instantaneous feasibility but ignores how trajectories drift over time. Safe reinforcement learning imposes policyIndex Terms—Mechanical conscience, distributed collaboralevel constraints but does not prevent uncertainty-induced norm tive intelligence, normative regulation, dependability, AI safety, violations at runtime. Runtime assurance frameworks react to trajectory-level compliance, emergent risk. breaches but apply binary interventions rather than continuous, graduated regulation. None of these approaches provides a I. I NTRODUCTION trajectory-level model of normative compliance that operates HE deployment of machine intelligence has moved well continuously, accounts for uncertainty, and extends naturally beyond isolated, centralized computing. Contemporary to the distributed multi-agent settings characteristic of DCI. This paper introduces mechanical conscience (MC)—a new systems increasingly operate as distributed collaborative inconcept and formal mathematical framework that addresses telligence (DCI)—a broad family of architectures in which this gap. The name intentionally draws an analogy to human autonomous or semi-autonomous agents learn, reason, and act conscience: not a rigid rule system, but a continuously active in a shared environment without any single point of global internal regulatory mechanism that evaluates whether one’s coordination. behavior remains within acceptable normative bounds across Prominent DCI paradigms illustrate the breadth of this time. In DCI environments, mechanical conscience functions shift. Edge-to-edge (E2E) architectures [1], [2] push inference as a shared regulatory substrate at the trajectory level, enabling and decision-making to the network edge, enabling real-time what we term resonant dependability: the emergent, bidirecphysical AI in autonomous vehicles and industrial automation, tional trust between human and machine intelligence that arises but at the cost of fragmented situational awareness across from sustained normative alignment. the system. Federated learning [3], [4] coordinates model The key contributions of this paper are as follows:
Abstract—Distributed collaborative intelligence (DCI)— encompassing edge-to-edge architectures, federated learning, transfer learning, and swarm systems—creates environments in which emergent risk is structurally unavoidable: locally correct decisions by individual agents compose into globally unacceptable behavioral trajectories under uncertainty. Existing approaches such as constrained optimization, safe reinforcement learning, and runtime assurance evaluate acceptability at the level of individual actions rather than across behavioral trajectories, and none addresses the multi-participant, uncertainty-laden nature of DCI deployments. This paper introduces mechanical conscience (MC), a novel concept and simplified mathematical framework that operationalizes trajectory-level normative regulation for both single-agent and distributed intelligent systems. Mechanical conscience is defined as a supervisory filter that minimally corrects a baseline policy’s actions to reduce cumulative deviation from a normatively admissible region, while accounting for epistemic uncertainty. We introduce associated constructs—conscience score, mechanical guilt, and resonant dependability—that provide an interpretable vocabulary and computable governance signals for this emerging field. Core theoretical properties are established: admissibility equivalence, existence of optimal regulation, and monotonic deviation reduction. Illustrative results demonstrate that MC-regulated agents maintain trajectory-level normative acceptability where conventional controllers drift outside admissible bounds, and that the framework naturally extends to suppress interaction-induced emergent risk in multi-agent DCI settings.
T
M. Batzorig, P. Ganbold, K. Park, and K. Yim are with the Department of Information Security Engineering, Soonchunhyang University, Asan, South Korea (e-mail: [email protected]; [email protected]; [email protected]; [email protected]). P. Jeong is with the Department of Mobility Convergence Security, Soonchunhyang University, Asan, South Korea (e-mail: [email protected]).
•
We formally define mechanical conscience as a trajectorylevel normative supervisory functional, with associated constructs: conscience score, conscience deviation, and mechanical guilt.
2
We introduce resonant dependability as the emergent trust that arises when human and machine intelligence sustain mutually acknowledged normative trajectories over time. • We establish core theoretical properties: admissibility equivalence, existence of optimal regulation, and monotonic deviation reduction. • We extend the formulation to distributed DCI settings, capturing emergent risk at individual, pairwise, and collective levels.
•
by design, emergent—they are not specified by any individual agent’s policy. This makes swarm systems particularly susceptible to interaction-induced normative drift, where an accumulation of locally innocuous decisions produces globally unacceptable collective behavior. Across all of these DCI paradigms, a common pattern emerges: locally rational behavior composes into globally problematic trajectories under uncertainty. The mechanical conscience framework is designed precisely to address this structural gap.
II. R ELATED W ORK The mechanical conscience framework intersects several distinct research threads: distributed collaborative intelligence and its emergent risks, constrained and shielded reinforcement learning, ethical-governor and consequence-engine architectures, deontic and normative multi-agent reasoning, AI alignment, and classical dependability theory. This section surveys each thread, identifies its limits relative to trajectory-level normative regulation in DCI, and positions mechanical conscience within the resulting landscape. A. Distributed Collaborative Intelligence and Emergent Risk The emergence of distributed, multi-participant intelligence architectures has introduced normative challenges that classical system safety frameworks were not designed to address. Edge-to-edge (E2E) computing [1], [2] relocates inference from centralized clouds to network-edge devices, enabling latency-sensitive applications such as autonomous driving and robotic surgery. However, E2E architectures fragment global situational awareness: no single node observes the full system state, and decisions made locally and independently can compose into globally unacceptable trajectories. Federated learning [3], [4] allows many participants to jointly train a shared model without revealing their raw data. While this design preserves privacy, it also decentralizes both learning and policy formation. Different participants may optimize under different implicit norms, and the aggregated model can inherit or amplify normative inconsistencies that are invisible to any individual participant. Recent surveys of trustworthy federated learning [7], [8] catalog these challenges along the axes of robustness, fairness, and privacy, but treat each as a static property of the trained model rather than as a property of behavioral trajectories evolving over time. The mechanical conscience framework complements this body of work by providing a runtime, trajectory-level normative regulator that operates after deployment and is agnostic to how the underlying federated model was trained. Transfer learning [5] accelerates deployment by initializing agents with knowledge from a source domain. Yet behaviors that were normatively appropriate in the source domain may be misaligned in the target environment. When such transferred agents are deployed in interactive DCI settings, their normative misalignment can propagate to other agents through shared observations or joint action spaces. Swarm and swarm-inspired learning [6] coordinates agents through local interaction rules, yielding scalable collective behavior. The global outcomes of swarm coordination are,
B. Constrained Control and Safety Filters Constrained optimal control and control barrier functions (CBFs) [9]–[11] enforce instantaneous safety or feasibility conditions on state-action pairs. CBF-based methods provide strong forward-invariance guarantees for a predefined safe set, but they evaluate acceptability at each time step independently and do not model how cumulative trajectory drift can render a system normatively unacceptable over time. Mechanical conscience complements these approaches by providing the trajectory-level regulation that point-wise methods inherently lack, while reusing them as natural building blocks for the satisfaction functions ϕi in Section III. C. Safe Reinforcement Learning and Shielding Safe reinforcement learning [12]–[14] incorporates safety constraints into the policy learning objective, typically through constrained Markov decision processes or Lyapunov-based methods. These approaches constrain policy behavior in expectation over the training distribution and do not directly prevent transient or uncertainty-induced norm violations at deployment time. A particularly relevant subfield is shielded reinforcement learning [15], in which an external runtime filter—the shield— restricts agent actions or corrects undesired ones to enforce formal safety specifications, typically expressed in temporal logic. Shielding has since been extended to dynamic temporallogic constraints [16] and to multi-agent settings with modelbased dynamic shielding (MBDS) [17], which synthesizes shields online to mitigate the conservativeness of static multiagent shields. Mechanical conscience shares the supervisoryfilter philosophy of shielding but differs along three axes that are essential for DCI deployment: 1) Shields enforce hard binary constraints derived from formal specifications; mechanical conscience produces continuous, graduated corrections via the deviation functional Ψ, accommodating soft norms (fairness, privacy, social acceptability) that resist hard logical specification. 2) Shields require formal synthesis from specification and a model of the environment; mechanical conscience requires only differentiable satisfaction functions and is therefore architecture- and policy-agnostic. 3) Shields are typically defined for a single normative dimension (safety); mechanical conscience aggregates a heterogeneous normative space N that natively spans safety, security, fairness, regulatory compliance, and other domains.
3
Recent safe-MARL methods such as MADAC [18], which enforces state-wise constraints with feasibility guarantees, and entropic-exploration coordination [19], which treats safety as a team-level concept, are directly relevant to the multi-agent extension in Section VI and are natural candidates for empirical comparison in future work.
continuous control is, to our knowledge, not provided by existing deontic frameworks. F. AI Alignment and Value Specification
AI alignment [29], [30] research investigates how to ensure that AI systems pursue objectives consistent with human D. Ethical Governor and Consequence-Engine Architectures values. Contemporary alignment methods range from reward modeling and inverse reinforcement learning to constitutional The closest conceptual antecedents of mechanical conscience AI and reinforcement learning from human feedback. While are the ethical governor and consequence-engine architectures these methods address what objectives a system should hold, developed in the machine ethics community. they do not provide a runtime operational model for how Arkin’s ethical governor [20] introduced an architectural those objectives are continuously enforced at the trajectory component that suppresses, restricts, or transforms the lethal level under uncertainty and in interaction with other agents. output of an autonomous system so that it falls within a Mechanical conscience is intended to serve as precisely permissible behavior set defined by laws of war and rules of enthis operational layer: the mechanism by which alignment gagement. Winfield et al. [21] proposed an internal-model-based objectives, once specified, are enacted in real time within a consequence engine that simulates the predicted outcomes of DCI environment. candidate actions and filters them through a safety/ethical logic. Dennis et al. [22] subsequently provided formal verification techniques for the ethical decision-making of such governors. G. Dependability and Trust in Distributed Systems Most recently, neuro-symbolic ethical governors [23] and Classical dependability theory [31] characterizes system the GRACE architecture [24] integrate learning-based risk- trustworthiness through attributes such as reliability, availability, assessment components with symbolic obligation reasoning, and safety. These attributes are defined for systems in isolation separating normative reasoning from instrumental optimization and are evaluated against static specifications. They do not to enable interpretability and contestability. capture the relational and trajectory-aware nature of trust in Mechanical conscience extends this lineage in three substan- human–machine DCI settings, where dependability is not a tive ways. First, governor and consequence-engine architectures static property of one system but an emergent property of evaluate acceptability of individual actions or short-horizon sustained interaction between human and machine intelligence. plans; mechanical conscience operates over behavioral trajec- The concept of resonant dependability introduced in this tories and introduces an explicit cumulative deviation signal paper extends classical dependability theory to this relational, (mechanical guilt GT ) that is auditable in real time. Second, trajectory-level setting, with mechanical conscience as its existing governors are typically defined for single agents or operational realization. small teams; mechanical conscience is formulated from the outset for distributed P multi-agent P DCI through the additive decomposition Ψtot = k Ψ(k) + i<j Ψij + Ψglobal , which H. Positioning of Mechanical Conscience captures interaction-induced emergent risk explicitly. Third, Table I summarizes the positioning of mechanical conscience governors and consequence engines focus on the supervisory relative to the principal threads surveyed above. Mechanical mechanism itself; mechanical conscience pairs that mechanism conscience is distinguished by the simultaneous combination with the systems-level concept of resonant dependability, of (i) trajectory-level rather than point-wise evaluation, (ii) articulating the long-term trust property that the supervisory a continuous, differentiable, multi-norm deviation functional, layer is intended to produce. (iii) a native multi-agent decomposition capturing interactionE. Deontic Logic and Normative Multi-Agent Systems Deontic logic and normative multi-agent systems [25], [26] provide a formal foundation for reasoning about obligations, permissions, and prohibitions, with recent extensions including defeasible deontic calculi [27] for resolving norm conflicts and deontic temporal logics [28] for formal verification of AI ethical properties. These approaches are highly expressive but operate at the symbolic level and produce decisions of the form “permitted/forbidden/obligatory.” Mechanical conscience can be viewed as a quantitative companion to deontic reasoning: each norm Ni ∈ N may be derived from a deontic specification, but the satisfaction function ϕi converts that specification into a real-valued degree of compliance, enabling continuous regulation through Ψ and integration into gradient-based control loops. This quantitative bridge between symbolic norms and
induced emergent risk, and (iv) integration with the systemslevel concept of resonant dependability. III. S YSTEM AND N ORMATIVE F ORMULATION A. Dynamical System Model We model an intelligent agent—or a DCI system abstracted as a single decision-making entity—as a discrete-time stochastic dynamical system: xt+1 = f (xt , ut , wt ),
(1)
where xt ∈ X is the system state at time t, ut ∈ U is the control action, and wt ∈ W captures environmental disturbances and model uncertainty. The system produces observations yt = h(xt , vt ),
(2)
4
TABLE I P OSITIONING OF MECHANICAL CONSCIENCE (MC) RELATIVE TO RELATED THREADS . ✓: ADDRESSED ; –: NOT ADDRESSED OR OUT OF SCOPE ; △: PARTIALLY ADDRESSED . Approach Constrained / CBF control Safe RL (CMDP) Shielding (LTL/STL) Ethical governor / consequence engine Deontic / normative MAS AI alignment Mechanical conscience (this work)
Trajectory-level evaluation
Continuous regulation
Multi-norm aggregation
Multi-agent native
– △ ✓ △ △ △ ✓
✓ – – △ – – ✓
– – – △ ✓ △ ✓
△ – △ – ✓ – ✓
where vt is sensing noise. A baseline policy πθ maps the estimated state x̂t to a candidate action: ubase = πθ (x̂t ). t
(3)
The baseline policy may be a learned policy (e.g., from federated or transfer learning), a hand-crafted controller, or any combination. Mechanical conscience operates as a post-hoc supervisory layer that does not require access to the internals of πθ . B. Normative Evaluation Space
which is zero if and only if (xt , ut ) ∈ Ct , and equals the weighted sum of violation magnitudes otherwise. The linear form (exponent p = 1) is intentionally simple: it is differentiable almost everywhere, convex, and interpretable as a violation cost that scales proportionally with severity. IV. T HE M ECHANICAL C ONSCIENCE F RAMEWORK A. Mechanical Conscience as a Supervisory Filter Mechanical conscience is realized as a supervisory filter sitting between the baseline policy and the actuators. Given a candidate action ubase = πθ (x̂t ) produced by the baseline t policy, the MC layer computes the minimally corrected action h i u∗t = arg min ∥u − ubase ∥2 + η Ψ(x̂t , u) , (8) t
We introduce a normative evaluation space N = {N1 , N2 , . . . , Nm }, where each Ni represents a normative category relevant to the deployment context—safety, security, u∈U privacy, fairness, mission compliance, or regulatory requirement. where η > 0 controls the strength of normative correction. The Each norm induces a real-valued satisfaction function: quadratic term ∥u − ubase ∥2 enforces minimal intervention: t ϕi (xt , ut , ξt ) ∈ R, (4) the MC layer deviates from the original intent only as much as is necessary to restore normative acceptability. This is where ξt encodes contextual variables such as environmental the operational core of mechanical conscience—a lightweight, state, social context, or applicable regulations. The sign architecture-neutral post-processing step that can be inserted afconvention is simple: ϕi ≥ 0 means norm Ni is satisfied; ter any baseline policy without modifying the policy itself. The ϕi < 0 means it is violated with severity |ϕi |. formulation is deliberately simple: Ψ provides a smooth scalar The conscience-feasible set—the normative admissible penalty, the quadratic deviation term is strictly convex, and region—at time t is: the resulting optimization can be solved efficiently by standard Ct = (xt , ut ) : ϕi (xt , ut , ξt ) ≥ 0, ∀i . (5) quadratic programming or gradient-based methods. When all ϕi are linear in u, the problem reduces to a quadratic program with This set captures not merely what is operationally possible, but linear constraints; when ϕi are nonlinear, projected gradient what remains acceptable under the active normative regime. descent suffices in practice. C. Conscience Score and Normative Deviation A scalar conscience score summarizing overall normative acceptability is: Γt =
m X
αi ϕi (xt , ut , ξt ),
(6)
i=1
where αi ≥ 0 is the importance weight of norm Ni . High Γt indicates a morally or operationally acceptable state-action pair; low Γt signals a conscience alarm. The normative deviation functional isolates the cost of violations: m X Ψ(xt , ut ) = αi max 0, −ϕi (xt , ut , ξt ) , (7) i=1
B. Trajectory Conscience Functional A point-in-time conscience score is insufficient; true conscience evaluates the projected consequences of current actions over a future horizon. We define the mechanical conscience functional as the expected cumulative trade-off between task utility and normative cost: "H # X MC k J =E γ R(xt+k , ut+k ) − β Ψ(xt+k , ut+k ) , k=0
(9) where R(xt , ut ) ≥ 0 is the operational task reward, Ψ is the normative deviation functional (Eq. 7), β > 0 is the conscience strength that governs the normative-utility tradeoff, γ ∈ (0, 1] is a temporal discount factor, and H is
5
the planning horizon. Maximizing J MC yields trajectories that prefer useful actions but penalize downstream normative deviation. The supervisory filter (8) is the per-step instantiation of this objective: with H = 0 and η = β, minimizing the negative of J MC plus the minimal-intervention regularizer recovers (8). For H > 0, mechanical conscience becomes a horizon-aware regulator naturally compatible with model predictive control architectures. C. Uncertainty-Aware Conscience
direct: under any β-regulated policy, E[GT ] is the running cost component of −J MC , so improving the conscience functional is equivalent to reducing expected mechanical guilt while preserving task utility. V. T HEORETICAL P ROPERTIES We establish the foundational mathematical properties of the mechanical conscience framework. The three principal propositions characterize faithfulness, well-posedness, and tunability of MC regulation in the single-agent setting; we then state corollaries that extend these properties to the uncertaintyaware variant (Ψuc , Eq. 11) and to the distributed multiagent setting (Section VI). All proofs follow directly from the definitions in Sections III and IV.
A realistic conscience must account for epistemic uncertainty. In DCI environments, normative classification is often uncertain: the applicable norm, the context ξt , or the state estimate x̂t may be unreliable. Let p(xt | y1:t ) be the state belief distribution maintained by the agent, and let p(c | x̂t , ut ) Proposition 1 (Admissibility Equivalence). The normative denote a posterior over normative classes c (e.g., admissible deviation functional satisfies / inadmissible) produced by a normative classifier operating Ψ(xt , ut ) = 0 ⇐⇒ (xt , ut ) ∈ Ct . (13) on the estimated state-action pair. We define the uncertainty Proof. By Eq. 7, Ψ(xt , ut ) = severity as Pm α max 0, −ϕ (x , u , ξ ) . Since α > 0 for all Ωt = 1 − max p(c | x̂t , ut ), (10) i t t t i i=1 i c i and each summand is non-negative, Ψ = 0 holds if and which lies in [0, 1] and is high when the system cannot only if max 0, −ϕi = 0 for every i, which is equivalent confidently classify the current state-action pair as norma- to ϕi (xt , ut , ξt ) ≥ 0 for all i. This is precisely the defining tively acceptable. An equivalent variance-based form Ωt = condition of Ct in Eq. 5. □ Var[Ψ(xt , ut )] is appropriate when Ψ is treated as a random Proposition 2 (Existence of Optimal Regulation). If U is functional under the state belief. compact and each ϕi is continuous in u, then the supervisory The full uncertainty-augmented conscience penalty is then optimization problem (8) admits a minimizer u∗t ∈ U. uc Ψ (xt , ut ) = Ψ(xt , ut ) + ρ Ωt , (11) Proof. Since each ϕi is continuous in u and the maps z 7→ max(0, −z) and finite addition preserve continuity, Ψ(x̂t , ·) is with ρ ≥ 0 controlling the degree of uncertainty aversion. continuous on U. The Euclidean norm ∥u − ubase ∥2 is also t Substituting Ψuc for Ψ in either (8) or (9) yields the uncertainty- continuous in u. The objective in (8) is therefore a continuous aware variants of the supervisory filter and the trajectory function on the compact set U, and by the extreme value conscience functional. This substitution ensures the system theorem it attains its infimum at some u∗ ∈ U. □ t becomes more conservative when it is uncertain—a property Remark. The continuity assumption on ϕ is mild: it is satisfied i consistent with how human conscience operates under moral by all common normative constructions used in practice, inambiguity, and particularly important in DCI deployments cluding barrier- function-style constraints [9], smooth distancewhere uncertainty is heterogeneous across participants and based norms, and bounded outputs of differentiable normative changes dynamically. classifiers. When all ϕi are linear in u, the optimization in (8) becomes a strictly convex quadratic program with a unique solution. D. Conscience Deviation and Mechanical Guilt To support interpretability and runtime monitoring, we introduce two cumulative quantities that summarize the normative behavior of the system over time. The conscience deviation at time t is the instantaneous normative violation cost, Dt = Ψ(xt , ut ), equivalently Ψuc (xt , ut ) when the uncertaintyaware form is used. Its discounted accumulation over a horizon T —termed mechanical guilt—is GT =
T X
γ t Dt .
(12)
t=0
Proposition 3 (Monotonic Regulation). Let πβ∗ denote the optimal policy maximizing the mechanical conscience functional (9) for a given conscience strength β ≥ 0, and let GβT denote the expected mechanical guilt accumulated under πβ∗ over horizon T . Then β 7→ E[GβT ] is non-increasing. Proof. Fix 0 ≤ β1 ≤ β2 . By optimality of πβ∗1 under J MC with conscience strength β1 , Adding (14) and (15) and simplifying yields " # " #! X X k k (β2 − β1 ) Eπβ∗ γ Ψk − Eπβ∗ γ Ψk ≥ 0. 1
A system with healthy mechanical conscience minimizes GT , not merely maximizing raw task reward. Mechanical guilt is computable in real time, requires no additional model beyond Ψ, and is directly auditable by human overseers, making accumulated normative stress a practical governance signal for DCI deployments. The relationship between J MC and GT is
2
k
k
(16) P Since β2 − β1 ≥ 0 and E k γ k Ψk = E[GT ], we obtain E[GβT1 ] ≥ E[GβT2 ]. □ Interpretation. Together, Propositions 1–3 establish that mechanical conscience is faithful (Prop. 1: zero deviation exactly
6
"H X
Eπβ∗
1
"H X
# k
γ Rk − β1 Eπβ∗
1
k=0
# k
γ Ψk
≥ Eπβ∗
"H X
2
k=0
"H X
# k
γ Rk − β1 Eπβ∗
2
k=0
# k
γ Ψk ,
(14)
k=0
where for brevity Rk = R(xt+k , ut+k ) and Ψk = Ψ(xt+k , ut+k ) under the indicated policy. Symmetrically, by optimality of πβ∗2 under conscience strength β2 , # " # " # " # " X X X X k k k k Eπβ∗ γ Rk − β2 Eπβ∗ γ Ψk ≥ Eπβ∗ γ Rk − β2 Eπβ∗ γ Ψk . (15) 2
2
k
1
k
1
k
k
characterizes normative admissibility, with no spurious zero VI. D ISTRIBUTED E XTENSION AND R ESONANT modes), well-posed (Prop. 2: a corrected action always exists D EPENDABILITY under mild regularity), and tunable (Prop. 3: enforcement A. Multi-Agent Mechanical Conscience strength is monotonically controllable through a single interIn a DCI system of n agents, the joint state is xt = pretable scalar β). These properties are what justify deploying (1) (n) (1) (n) (xt , . . . , xt ) and the joint action is ut = (ut , . . . , ut ), mechanical conscience as a governance layer with auditable with global dynamics xt+1 = F (xt , ut , wt ). We distinguish behavior. (k) three levels of normative constraint. Individual norms ϕi govern each agent’s own behavior. Pairwise norms ϕij constrain interactions between pairs of agents (e.g., collision avoidance, A. Extensions: Uncertainty and Multi-Agent Settings mutual privacy). Collective norms ϕglobal govern emergent The same theoretical structure carries over, with mild system-level properties that no individual agent controls modifications, to the uncertainty-aware penalty Ψuc and to directly. the distributed multi-agent cost Ψtot . The distributed mechanical conscience cost aggregates all three levels: Corollary 1 (Uncertainty-Aware Equivalence). n X X (k) (k) Ψuc (xt , ut ) = 0 ⇐⇒ (xt , ut ) ∈ Ct and Ωt = 0. Ψtot (xt , ut ) = Ψ(k) (xt , ut )+ Ψij (xt , ut )+Ψglobal (xt , ut ). Proof. Ψuc = Ψ + ρ Ωt with Ψ ≥ 0, Ωt ≥ 0, and ρ > 0. The i<j k=1 (17) sum vanishes iff each summand vanishes; Ψ = 0 characterizes Ct by Prop. 1, and Ωt = 0 characterizes confident normative Each term is computed using the same deviation functional classification by Eq. 10. □ (Eq. 7) applied to the relevant constraint functions. Crucially, Interpretation. Corollary 1 makes precise the intuitive claim that many DCI failures arise not from individual misconduct but an uncertainty-aware MC layer treats confident admissibility from interaction-induced emergence—the pairwise and global as the zero-cost configuration: even if the system is operating terms in (17) are the formal handles for detecting and penalizing in the nominally admissible region, residual uncertainty incurs such emergent normative violations. penalty until classification confidence is restored. This is the formal handle for the conservative behavior described qualitatively in Section IV. Corollary 2 (Multi-Agent Admissibility). For the distributed conscience cost Ψtot in Eq. 17, Ψtot (xt , ut ) = 0 if and only if every individual, pairwise, and global norm is satisfied simultaneously. Proof. Ψtot is a non-negative sum of three non-negative components. Each component vanishes iff its constituent norms are satisfied (by Prop. 1 applied at the respective level). Hence Ψtot = 0 iff all three layers of normative constraint are simultaneously satisfied. □ Interpretation. Corollary 2 formalizes the central claim of the multi-agent extension: emergent risk in DCI is captured precisely when individual compliance is necessary but not sufficient. A multi-agent system can have every individual term Ψ(k) = 0 while Ψij or Ψglobal remain strictly positive—this is the mathematical signature of interaction-induced emergent risk that motivates Section VI. Propositions 2 and 3 also extend to the multi-agent setting under analogous compactness and continuity assumptions on the joint action space, yielding wellposed and monotonically tunable distributed regulation.
B. Resonant Dependability We introduce resonant dependability as the central normative goal of the MC framework. Resonant dependability describes the emergent, bidirectional trust that arises when human and machine intelligence operate within mutually acknowledged normative trajectories over sustained time. The qualifier resonant is deliberate. Classical dependability theory [31] treats reliability, availability, and safety as properties of a system evaluated against static, externally specified requirements. Resonant dependability is different in three ways: • Relational: it is a property of the interaction between human and machine intelligence, not of either in isolation. A machine that is technically reliable but whose behavior is normatively opaque or unpredictable to its human principals does not exhibit resonant dependability. • Trajectory-aware: it requires sustained normative alignment across time. A single correct decision does not establish resonant dependability; a continuous trajectory of MC-regulated behavior does. • Emergent: it cannot be engineered by a single design decision but arises from the accumulation of mechanical conscience regulation over many interactions.
7
State dim. 2
Normalized value
1 Baseline C: Normative Admissible Region Drift
MC-regulated
Task reward Normative deviation
0.5
0 0
Correction
1
2 3 Conscience strength β
4
Fig. 2. Trade-off between task reward and normative deviation as a function of conscience strength β. The balanced operating regime (β ≈ 0.8–2.0) achieves
State dim. 1 substantial deviation reduction with moderate performance cost, revealing the interpretable conscience operating point. Fig. 1. Trajectory-level regulation. A baseline controller drifts outside the normative admissible region C; the MC layer continuously redirects the trajectory toward acceptable evolution.
Formally, let Bt denote the observable behavior of the DCI system and let Btvalid be the normatively acceptable behavior space acknowledged by the human principal at time t. The behavioral deviation is: ∆t = d Bt , Btvalid , (18) where d(·, ·) is an appropriate behavioral distance. The MC layer enforces correction whenever ∆t > τ for a tolerance threshold τ . Resonant dependability is attained in the limit: E[GT ] → 0
as T → ∞,
(19)
establishing a trajectory-level validation of the human–machine relationship in which the accumulated mechanical guilt vanishes over sustained operation. VII. I LLUSTRATIVE E VALUATION A. Single-Agent Trajectory Regulation Consider a discrete-time linear system A ∈ R2×2 , B ∈ R2×1 , with an admissible region defined by ϕ1 (x) = r2 − ∥x∥2 ≥ 0 and a control bound ϕ2 (u) = umax − |u| ≥ 0. Fig. 1 shows that without MC, trajectory drift eventually exits C, whereas the MC-regulated agent maintains admissibility throughout. The quadratic cost in (8) ensures the correction is minimal—task performance is preserved where possible.
a) Setup.: We instantiate n = 4 agents on a shared 2D workspace, each pursuing a private goal location selected uniformly at random in [−1, 1]2 . Each agent runs a baseline (k) (k) policy ut = −K(xt − g (k) ) that drives it toward its goal, producing trajectory crossings that violate a pairwise proximity norm ϕij (x) = ∥x(i) − x(j) ∥2 − d2min with dmin = 0.15. We compare four regulators: (i) the unregulated Baseline, (ii) Individual MC, which applies the per-agent supervisory filter using only individual norms, (iii) Pairwise MC, which augments the cost with the pairwise terms Ψij , and (iv) Full MC, which uses the complete decomposition Ψtot in Eq. 17 including the collective term Ψglobal that penalizes coordinated deadlock. We report the near-collision rate (fraction of timesteps violating ϕij ≥ 0), task completion rate, and average mechanical guilt Ḡ over 200 episodes. TABLE II M ULTI - AGENT INTERACTION RISK : FOUR REGULATOR VARIANTS ON THE n=4 SHARED - WORKSPACE TASK . L OWER IS BETTER FOR NEAR - COLLISION RATE AND MECHANICAL GUILT; HIGHER IS BETTER FOR TASK COMPLETION . S TANDARD ERRORS OVER 200 EPISODES ARE BELOW 1% FOR RATE METRICS AND BELOW 5% RELATIVE FOR Ḡ. Regulator Baseline (no MC) Individual MC Pairwise MC Full MC (Ψtot )
Near-collision rate (%)
Task completion rate (%)
Mechanical guilt Ḡ (norm.)
18.7 17.9 3.1 2.4
96.4 95.8 92.2 91.5
1.00 0.92 0.18 0.13
b) Results.: Table II summarizes the comparison. The unregulated baseline produces near-collisions in roughly 19% Fig. 2 illustrates Proposition 3 empirically. As β increases, of time-steps—a clear instance of interaction-induced emergent cumulative normative deviation decreases monotonically while risk despite each agent’s per-step decisions being locally task reward degrades gracefully. Three regimes are identifiable: rational with respect to its own goal. Individual MC, which under-regulated (β < 0.8), balanced (0.8 ≤ β ≤ 2.0), and lacks visibility into pairwise norms, reduces this rate only over-conservative (β > 2.0). Selecting β within the balanced marginally to 17.9%, confirming that individual-level reguregime constitutes the practical design problem for deploying lation alone is insufficient when emergent risk arises from mechanical conscience. interactions rather than from individual misconduct (consistent with Corollary 2). Adding pairwise terms reduces the rate to C. Multi-Agent Interaction Risk 3.1%, an improvement of more than 6× over the baseline, We next demonstrate the framework’s ability to suppress with only a 4-point reduction in task completion rate. The interaction-induced emergent risk in a multi-agent DCI setting. full Ψtot formulation, which additionally penalizes collective B. Trade-off Analysis: Conscience Strength
8
deadlock through Ψglobal , drives the rate down to 2.4% while keeping mechanical guilt at 13% of the baseline level. The graceful degradation of task performance across these regulators reflects the minimal-intervention property of the supervisory filter: corrections are applied only to the extent necessary to restore admissibility, and only in the dimensions that the active normative regime requires. VIII. D ISCUSSION
can be updated at runtime as context or regulations change, without retraining the underlying policy. This makes mechanical conscience particularly suitable as a substrate for emerging governance frameworks (such as the EU AI Act and analogous regulatory regimes), in which auditability, contestability, and runtime adaptability of safety mechanisms are first-class requirements. D. Limitations and Future Work
A. Novelty of Terminology
Several limitations of the current formulation point to natural directions for further research. The terms introduced in this paper—mechanical conscience, a) Norm specification and learning.: The framework conscience score, conscience deviation, mechanical guilt, and assumes that normative constraint functions ϕi can be specified resonant dependability—constitute a new conceptual vocabin advance. In practice, norms may be implicit, contested, ulary for the emerging field of normative AI. Each term or dynamically evolving—a particularly acute challenge in corresponds to a distinct mathematical object or property within federated or swarm DCI settings where participants hold the framework: the conscience score Γt (Eq. 6) aggregates norheterogeneous normative expectations. Integration with norm mative satisfaction; the deviation functional Ψ (Eq. 7) quantifies learning from human feedback, inverse constraint learning, and violation magnitude; mechanical guilt GT (Eq. 12) accumulates large language models acting as runtime normative classifiers trajectory-level normative stress; and resonant dependability are promising directions, with the satisfaction functions ϕi (Eq. 19) formalizes the goal of sustained normative alignment. providing a clean interface between learned components and Establishing this vocabulary in close correspondence with mathematical structure is essential for coherent discourse across the regulatory loop. b) Distributed coordination of mechanical conscience.: the AI safety, control theory, and human–machine interaction Section VI P formalizes the additive decomposition Ψtot = communities, which currently use overlapping but disjoint P (k) Ψ + terminologies for closely related phenomena. k i<j Ψij + Ψglobal , but does not specify how individual agents should negotiate or share responsibility for pairwise and global terms. A communication-efficient B. Design Principles and Extensibility distributed protocol for coordinating Ψtot across heterogeThe mathematical model presented in this paper has been neous agents—compatible with federated learning’s privacy kept deliberately parsimonious, but each component is designed constraints—is an important open problem. as an extensible primitive rather than a closed specification. c) Empirical validation at scale.: The illustrative evalThe linear deviation functional in Eq. 7 can be replaced by uation in Section VII demonstrates the qualitative behavior P a higher-order penalty i αi max(0, −ϕi )p with p > 1 when of mechanical conscience but does not establish quantitative applications require stronger sensitivity to large violations; competitiveness against state-of-the-art shielded RL or safe convexity is preserved for any p ≥ 1. The uncertainty term MARL baselines. Large-scale empirical evaluation on physical Ωt in Eq. 10 can be elaborated into a full Bayesian filter or robotic and networked agent platforms, with direct comparison replaced by ensemble-based or conformal prediction estimates to MADAC, model-based dynamic shielding, and ethicalwithout altering the rest of the framework. The single-step governor architectures, is the next methodological priority. supervisory filter in Eq. 8 can be generalized to a horizond) Resonant dependability as a convergence theory.: H model predictive control formulation that optimizes J MC The condition E[GT ] → 0 characterizes resonant dependability directly, providing predictive trajectory-level correction at the asymptotically but does not yet provide rate-of-convergence cost of additional computation. At each level of elaboration, the guarantees or robustness conditions under adversarial behavior three theoretical properties of Section V continue to hold under of subsets of agents. A systems-theoretic treatment that places mild regularity conditions, so the framework’s interpretability resonant dependability in the same mathematical register as and tunability are preserved as it scales to more demanding classical dependability attributes (reliability functions, MTTF, deployments. hazard rates) is a natural next step. C. Relation to AI Safety and Alignment Mechanical conscience provides an operational bridge between high-level alignment objectives and runtime system behavior. Where alignment research specifies what a system should value, MC specifies how those values are continuously enforced at the trajectory level under uncertainty. The conscience strength β and the normative evaluation space N together provide interpretable, auditable levers for governance of DCI systems: β controls the normative-utility trade-off, and N encodes the applicable normative regime. Both parameters
IX. C ONCLUSION This paper has introduced mechanical conscience—a formal mathematical framework for trajectory-level normative regulation of intelligent systems, motivated by the structural challenges of distributed collaborative intelligence. In DCI environments—spanning edge-to-edge architectures, federated learning, transfer learning, and swarm systems—emergent risk arises inevitably from the composition of locally correct decisions across interacting agents under uncertainty. Mechanical conscience addresses this challenge by providing
9
a lightweight supervisory filter that continuously minimizes cumulative normative deviation without requiring access to the internals of the underlying policy. The framework rests on a small set of mathematical constructs—the conscience-feasible set C, the conscience score Γt , the normative deviation functional Ψ, the uncertaintyaugmented penalty Ψuc , and mechanical guilt GT —each with a clear operational interpretation and a direct correspondence to a measurable runtime quantity. We have established three foundational theoretical properties (admissibility equivalence, existence of optimal regulation, and monotonic deviation reduction), extended the framework to Pmulti-agent P DCI through the additive decomposition Ψtot = k Ψ(k) + i<j Ψij +Ψglobal , and connected the resulting machinery to the systems-level concept of resonant dependability—the emergent, bidirectional trust between human and machine intelligence that accumulates through sustained MC-regulated trajectory alignment. By positioning mechanical conscience explicitly against the principal research threads it intersects—constrained and shielded reinforcement learning, ethical-governor and consequence-engine architectures, deontic and normative multiagent systems, AI alignment, and classical dependability theory—we have argued that the simultaneous combination of trajectory-level evaluation, continuous and differentiable regulation, multi-norm aggregation, and native multi-agent decomposition is what distinguishes the framework and makes it suited to the DCI deployments that motivate it. We see mechanical conscience and resonant dependability as foundational constructs for a broader research agenda: a quantitative, runtime-anchored account of how human and machine intelligence can sustain mutually acknowledged normative trajectories at scale. Realizing this agenda will require continued work on norm learning, distributed coordination of the conscience layer, large-scale empirical validation, and a convergence theory for resonant dependability under adversarial conditions. The mathematical structures developed here are intended as the foundation on which that broader investigation can proceed. R EFERENCES [1] W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016. [2] M. Satyanarayanan, “The emergence of edge computing,” Computer, vol. 50, no. 1, pp. 30–39, 2017. [3] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), ser. Proceedings of Machine Learning Research, vol. 54. PMLR, 2017, pp. 1273–1282. [4] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020. [5] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2010. [6] M. Dorigo, G. Theraulaz, and V. Trianni, “Swarm robotics: Past, present, and future,” Proceedings of the IEEE, vol. 109, no. 7, pp. 1152–1165, 2021. [7] Y. Zhang, D. Lu, J. Wang, X. Zhang, and H. Yu, “A survey of trustworthy federated learning: Issues, solutions, and challenges,” ACM Transactions on Intelligent Systems and Technology, vol. 15, no. 4, pp. 1–47, 2024.
[8] Y. Liu, Y. Wu, L. Liu, and H. Yu, “Trustworthy distributed AI systems: Robustness, privacy, and governance,” ACM Computing Surveys, vol. 57, no. 6, pp. 1–42, 2024. [9] A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in 2019 18th European Control Conference (ECC), 2019, pp. 3420–3431. [10] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, UK: Cambridge University Press, 2004. [11] S. Prajna and A. Jadbabaie, “Safety verification of hybrid systems using barrier certificates,” in International Workshop on Hybrid Systems: Computation and Control (HSCC), ser. Lecture Notes in Computer Science, vol. 2993. Springer, 2004, pp. 477–492. [12] J. Garcı́a and F. Fernández, “A comprehensive survey on safe reinforcement learning,” Journal of Machine Learning Research, vol. 16, no. 1, pp. 1437–1480, 2015. [Online]. Available: https: //jmlr.org/papers/v16/garcia15a.html [13] E. Altman, Constrained Markov Decision Processes. Boca Raton, FL: Chapman & Hall/CRC, 1999. [14] F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe modelbased reinforcement learning with stability guarantees,” in Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., 2017, pp. 908–918. [Online]. Available: https://proceedings.neurips.cc/ paper/2017/hash/766ebcd59621e305170616ba3d3dac32-Abstract.html [15] M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018. [16] S. B. Yüksel, A. T. Buyukkocak, and D. Aksaray, “Shielded reinforcement learning under dynamic temporal logic constraints,” arXiv preprint arXiv:2603.17152, 2026. [17] W. Xiao, Y. Lyu, and J. Dolan, “Model-based dynamic shielding for safe and efficient multi-agent reinforcement learning,” in Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2023, pp. 1587–1596. [18] Z. Li and N. Azizan, “Safe multi-agent reinforcement learning with convergence to generalized Nash equilibrium,” arXiv preprint arXiv:2411.15036, 2024. [19] A. A. Aydeniz, E. Marchesini, R. Loftin, C. Amato, and K. Tumer, “Safe multiagent coordination via entropic exploration,” arXiv preprint arXiv:2412.20361, 2024. [20] R. C. Arkin, P. Ulam, and B. Duncan, “An ethical governor for constraining lethal action in an autonomous system,” in Technical Report GIT-GVU-09-02, Georgia Institute of Technology, 2009. [21] A. F. T. Winfield, C. Blum, and W. Liu, “Towards an ethical robot: Internal models, consequences and ethical action selection,” Lecture Notes in Computer Science (Advances in Autonomous Robotics Systems), vol. 8717, pp. 85–96, 2014. [22] L. A. Dennis, M. Fisher, M. Slavkovik, and M. Webster, “Towards verifiably ethical robot behaviour,” in Proceedings of the AAAI Workshop on AI and Ethics, 2015. [23] A. Aueawatthanaphisut, “A real-time neuro-symbolic ethical governor for safe decision control in autonomous robotic manipulation,” arXiv preprint arXiv:2603.14221, 2026. [24] F. Jahn, Y. Muskalla, L. Dargasz, P. Schramowski, and K. Baum, “Breaking up with normatively monolithic agency with GRACE: A reasonbased neuro-symbolic architecture for safe and ethical AI alignment,” arXiv preprint arXiv:2601.10520, 2026. [25] G. Boella, L. van der Torre, and H. Verhagen, “Introduction to the special issue on normative multiagent systems,” Autonomous Agents and Multi-Agent Systems, vol. 17, no. 1, pp. 1–10, 2008. [26] U. Furbach, C. Schon, and F. Stolzenburg, “Automated reasoning for robot ethics,” in Advances in Artificial Intelligence and Its Applications. Springer, 2015, pp. 53–68. [27] T. Olson, R. Salas-Damian, and K. D. Forbus, “A defeasible deontic calculus for resolving norm conflicts,” arXiv preprint arXiv:2407.04869, 2024. [28] T. V. Priya and S. Rao, “Deontic temporal logic for formal verification of AI ethics,” arXiv preprint arXiv:2501.05765, 2025. [29] S. Russell, Human Compatible: Artificial Intelligence and the Problem of Control. Penguin Publishing Group, 2019. [Online]. Available: https://books.google.co.kr/books?id=M1eFDwAAQBAJ [30] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané, “Concrete problems in AI safety,” arXiv preprint arXiv:1606.06565, 2016. [31] J.-C. Laprie, Dependability: Basic Concepts and Terminology: In English, French, German, Italian and Japanese, ser. Dependable Computing and Fault-Tolerant Systems. Vienna, Austria: Springer-Verlag, 1992, vol. 5.