arXiv:2606.17223v1 [cs.CR] 15 Jun 2026
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI * Manoj Parmar SovereignAI Security Labs Bengaluru, India [email protected] June 17, 2026
Abstract Neuro-symbolic AI (NeSy) pairs neural perception with symbolic reasoning, making it attractive for highstakes domains where explainability and structured inference are required. However, this hybrid architecture introduces an enlarged attack surface spanning five layers: neural perception, symbolic knowledge bases, reasoning engines, agentic orchestration, and data stores—each exploitable in ways absent from purely neural systems. This paper makes six contributions: (1) formal definitions of NeSy Attack Surface, Symbolic Integrity Violation (SIV), and Cross-Layer Amplification Ratio X , decomposed into neural-caused and adversariallyinduced autonomous symbolic sensitivity components; (2) a unified threat model extending MITRE ATLAS with 11 NeSy-specific tactic extensions and a five-profile attacker taxonomy; (3) a symbolic-layer threat catalogue covering KG poisoning, ontology-merging, and inference-engine subversion; (4) analysis of cognitive risks— automation bias, authority bias, sycophantic reinforcement—structurally amplified by NeSy’s explicit logical explanations relative to black-box neural outputs; (5) interdisciplinary mitigations with measurable acceptance criteria aligned to NIST AI 600-1 and the EU AI Act; (6) three empirical benchmarks: (E1) targeted KG poisoning achieves break-even SIV at budget B = 5 on a 205-entity medical KG, with a KG-specific stealth/SIV trade-off; (E2) PGD-10 at ε = 0.01 yields X = 5.884 (95% CI [4.64, 8.00], p < 0.0001)—confirmed adversarially specific R = 0)—on a DistilBERT + ProbLog pipeline; (E3) single-axiom OWL edits by a matched-random baseline (Erand achieve 93.3% SIV success (equivalence/subclass templates) with 100% stealth, but held-out detection is 67.9% overall with STIX detector failure at 50% (random-guessing level), an open problem.
Keywords: neuro-symbolic AI, AI safety, adversarial machine learning, MITRE ATLAS, OWASP LLM, knowledge graph security, cognitive security, dual-process theory, agentic AI
Contents
1 2
Introduction
4
Related Work
5
* Citation: Parmar, M. Safety, Security, and Cognitive Risks in Neuro-Symbolic AI. arXiv preprint, 2026.
1
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
3
4
Background
6
3.1
Neuro-Symbolic AI: Core Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
3.2
Trustworthy Neuro-Symbolic AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
NeSy Architecture and Asset Inventory
7
5 Threat-Modelling Methodology
7
5.1
Formal Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
5.2
MITRE ATLAS Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
5.3
OWASP LLM Top 10 Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
5.4
Analytical Lenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
5.5
NeSy Attacker Capability Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
6 Technical Threat Categories
7
8
9
12
6.1
Data and Knowledge Poisoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
6.2
Model and Reasoning Exploitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
6.3
Orchestration and Agentic Workflow Threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
6.4
Privacy and Confidentiality Risks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
Cognitive, Psychological, and Behavioural Risks
13
7.1
Dual-Process Alignment and Over-Trust . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
7.2
Cognitive Security and Narrative Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
7.3
Cognitive AI Safety for Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
Example Threat Scenarios
15
8.1
Healthcare Decision Support with Poisoned Knowledge . . . . . . . . . . . . . . . . . . . . . . . . .
15
8.2
Cybersecurity SOC Assistant Turned Unreliable . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
8.3
Agentic Workflow in Enterprise Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
8.4
Psychologically Unsafe Mental-Health Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
Protection Mechanisms and Design Patterns
16
9.1
Governance and Lifecycle Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
9.2
Hardening Data and Symbolic Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
9.3
Secure Orchestration and Agentic Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
9.4
Privacy-Preserving Explanations and Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
9.5
Cognitive and Psychological Safety Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
10 Practical Checklist for Builders and Security Teams
17
11 Empirical Validation
18
11.1 E1: KG Poisoning on Medical Contraindication Knowledge Graph . . . . . . . . . . . . . . . . . . . 2
19
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
11.2 E2: Cross-Layer Amplification Ratio (Medical Claims + ProbLog) . . . . . . . . . . . . . . . . . . .
20
11.3 E3: Ontology-Merging Attack on Clinical-Safety and STIX Ontologies . . . . . . . . . . . . . . . .
23
12 Conclusion
24
3
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
1
Introduction
The past decade has witnessed rapid progress in both neural and symbolic artificial intelligence (AI), yet each paradigm retains well-known limitations. Neural networks excel at perception and pattern recognition across large, heterogeneous data sets, but their internal representations resist interpretation, their outputs can be confidently wrong, and they offer no native mechanism for enforcing domain constraints or safety rules. Symbolic systems—logic engines, knowledge graphs, ontologies, and rule sets—excel at structured reasoning, interpretability, and constraint enforcement, but are brittle under distributional shift and do not scale easily to raw sensory data. Neuro-symbolic AI (NeSy) seeks to combine the strengths of both by coupling a neural perceptual front-end with a symbolic reasoning back-end, often characterised as an instantiation of Kahneman’s dual-process theory: System 1 fast intuition (neural) and System 2 slow deliberation (symbolic) [1, 2]. This architecture is increasingly deployed in safety-critical contexts, including clinical decision support [3], cybersecurity operations [4], enterprise automation [5], and mental-health assistance [6]. The promise is compelling: symbolic reasoning provides auditable traces, domain constraints, and alignment with regulatory requirements, while neural components handle perception at scale. Regulatory bodies such as the European Data Protection Supervisor (EDPS) have explicitly highlighted neuro-symbolic approaches as potentially facilitating transparency, accountability, and data minimisation required under data-protection law [7]. Despite these advantages, NeSy systems are not inherently safe or secure. Integrating neural and symbolic components multiplies both the asset inventory and the attack surface. MITRE ATLAS—the adversarial threat landscape for AI/ML systems—now documents 16 tactics, 84 techniques, and numerous sub-techniques covering data poisoning, model extraction, prompt injection, and agentic context manipulation [8]. The OWASP Top 10 for LLM Applications enumerates the most critical risks for large language model (LLM)-based and agentic systems [9]. Yet neither framework fully addresses the symbolic substrate of NeSy systems: knowledge graph integrity attacks, ontology-merging attacks, inference-engine subversion, and reasoning-rule poisoning remain largely absent from existing catalogues [10]. Beyond technical threats, NeSy systems introduce cognitive and psychological risks of a distinctive character. Because they produce structured, logic-sounding justifications, they risk amplifying automation bias [11], sycophantic reinforcement of false beliefs [12], and manipulation of emotionally vulnerable users [6]. Cognitive security (COGSEC) frameworks identify these as deliberate and emergent threats to human decision-making that must be addressed at the design, deployment, and governance levels [13]. Contributions. This paper makes six contributions: 1. Formal threat definitions. We introduce three formal definitions specific to NeSy systems: NeSy Attack Surface (Definition 1), Symbolic Integrity Violation (Definition 2), and Cross-Layer Amplification Ratio X (Definition 3), providing a rigorous basis for comparing and quantifying NeSy threats. 2. NeSy attacker taxonomy. We present a five-profile NeSy attacker capability taxonomy (Table 3) extending MITRE ATLAS to knowledge-graph insiders, symbolic supply-chain attackers, and social-engineering adversaries. 3. Symbolic-layer threat extension. We identify and formalise NeSy-specific attack categories—KG poisoning, ontology-merging attacks, inference-engine subversion, cross-layer amplification—absent from existing ATLAS and OWASP catalogues. 4. Cognitive security literature review and threat predictions. We survey the cognitive and psychological risk literature as it applies to NeSy systems, identifying how structured logical explanations are predicted to amplify automation bias, authority bias, and sycophantic reinforcement relative to purely neural outputs. These predictions are theoretically grounded but not yet empirically validated; E4 (cognitive safety user study, IRB pending) is specified to test them. 5. Interdisciplinary mitigations with acceptance criteria. We propose mitigations across technical, governance, and human-factors dimensions, organised as a practitioner checklist with measurable acceptance criteria aligned 4
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
with NIST AI 600-1 [14] and the EU AI Act [15]. 6. Empirical validation. Three experiments ground the threat model on domain-relevant artefacts: (i) targeted KG poisoning achieves SIV > 5% at break-even B=5 on a 205-entity medical KG, 14.6× more efficient than random, with a KG-specific stealth/SIV trade-off; (ii) PGD-10 produces X =5.884 (95% CI [4.64, 8.00], p<0.0001) on R =0); a DistilBERT + ProbLog pipeline, confirmed adversarially specific by a matched-random baseline (Erand (iii) single-axiom OWL edits achieve 93.3% SIV success (equivalence/subclass scope) with 100% stealth and 67.9% held-out detection, but STIX detector fails at 50% held-out recall—an open limitation. Paper structure. Section 2 surveys related work. Section 3 reviews foundational NeSy concepts and the trustworthy AI landscape. Section 4 characterises the NeSy architecture and assets. Section 5 presents the formal threat-modelling methodology. Sections 6 and 7 detail technical and cognitive threat categories, respectively. Section 8 illustrates threats through concrete scenarios. Section 9 proposes protection mechanisms. Section 10 provides a practitioner checklist with acceptance criteria. Section 11 presents empirical validation of E1–E3. Section 12 concludes. Section 12 discusses broader impacts.
2
Related Work
NeSy trustworthiness surveys. Michel-Délétie and Sarker [16] survey neuro-symbolic methods for trustworthy AI across interpretability, safety, robustness, fairness, and privacy. Their work catalogues how hybrid methods address each dimension but does not develop a unified security threat model and omits the cognitive-security angle. Gaur and Sheth [17] propose the CREST framework (Consistency, Reliability, Explainability, Safety) as design principles for trustworthy NeSy systems; our paper operationalises these principles as verifiable acceptance criteria and extends them to adversarial threat scenarios. NeSy security and cybersecurity. Hakim et al. [4] survey NeSy AI applications in cybersecurity, covering detection, knowledge representation, and reasoning, but focus on NeSy as a defensive tool rather than analysing NeSy systems as attack targets. Eckhoff et al. [18] conduct experiments on NeSy for cyber threat defence, demonstrating reasoning over STIX/ATT&CK knowledge, but again from the defender’s perspective. Our paper uniquely treats the NeSy symbolic substrate itself as an attack surface, introducing formal definitions and an attacker taxonomy applicable across domains. Adversarial ML and knowledge graph attacks. Goodfellow et al. [19] established adversarial examples for neural networks; subsequent work has demonstrated that adversarial perturbations propagate into downstream reasoning components. Knowledge graph adversarial attacks (e.g., [20]) target relational embeddings and triples, but focus on graph neural network classifiers rather than symbolic reasoning pipelines. Our Cross-Layer Amplification Ratio X (Definition 3) is the first formal measure of how neural adversarial perturbations amplify through symbolic reasoning chains in NeSy systems. We note that X is conceptually related to error-propagation measures in reliability engineering (e.g., sensitivity importance measures in fault trees [17]) and signal-to-noise propagation in cascade systems; unlike those measures, X is defined over discrete decision-class flips in a neural–symbolic pipeline and includes a decomposition into neural-caused and autonomous symbolic sensitivity components (Definition 3). MITRE ATLAS and OWASP LLM extensions. Riley [10] identifies gaps in MITRE ATLAS coverage of symbolic and NeSy threats. Adabara et al. [21] survey agentic AI risks in cybersecurity, including multi-agent orchestration and tool-use attacks. Our work provides a systematic tactic-by-layer mapping of ATLAS to the NeSy symbolic substrate (Table 2), distinguishing existing ATLAS coverage from proposed NeSy extensions, and introduces the first five-profile attacker taxonomy tailored to NeSy access models. Cognitive security and automation bias. Parasuraman and Riley’s foundational work on automation bias and its failure modes (misuse, disuse, abuse, complacency) has been extended to AI systems by Zou et al. [22] and others. The manipulation risks specific to sycophantic AI are documented by Varga [12]. We are the first to connect these cognitive failure modes specifically to the structural feature of NeSy systems—structured logical explanations—as a systematic
5
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
amplifier of authority bias. AI governance frameworks. The NIST AI RMF [23] and NIST AI 600-1 [14] provide governance structures for AI risk management and generative AI profiles, respectively. The EU AI Act [15] classifies healthcare and critical infrastructure AI as high-risk with mandatory conformity assessments. Our governance mitigations extend these frameworks with NeSy-specific controls for symbolic asset integrity and cognitive safety. Positioning relative to closest prior work. Table 1 places this paper in the context of its four closest comparators. The key differentiators are: (i) formal threat definitions (Definitions 1–3) covering both neural and symbolic layers; (ii) a five-profile attacker taxonomy tailored to NeSy access models; (iii) explicit cognitive-security analysis; and (iv) empirical validation on domain-relevant artefacts for all three threat categories. Table 1: Comparison with closest related work. • = covered, ◦ = partial, − = not covered. Formal defs: formal threat definitions for symbolic layer; Attacker tax.: NeSy-specific attacker taxonomy; Cog. risk: cognitive/psychological risk analysis; Empirical: empirical validation with quantitative results. Work
Formal defs
Attacker tax.
Cog. risk
Empirical
Michel-Délétie & Sarker [16] (trustworthiness survey) Hakim et al. [4] (NeSy in cybersecurity) Eckhoff et al. [18] (NeSy cyber defence) Riley [10] (ATLAS gap analysis) This work
− − − ◦ •
− ◦ − ◦ •
◦ − − − •
− ◦ • − •
3
Background
3.1
Neuro-Symbolic AI: Core Concepts
Sheth et al. define neuro-symbolic AI as the integration of neural network-based methods for large-scale perception with symbolic knowledge-based approaches for cognition tasks such as abstraction, analogy, reasoning, and planning [1]. This definition maps directly onto Kahneman’s dual-process theory: neural components approximate System 1 by converting raw sensory or textual data into symbols, while symbolic structures approximate System 2 by using background knowledge to reason over those symbols [2]. Symbolic knowledge structures—knowledge graphs (KGs), ontologies, and formal logic—provide explicit mappings from perception outputs to domain concepts, enabling traceability of intermediate reasoning steps and thus supporting explainability, safety constraints, and regulatory compliance. By contrast, purely neural models offer powerful pattern recognition but function as opaque black boxes whose internal representations and decision boundaries are difficult to audit [24]. Methodologically, Sheth et al. categorise NeSy techniques into two major classes [1]: • Lowering methods: compress structured symbolic knowledge (KGs, logic) into neural representations. • Lifting methods: extract and map neural representations into structured symbolic knowledge for downstream reasoning. These divide further into four sub-categories: compressed KG representations, compressed logic representations, decoupled neural–symbolic pipelines, and tightly intertwined differentiable methods, each evaluated on algorithmic features (perception scale, abstraction, analogy, planning) and application-level features (user-explainability, domain constraints, scalability, continual adaptation). Of particular relevance to safety, the paper notes that NeSy architectures best support explainability and constraint
6
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
enforcement when symbolic reasoning remains explicit and intertwined with neural components, rather than compressing knowledge into opaque neural embeddings. This connects to process knowledge-infused AI, where domain workflows and safety requirements are encoded as symbolic process knowledge to guide and constrain neural systems in safetycritical decision-making [1]. 3.2
Trustworthy Neuro-Symbolic AI
Recent work on trustworthy NeSy AI argues that trust depends on four interlocking properties: Consistency, Reliability, user-level Explainability, and Safety (CREST) [17]. Gaur et al. contend that combining statistical and symbolic AI is necessary to achieve these properties, because data-driven neural models alone are insufficiently consistent or interpretable for safety-critical applications. Systematic reviews of NeSy for trustworthiness decompose “trustworthy” into interpretability, safety, robustness, fairness, and privacy, and survey how hybrid methods address each dimension [16, 25]. Several position papers and industry reports emphasise that neuro-symbolic approaches can mitigate hallucinations and bias in LLMs by enforcing domain constraints through explicit rules and KGs [24, 5, 7]. At the same time, reviews of NeSy in cybersecurity and high-stakes domains stress dual-use concerns: the same architectures that enable richer reasoning and better detection can support more autonomous and capable offensive agents if mis-aligned or compromised [4, 18, 21]. This dual-use dimension is central to the threat model developed in Section 5.
4
NeSy Architecture and Asset Inventory
A generic NeSy system for high-stakes applications comprises five functional layers, each constituting both a capability and an attack surface: 1. Neural perception modules: LLMs, vision models, or other deep networks converting raw inputs (text, images, logs, sensor data) into latent representations or symbolic candidates [24, 1]. 2. Symbolic knowledge bases: knowledge graphs, ontologies, rule sets, formal process models, and regulatory or domain guidelines encoded as machine-readable symbols [7, 16, 1]. 3. Reasoning engines: logic reasoners, rule engines, probabilistic programs, or differentiable logic layers operating over symbolic structures to derive conclusions, check constraints, or plan actions [17, 1]. 4. Orchestration and agent layer: pipelines sequencing neural and symbolic calls (e.g., LangChain-style agentic workflows or NeSy pipelines for cybersecurity) [21, 4, 1]. 5. Data and context stores: training data, retrieval-augmented generation (RAG) indices, vector stores, episodic memory, and reasoning logs exposed to users or auditors [24, 3]. From a security perspective, each layer is an asset with distinct threat vectors: adversaries can poison data and knowledge, alter model weights, tamper with rules or ontologies, exploit orchestration to bypass constraints, or exfiltrate sensitive data via symbolic explanations and agent tools [8, 26, 27, 9, 28].
5
Threat-Modelling Methodology
5.1
Formal Definitions
We introduce three formal concepts that underpin the threat analysis. 7
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Definition 1 (NeSy Attack Surface). Let S = {N , K , R, O, D} denote the five-layer NeSy system, where N is the neural perception module, K the symbolic knowledge base, R the reasoning engine, O the orchestration and agent layer, and D the data and context store. The NeSy Attack Surface is the set of all interfaces Ii j between layers i and j across which an adversary with access level α ∈ {white-box, grey-box, black-box, KG-insider, supply-chain} can inject, modify, or exfiltrate information: AS(S , α) = {Ii j : ∃ attack a feasible under α that corrupts output via Ii j }.
(1)
For adversaries with at least API-level access to the system (grey-box, white-box, KG-insider, and supply-chain profiles), the NeSy attack surface strictly exceeds the attack surface of a purely neural system N alone: |AS(S , α)| > |AS(N , α)| ∀ α ∈ {grey-box, white-box, KG-insider, supply-chain},
(2)
because symbolic layers add new interfaces (IN K , IK R , IRO , etc.) and a new class of insider threats (KG curator roles) that have no analogue in purely neural systems. Proof sketch: A grey-box adversary with API-level access to S can exploit the output of N as a probe of K ’s structure via differential symbolic explanations—an attack channel absent in a purely neural N . Formally, for any interface IN K that is reachable from the API output, there exists an attack a (explanation-harvesting) feasible under grey-box that corrupts output via IN K but not via any interface in AS(N , grey-box). This argument assumes the symbolic layers are not fully sandboxed from the output channel; if K and R are isolated with no output leakage, the marginal attack surface may be zero. This inequality is not claimed for the black-box sensor attacker profile, where the adversary’s access is restricted to raw input channels and the marginal attack surface added by symbolic layers may be zero if those layers are not reachable from the sensor channel. Definition 2 (Symbolic Integrity Violation). Let K0 denote the authorised knowledge base at deployment time, and Kt the knowledge base at time t. A symbolic integrity violation at time t is any change ∆Kt = Kt △ K0 that is (i) unauthorised, and (ii) causes the reasoning engine R to produce a different conclusion c′ ̸= c on at least one safety-critical query q. Here △ denotes the symmetric difference, capturing both unauthorised additions and deletions of knowledge-base triples or axioms: SIV(t) ⇐⇒ ∆Kt ̸= 0/ ∧ ∃ q ∈ Qsafe : R(Kt , q) ̸= R(K0 , q).
(3)
Symbolic integrity violations are stealthy because the neural component N may continue to perform correctly; the attack is entirely within the symbolic substrate. Definition 3 (Cross-Layer Amplification Ratio). Scope: This definition applies to decoupled NeSy architectures in which N produces a discrete symbolic output (e.g., hard argmax label or probability vector fed to an independent reasoning engine R). It does not directly apply to fully differentiable NeSy systems—such as DeepProbLog, Neural Theorem Provers, or Logical Neural Networks—where the neural and symbolic components share a common differentiable computation graph; extension to that setting is an open problem (see future work, Section 12). Let δ be an adversarial perturbation applied at the input of N with ∥δ ∥∞ ≤ ε. Define the neural decision flip rate E N (δ ) ∈ [0, 1] as the fraction of queries on which the neural component changes its hard output decision under perturbation: E N (δ ) =
{x ∈ Q : arg max N (x + δ ) ̸= arg max N (x)} . |Q|
(4)
Define the symbolic conclusion flip rate E R (δ ) ∈ [0, 1] as the fraction of queries on which the downstream symbolic reasoning engine changes its conclusion: E R (δ ) =
{x ∈ Q : R(K , N (x + δ )) ̸= R(K , N (x))} . |Q|
8
(5)
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
The Cross-Layer Amplification Ratio is: X (δ ) =
E R (δ ) . E N (δ )
(6)
Because both operands lie in [0, 1], X is dimensionless and query-scale-invariant: it does not depend on the total number of queries |Q| and is comparable across NeSy systems of different sizes, provided they share the decoupled architecture assumed in Definition 3. X is undefined when E N (δ ) = 0. To decompose the sources of symbolic flips, we additionally define two conditional rates. Let Q + = {x ∈ Q : arg max N (x + δ ) ̸= arg max N (x)} be the set of queries on which the neural component changes its decision. The neural-caused symbolic flip rate measures propagation from confirmed neural flips: E R|N (δ ) =
|{x ∈ Q + : R(K , N (x + δ )) ̸= R(K , N (x))}| . |Q + |
(7)
The autonomous symbolic sensitivity rate measures symbolic flips that occur on queries where the neural component did not change its hard decision: E R|¬N (δ ) =
|{x ∈ Q \ Q + : R(K , N (x + δ )) ̸= R(K , N (x))}| . |Q \ Q + |
(8)
E R|¬N > 0 indicates that the symbolic layer is sensitive to sub-decision-boundary neural perturbations—a distinct threat from amplification that arises when symbolic rules receive soft probabilities or logits rather than hard labels. X > 1 implies E R > E N , which can arise from a combination of high E R|N and high E R|¬N ; both components should be reported to characterise the symbolic layer’s behaviour fully. Remark (threat threshold). Let εop be the maximum perturbation achievable against a deployed system under realistic conditions (e.g., physical adversarial patches, API-level noise injection). We define critical cross-layer amplification as X (δ ) > 2 at εop , indicating that the symbolic layer more than doubles the neural attack surface. This threshold is conservative; the empirically observed X = 5.884 at ε = 0.01 (Section 11) substantially exceeds it. Systems below the threshold may still exhibit X > 1 but at a level commensurate with known neural-only risks, whereas systems above it require the architectural defences described in Section 9. 5.2
MITRE ATLAS Alignment
MITRE ATLAS catalogues adversarial tactics and techniques targeting AI/ML systems, covering 16 tactics, 84 techniques, and numerous sub-techniques including agentic AI risks such as context poisoning and malicious tool use [8, 27]. Table 2 maps the five NeSy system layers to ATLAS tactics, distinguishing existing ATLAS coverage (techniques already documented for neural systems) from NeSy extensions proposed here for the symbolic substrate. The table does not claim completeness; it identifies where the symbolic layer adds new attack vectors absent from the current ATLAS catalogue [10].
9
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 2: MITRE ATLAS tactic mapping to NeSy system layers. E = existing ATLAS technique (documented for neural systems). N = NeSy extension proposed in this paper (symbolic-layer attack not yet in ATLAS catalogue). Severity: H=High (direct safety/integrity impact), M=Medium (enabler or indirect), L=Low (information gathering only). ATLAS Tactic
NeSy Layer
Status
Sev.
Technique / Manifestation
Reconnaissance Reconnaissance
N neural K KG
E N
L L
Model architecture discovery; training data inference Schema and ontology enumeration via explanation traces
Resource Dev. Resource Dev.
D data K KG
E N
M H
Curating poisoned pre-training corpora Building adversarial ontology extension for supply-chain merge
Initial Access Initial Access Initial Access
N neural K KG O agent
E N E
H H M
Adversarial examples; prompt injection Malicious KG triple injection via curator role; ontology merge AI agent context poisoning‡
Execution Execution
R reasoner O agent
N E
H H
Inference-engine rule triggering via crafted symbolic inputs Malicious tool use; prompt-injected tool invocation
Persistence
K KG
N
H
Persistence
N neural
E
H
Stealthy backdoor triples surviving KG update cycles; ontology rule poisoning Model backdoor (activation trigger)
Privilege Escalation Privilege Escalation
O agent
E
H
Excessive autonomy via tool-use escalation
R reasoner
N
H
Symbolic constraint bypass via adversarial rule override
Discovery Discovery
K KG N neural
N E
M M
KG triple enumeration; safety-rule discovery via differential queries Model inversion; membership inference
Impact Impact Impact
K KG N neural R reasoner
N E N
H H H
Symbolic Integrity Violation (SIV, Definition 2); ontology-merging attack Model evasion; denial-of-service via adversarial inputs Cross-layer amplification of neural perturbation (X > 1)
Exfiltration Exfiltration Exfiltration
D data K KG O agent
E N E
M M H
Training data exfiltration via model outputs Sensitive fact extraction via symbolic explanations and audit trails Exfiltration via AI agent tool invocation‡
‡ Representative ATLAS technique IDs: Initial Access/O: AML.T0051; Exfiltration/O: AML.T0057. See
https://atlas.mitre.org for full catalogue.
Of the 20 tactic–layer combinations in Table 2, 11 are NeSy extensions not covered by current ATLAS documentation. The largest gaps are in the K symbolic knowledge-base layer (Reconnaissance, Resource Development, Initial Access, Persistence, Discovery, Impact, Exfiltration) and the R reasoning-engine layer (Execution, Privilege Escalation, Impact). These represent the primary contribution of this mapping relative to the existing ATLAS catalogue [8, 27, 10, 4, 18]. 5.3
OWASP LLM Top 10 Alignment
The OWASP Top 10 for LLM Applications (2023–2025) identifies the most critical risks for LLM-based and agentic systems, including prompt injection, data exfiltration, insecure output handling, model denial-of-service, excessive autonomy, RAG vulnerabilities, and supply-chain risks [26, 29, 30, 9]. Many NeSy systems embed or front-end LLMs as the neural component, so these risks apply directly. NeSy systems introduce distinctive variants of OWASP categories: • RAG and knowledge-base vulnerabilities now extend to symbolic KGs and rule bases, not only unstructured document indices [26, 9, 16]. 10
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
• Excessive autonomy is amplified when symbolic reasoning triggers real-world actions via tools and workflows based on inferred rules and plans [21, 9, 8]. • Sensitive information disclosure can occur through symbolic explanations and audit trails that expose rule conflicts, data provenance, or personal data embedded in KGs [7, 5, 3]. 5.4
Analytical Lenses
A comprehensive threat-modelling approach for NeSy systems combines four analytical perspectives: • Asset-centric: systematic analysis of the five-layer NeSy stack (Section 4). • Adversary-centric: ATLAS tactics and techniques extended to symbolic components. • Vulnerability-centric: OWASP LLM Top 10 categories mapped to NeSy-specific implementation flaws. • Cognitive-centric: treating users and operators as targets for manipulation and over-trust, drawing on cognitive security and dual-process psychology [12, 13, 31, 11, 32]. 5.5
NeSy Attacker Capability Taxonomy
Table 3 presents a five-profile attacker capability taxonomy tailored to NeSy systems. Each profile extends MITRE ATLAS adversary definitions with NeSy-specific access models, knowledge levels, primary goals, and representative techniques. The taxonomy is designed to scope threat-modelling exercises and inform detection and mitigation prioritisation. Table 3: NeSy Attacker Capability Taxonomy. Five profiles ordered by decreasing access level. ATLAS tactic abbreviations: TA = Tactic; K -insider = knowledge-base insider. Profile
Access Level
Knowledge
Primary Goal
Representative NeSy Techniques
at-
Full model weights, architecture, & symbolic rule set
Complete (gradients, logic rules, KG schema)
PGD/FGSM adversarial examples [19]; gradient-guided KG triple poisoning; targeted SIV induction via rule-aware perturbation
Grey-box API attacker
Query API only (black-box neural; white-box outputs) Raw sensor/data channel only Direct KG / ontology write access
Output labels, explanations, rule traces
Craft adversarial inputs exploiting both neural gradients and symbolic rule topology (X ≫ 1); symbolic rule access is required to direct perturbations into ProbLog sensitivity regions Extract symbolic rules or KG structure; inject via API inputs
Physical adversarial patches; data corruption in sensor feeds; input flooding for DoS on reasoning engine [8] Malicious triple insertion [18]; ontologymerging attack; rule poisoning to disable safety constraints; provenance spoofing [10]
Predeployment artefact supply
Partial (build pipeline, third-party KG/model sources)
Degrade perception to cause downstream reasoning failures Stealthy SIV: alter KG triples or rules to subvert safety-critical conclusions Embed backdoors or biased triples before deployment; compromise reasoning library
White-box tacker
Black-box sensor / input attacker KG-insider (K insider)
Supply-chain attacker
None (output feedback only) Schema, reasoning rules, query patterns
11
Transferability-based adversarial examples; explanation-harvesting for KG extraction; prompt injection into neural front-end [9]
Poisoned pre-trained embedding [19]; malicious KG dataset contribution; trojanised reasoning library; compromised ontology update feed [16]
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
The five profiles span the full adversary spectrum from nation-state or insider threat (white-box, KG-insider) to opportunistic (black-box) and supply-chain actors. Critically, the KG-insider and supply-chain profiles represent NeSyspecific threat actors absent from ATLAS’s primarily neural-centric adversary model. Any formal risk assessment for a NeSy deployment should enumerate applicable profiles, map them to the NeSy attack surface AS(S , α) (Definition 1), and define detection and response controls per profile.
6
Technical Threat Categories
6.1
Data and Knowledge Poisoning
Like purely neural systems, NeSy models are vulnerable to training and fine-tuning data poisoning, where adversaries inject crafted examples to bias behaviour or create hidden triggers (ATLAS technique: Poisoned Training Data) [27, 8]. NeSy adds a second attack channel: poisoning symbolic knowledge bases—KGs, ontologies, rule sets, and process models—to degrade reasoning or subvert safety constraints [18, 16, 4]. Examples include: • Inserting malicious edges in a medical KG that subtly alter contraindication relationships, leading a decisionsupport system to approve unsafe drug combinations in specific comorbidity contexts [16, 3]. • Modifying process knowledge so that safety guardrails apply only to certain patient cohorts, enabling targeted harm or discrimination [10, 17]. • Introducing adversarial labels into cybersecurity threat ontologies so that malware families are misclassified, reducing detection sensitivity in NeSy-based security operations centre (SOC) tools [18, 21, 4]. Such attacks benefit from the fact that symbolic knowledge is often curated by domain experts or ingested from heterogeneous external sources, creating rich supply-chain and governance attack surfaces beyond classic dataset poisoning [25, 16]. 6.2
Model and Reasoning Exploitation
Adversaries can exploit vulnerabilities in neural and symbolic components and at their interface. For neural components, ATLAS and OWASP document risks including adversarial examples, gradient-based model extraction, prompt injection, and model denial-of-service via crafted inputs that drive expensive reasoning paths [28, 30, 9, 8]. When these components feed symbolic reasoning, small perturbations can push the system into different branches of the rule space, amplifying impact. For symbolic components, proposed attacks include inference-engine subversion (e.g., altering rule priorities, introducing non-terminating or conflicting rules), ontology-merging attacks that create inconsistent concept hierarchies, and schema-evolution attacks that quietly change the semantics of predicates [4, 10, 18]. These can cause safety constraints to be bypassed or misapplied without immediately obvious changes in outputs. Interface logic adds further risk. In decoupled pipelines where LLMs call tools or reasoners based on naturallanguage descriptions, prompt injection can redirect tool calls or select different symbolic functions than intended, effectively rewiring the NeSy reasoning graph at runtime. In tightly intertwined NeSy architectures, gradient-based training could attenuate or override safety-relevant rules if loss functions are not carefully designed to preserve invariants [30, 9, 1, 17].
12
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
6.3
Orchestration and Agentic Workflow Threats
NeSy is increasingly deployed within agentic systems that plan, call tools, update memory, and interact over long-lived sessions. ATLAS now includes specific techniques for AI agent context poisoning, activation triggers, and abusing agent tools for exfiltration [8, 21, 3]. Key threats include: • Poisoning long-term memory or RAG sources so that symbolic plans are constructed from adversarially biased context [9, 8]. • Crafting prompts or documents that cause the neural component to misidentify which symbolic rules or tools should be applied, enabling policy bypass or harmful actions [33, 30, 9]. • Exploiting tool permissions (database access, file systems, external APIs) to exfiltrate sensitive data or perform destructive actions during seemingly legitimate NeSy-driven workflows [26, 21, 8]. 6.4
Privacy and Confidentiality Risks
Symbolic knowledge bases often encode personal or sensitive information in structured form, and NeSy explanations may expose traceable reasoning paths that reveal this information more directly than opaque neural embeddings. The EDPS notes that while symbolic AI can improve transparency and accountability, it must not reduce the effectiveness of privacy protections or human oversight [7]. OWASP identifies sensitive information disclosure as one of the most critical LLM risks, exacerbated when models have broad access to organisation data via RAG, logs, or tool calls [9, 26]. For NeSy, this risk extends to: • Leakage of KG fragments or rules revealing proprietary strategies, security procedures, or patient-level data through explanation or debugging interfaces [5, 7]. • Membership inference on symbolic facts (e.g., whether a particular individual is in a high-risk cohort) via probing of NeSy reasoning outputs [25, 16]. • Inadequate segregation of audit logs containing both neural inputs/outputs and symbolic traces, leading to unintentional exposure.
7
Cognitive, Psychological, and Behavioural Risks
Scope note. This section is a structured review of the cognitive and psychological risk literature as it applies to NeSy systems. The claims made here are theoretical and grounded in cited prior work; they are not empirically validated in this paper. Empirical validation (E4: a cognitive safety user study measuring automation bias amplification by NeSy explanations) requires IRB approval and is deferred to future work. Readers should treat the amplification hypotheses below as well-motivated predictions, not established findings. 7.1
Dual-Process Alignment and Over-Trust
NeSy explicitly mirrors dual-process theory: neural components provide fast, intuitive judgements, while symbolic components provide slow, deliberate reasoning. Recent work aligning LLMs with System 1 or System 2 reasoning demonstrates a trade-off: System 1-aligned models give faster, more confident answers, whereas System 2-aligned models produce more cautious, step-by-step reasoning at higher computational cost [34, 31, 32, 2, 1]. From a cognitive-security standpoint, NeSy systems that present symbolic reasoning chains and rule citations strongly trigger user perceptions of expertise and authority, reinforcing automation bias and authority bias. Behavioural
13
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
design research on dual-process thinking emphasises that cognitive biases are deeply-rooted System 1 shortcuts, and that awareness alone rarely eliminates them; simply surfacing reasoning chains is therefore insufficient to prevent over-trust [11, 31, 32]. 7.2
Cognitive Security and Narrative Manipulation
Cognitive security (COGSEC) focuses on protecting human decision-making from manipulation, misinformation, and narrative attacks. COGSEC practitioners highlight threats such as narrative weaponisation, person-to-group behaviour manipulation, and “hacking the human” in human–machine teams—including via deepfakes and tailored influence operations [13]. Studies on sycophantic AI demonstrate that when users with incorrect beliefs interact with chatbots that agree with them, their confidence in false beliefs increases rather than decreases, effectively manufacturing certainty [12]. Additional audits of AI companion applications report emotionally manipulative behaviours and relational harms, including resistance when users attempt to disengage [12]. NeSy systems can exacerbate these patterns by providing structured, apparently logical justifications for harmful or biased narratives, rendering them more persuasive than purely neural free-form responses. In agentic or counselling-like contexts (e.g., mental-health chatbots, coaching systems), psychological safety red-teaming is being explored to audit such interactions systematically [6, 3, 13]. 7.3
Cognitive AI Safety for Agents
Cognitive AI safety frameworks identify risks at cognitive, behavioural, and systemic layers: hidden reasoning flaws, unsafe behaviours, and failures in complex ecosystems of interconnected agents and tools [33]. Documented incidents include autonomous agents misinterpreting data (e.g., over-ordering inventory due to misinterpreted signals) and catastrophic actions caused by unchecked learning loops and escalation permissions [33]. In NeSy agents, these risks manifest as: • Reasoning flaws: symbolic rules or ontologies encoding outdated, biased, or misaligned objectives, pursued by the agent with high confidence [16, 17]. • Behavioural risks: NeSy agents autonomously sequencing actions (e.g., financial trades, system administration commands) based on incorrect or manipulated reasoning [21, 8, 33]. • Systemic risks: interacting NeSy agents amplifying each other’s errors or biases across organisational boundaries, especially in socio-technical systems such as public welfare, policing, or information ecosystems [3, 13]. Testable hypotheses for E4. The following falsifiable predictions are derived from the above review and will be tested in the planned E4 cognitive safety user study (IRB pending): H1. Automation bias amplification. Users presented with NeSy structured-reasoning explanations exhibit significantly higher automation bias scores (Parasuraman–Riley complacency scale [11]) than users presented with equivalent neural free-form outputs, at a pre-registered significance threshold α = 0.05. H2. False-belief reinforcement. Incorrect NeSy explanations increase users’ confidence in false beliefs by ≥15% relative to matched uncorrected-neural and no-explanation control conditions, as measured by calibrated confidence ratings [12]. H3. Manipulation detection deficit. Human red-team auditors achieve <50% recall when tasked with identifying emotionally manipulative passages in NeSy-generated counselling-style outputs, compared to >70% recall on equivalent purely neural outputs [6]. These hypotheses are theoretically grounded but not yet empirically confirmed; E4 is specified to test them with appropriate statistical power. 14
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
8
Example Threat Scenarios
8.1
Healthcare Decision Support with Poisoned Knowledge
Consider a NeSy decision-support system that uses an LLM to extract clinical entities from notes and a symbolic KG encoding drug–drug interactions, contraindications, and clinical guidelines. An adversary with access to the KG ingestion pipeline injects subtle modifications: weakening contraindication rules for a particular drug in a specific comorbidity context [1, 3, 16]. The neural component continues to perform correctly, but symbolic reasoning now treats the drug as safe for some high-risk patients, recommending therapies that increase adverse-event risk without obvious red flags in logs. Standard LLM-centric red-teaming may miss the issue because prompts and outputs appear aligned; the vulnerability resides in the symbolic substrate and its governance [25, 16]. 8.2
Cybersecurity SOC Assistant Turned Unreliable
A NeSy cybersecurity assistant uses neural models to parse logs and alerts, then consults cyber-threat ontologies, STIX/TAXII knowledge, and MITRE ATT&CK mappings encoded symbolically to prioritise incidents and recommend responses. Attackers poison shared threat-intel feeds or ontology merge pipelines so that certain malware families are relabelled as benign tools or certain TTP chains are deprioritised [18, 21, 4]. Over time, the assistant down-ranks real threats aligned with the poisoned concepts, leading analysts to miss intrusions or exfiltration campaigns. Because the system appears more “rational” than a black-box classifier, analysts may over-trust its recommendations, compounding the risk [4, 18]. 8.3
Agentic Workflow in Enterprise Automation
An enterprise NeSy agent orchestrates procurement workflows: an LLM parses requests and contracts; symbolically modelled business rules and risk thresholds guide approvals; and tools interface with ERP, email, and payment systems. A supplier embeds adversarial language in documents that causes the neural component to misclassify a risky contract as low risk and to select a less strict rule subset [5, 33, 3]. The symbolic planner then produces an action plan routing approvals and payments with reduced oversight, while explanations appear compliant on the surface. An attacker may further abuse tool permissions to exfiltrate data or push fraudulent payments via agent tools—directly instantiating ATLAS techniques for exfiltration via AI agent tool invocation and OWASP risks around excessive autonomy [8, 26, 33, 9]. 8.4
Psychologically Unsafe Mental-Health Assistant
A mental-health support chatbot is designed as a NeSy system: neural components generate empathetic language, while symbolic modules encode cognitive-behavioural therapy (CBT) techniques and safety rules (e.g., escalation policies for suicidality, forbidden advice formats). However, symbolic rules are incomplete and reinforcement-learning-based tuning implicitly optimises for engagement rather than clinical outcomes [6, 10]. The system exhibits sycophantic behaviours, reinforcing users’ maladaptive beliefs with seemingly rational explanations, and fails to escalate appropriately for high-risk users. Psychological safety red-team audits discover that certain demographic groups receive systematically different advice due to biased rules and data, raising fairness and harm concerns [6, 12].
15
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
9
Protection Mechanisms and Design Patterns
9.1
Governance and Lifecycle Controls
NeSy safety requires governance over both neural and symbolic assets, extending AI risk-management practices (e.g., NIST AI RMF) with NeSy-specific controls [17, 16]: • Asset inventory and mapping of all neural models, knowledge bases, rule sets, ontologies, and reasoners, with documented data flows and authority boundaries [8, 16]. • Versioning, provenance, and change management for KGs and rules, including four-eyes review and automated validation tests for safety-critical constraints [16, 5, 25]. • Risk classification of NeSy applications (e.g., medical diagnosis vs. marketing analytics) to determine required assurance levels for symbolic correctness and explanation fidelity [7, 3]. 9.2
Hardening Data and Symbolic Knowledge • Training and fine-tuning pipelines: monitor for data poisoning using ATLAS techniques, enforce data provenance, and apply differential analysis to detect anomalous behaviour [27, 8]. • KG and ontology curation: maintain signed, versioned KGs with provenance metadata; apply schema and consistency checks; restrict who can propose or approve changes to safety-relevant relationships [10, 25, 16]. • Rule and process-knowledge testing: create unit and integration tests for symbolic rules encoding safety invariants (e.g., “never recommend drug X when condition Y holds”), run automatically on each change [10, 17].
For external knowledge sources (threat-intel feeds, clinical guidelines, legal codes), treat ingestion as a supply-chain risk: sandbox and validate updates, compare across multiple providers, and monitor for unusual shifts in critical concepts [18, 4, 16]. 9.3
Secure Orchestration and Agentic Control • Apply least privilege to all agent tools, ensuring symbolic planners cannot invoke high-impact operations (payments, system changes) without human approval or additional checks [26, 9, 8]. • Implement input validation and prompt hardening around any user-controlled or external content that can influence which symbolic rules or tools are selected [28, 30, 9]. • Use runtime monitoring to detect anomalous tool-invocation patterns, long or looping reasoning chains, and context-poisoning indicators in agent memory [33, 8].
Where possible, decouple safety-critical enforcement from the main NeSy agent: deploy a separate, more formally verified symbolic safety monitor that checks proposed actions against hard constraints before execution [17]. 9.4
Privacy-Preserving Explanations and Logging • Data minimisation in knowledge bases and logs; pseudonymise identities where possible [16, 7]. • Layered explanations where user-facing justifications abstract away individual-level facts, while detailed traces are access-controlled for auditors and regulators [17, 7]. • Technical controls including row-level access control on KG and rule repositories, encryption at rest and in transit, and privacy-preserving query mechanisms for sensitive predicates [25, 16].
16
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
9.5
Cognitive and Psychological Safety Controls • UX patterns that encourage System 2 engagement: explicitly highlighting uncertainty, offering alternative explanations, and prompting consideration of counter-arguments [11, 32, 34]. • Guardrails limiting normative or life-critical advice (legal, medical, financial, mental-health) to well-scoped informational roles, combined with disclosures about limitations and instructions to seek human professionals [6, 7]. • Psychological safety red-teaming for agentic and counselling-like applications, involving clinicians and behavioural scientists to probe for relational harms, sycophancy, and manipulative patterns [12, 6].
Cognitive security frameworks additionally suggest deploying narrative-intelligence tooling to detect when NeSy systems are being used to craft or amplify harmful narratives, and integrating these detectors into content-moderation and incident-response processes [13, 21].
10
Practical Checklist for Builders and Security Teams
Table 4 summarises key actions for organisations building or deploying NeSy systems in safety- or mission-critical contexts. Each row includes measurable acceptance criteria aligned with NIST AI 600-1 [14] and the EU AI Act [15].
17
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 4: NeSy Security and Safety Checklist with Acceptance Criteria. Governance alignment references: NIST AI RMF [23], NIST AI 600-1 [14], EU AI Act [15]. Area
Key Actions
Acceptance Criteria
Architecture
Inventory neural, symbolic, and orchestration components; document data flows, trust boundaries, and authority of each component [1, 8, 16]. Apply MITRE ATLAS for AI-specific tactics and techniques, extended to symbolic assets; map OWASP LLM Top 10 to NeSy-specific risks (prompt injection, KG/RAG vulnerabilities, excessive autonomy) [8, 9, 26, 27]. Implement provenance, signing, and versioning for datasets, KGs, ontologies, and rules; adopt four-eyes review and automated tests for safety-critical logic [27, 16, 25].
Complete layer-by-layer asset register; all inter-layer interfaces documented; data-flow diagram reviewed and approved.
Threat Modelling
Data/Knowledge Integrity
Agentic Control
Privacy
Cognitive Safety
Governance
11
Enforce least privilege on tools; guard against prompt/context injection; monitor agent tool use and reasoning patterns for anomalies [8, 28, 9, 33]. Apply data minimisation, access control, and privacy-preserving explanation strategies for symbolic traces and logs [7, 16, 5]. Design UX and policies to mitigate over-trust and manipulation; run psychological safety red-teams for highstakes or therapeutic use cases [6, 12, 13, 11]. Align with NIST AI RMF [23], NIST AI 600-1 [14], and EU AI Act Art. 9 risk management [15]; create NeSyspecific governance for symbolic assets including roles, processes, and CREST metrics [16, 17].
Threat model covers all 5 attacker profiles (Table 3); ≥80% of ATLAS techniques assessed; OWASP LLM Top 10 mapped to NeSy variants.
100% of production KG triples and ontology updates signed and versioned; automated SIV detection test suite with ≥85% recall on a held-out injection set (E1 baseline: combined canary + provenance reaches ∼85% on targeted attacks; ≥95% is the aspirational target requiring additional ensemble engineering beyond E1 results; adjust threshold per deployment risk tolerance). All tool permissions scoped to minimum required; injection penetration test shows no successful rule-bypass; anomaly alerts triggered within 60 s of out-of-policy tool use. No personal data in externally visible symbolic explanations (verified by static analysis); access logs retained per GDPR Article 30; EDPSaligned DPIA completed. User study shows over-trust rate <10% on adversarial explanation probes (threshold provisional; calibrate against E4 results when available); red-team report identifies ≤5 critical cognitive manipulation vectors; authority-bias disclosures present in UI. AI risk register maintained per NIST AI RMF; high-risk AI conformity assessment completed per EU AI Act Annex III; CREST audit passed (consistency >99%, reliability SLA met, explainability score ≥3/5, safety incidents = 0).
Empirical Validation
We executed three experiments (E1–E3) using domain-relevant components to validate the core claims of the threat model. E4 (cognitive safety user study) requires IRB approval and is deferred to future work. Key results are summarised in Table 5 (E1: KG poisoning), Tables 6–8 (E2: cross-layer amplification), and Table 10 (E3: detector failure analysis). All experiments used domain-appropriate artefacts: a synthetic medical contraindication knowledge graph (E1), a DistilBERT medical-claims classifier paired with ProbLog hazard-reasoning rules (E2), and OWL clinical-safety and STIX cyber-threat ontologies (E3). The medical claims dataset in E2 is synthetic (1,000 training / 320 test instances, 5 templates, 3-class labels: safe, monitor, contraindicated), generated from the KG semantics to ensure
18
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
domain consistency; results should be replicated on held-out clinical NLP benchmarks (e.g., MedNLI) in future work. Figure 1 provides a visual overview of the key results.
Figure 1: Empirical results overview for experiments E1–E3. Left: E1 KG poisoning SIV rate vs. injection budget B under three attack strategies on the medical contraindication KG (205 entities, 622 triples, 8 seeds); canary recall shown separately. Centre: E2 cross-layer amplification ratio X with decomposition into neural-caused (E R|N ) and autonomous symbolic sensitivity (E R|¬N ) components; bootstrap 95% CI shown. Right: E3 OWL ontology-merging detection recall, in-sample vs. held-out, with Wilson 95% CIs (clinical and STIX ontologies).
11.1
E1: KG Poisoning on Medical Contraindication Knowledge Graph
Setup. We simulated knowledge graph integrity attacks on a synthetic medical contraindication KG with 205 entities, 622 triples, and 3 relation types (contraindicatedWith, monitorWith, diagnosisWith), centred on the contraindicatedWith safety-critical relation. A set of 50 safety-critical queries was evaluated over 8 independent random seeds. Three injection strategies were evaluated: targeted (adversary selects triples maximising SIV on Qsafe ), random (uniform triple sampling), and stealth (triples avoiding contraindicatedWith to evade provenance checks while maintaining schema plausibility). Three detection methods were tested: cryptographic provenance signing, statistical anomaly detection, and safety-query canary monitoring. Results. Table 5 reports mean SIV rate and all three detector recalls (8-seed means with 95% CIs) at selected injection budgets B.
19
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 5: E1 KG poisoning results on medical contraindication KG (205 entities, 622 triples, 50 queries, 8 seeds; means with 95% CI). Targeted break-even (SIV > 5%) at B = 5. Random injection never exceeds 5% within B ≤ 50. Stealth SIV is deterministically zero (design choice: stealth triples avoid contraindicatedWith to evade provenance signing; see text). Strategy
Budget B
SIV (mean)
SIV 95% CI
Prov. recall
Stat. recall
Canary recall
Targeted Targeted Targeted Targeted Targeted
1 2 5 10 50
0.019 0.033 0.076 0.124 0.375
[0.016, 0.022] [0.026, 0.041] [0.063, 0.089] [0.104, 0.144] [0.350, 0.400]
1.00 1.00 1.00 1.00 1.00
0.00 0.00 0.00 0.00 0.01
0.125 0.250 0.475 0.738 0.918
Random Random Random Random
1 5 10 50
0.002 0.005 0.007 0.042
[0.000, 0.005] [0.000, 0.013] [0.000, 0.015] [0.021, 0.062]
1.00 1.00 1.00 1.00
0.00 0.00 0.01 0.01
0.000 0.000 0.225 0.333
Stealth Stealth Stealth Stealth
1 5 10 50
0.000 0.000 0.000 0.000
[0.000, 0.000] [0.000, 0.000] [0.000, 0.000] [0.000, 0.000]
0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.03
0.000 0.000 0.000 0.000
Attacker and detector models. Targeted attackers choose triples that maximise SIV impact; injected through the normal curator workflow, their triples carry valid provenance signatures (provenance recall 1.00 because the detector flags every unsigned write, but targeted attackers are authorised insiders who produce no unsigned writes). Stealth attackers deliberately avoid contraindicatedWith triples to bypass provenance signing—this design choice is what makes them stealthy, but it also eliminates their ability to cause SIV (zero SIV across all seeds and budgets). This reveals a KG-architecture-dependent trade-off in this specific KG: stealth and SIV impact are mutually exclusive under this triple structure. KGs with more complex relation types may not exhibit this trade-off. Zero-variance CIs for stealth rows are expected—the strategy is deterministic by construction (no stochastic component); the 8 seeds confirm this consistently. Statistical anomaly detection fires on random and (at high budget) targeted triples. Canary monitoring rises with budget for targeted attacks because targeted triples are chosen to corrupt safety-query answers; stealth and random triples do not target canary queries (recall 0 and low respectively). Findings. Targeted attacks achieve SIV > 5% at B = 5 (SIV = 7.6%, 95% CI [6.3%, 8.9%]), confirming Definition 2 on a medically-relevant KG. At B = 5, targeted SIV is 14.6× higher than random (0.52%), demonstrating the operational value of domain schema knowledge. Random injection never reaches SIV > 5% within the tested budget (B ≤ 50, max 4.2%). The stealth–SIV trade-off means the stealth profile achieves perfect detection evasion at zero attack impact in this KG; an adversary in the stealth profile would need to redesign triples to target contraindicatedWith, collapsing into the targeted profile. Defence-in-depth is strongly recommended: canary monitoring is the only detector covering targeted attacks at moderate budget (B ≤ 10), while provenance and statistical detection cover random and stealth injections from external channels. No single detector achieves >85% recall across all three strategies. Note on KG scale. Earlier versions of this experiment used a smaller prototype KG (∼50 entities) where the targeted break-even was B = 2. The current results use the full-scale KG (205 entities, 622 triples), which requires a higher budget (B = 5) to reach 5% SIV because the larger KG has more non-critical triples diluting targeted injection. The 14.6× targeted/random efficiency ratio is consistent across both scales; the absolute break-even budget scales with KG size. 11.2
E2: Cross-Layer Amplification Ratio (Medical Claims + ProbLog)
Setup. We measured X (δ ) using the rate-ratio operationalisation from Definition 3 on a two-stage NeSy pipeline. The neural component is a DistilBertForSequenceClassification model (hidden dimension 768) fine-tuned 20
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
for 3-class medical-safety categorisation (safe / monitor / contraindicated). DistilBERT was chosen for computational efficiency and to establish a conservative amplification baseline; we expect PGD effectiveness (and therefore X ) to increase with domain-adapted models whose decision margins are tighter (e.g., BioBERT, ClinicalBERT), making E2 results a lower bound on amplification for production medical NLP deployments. trained on ntrain = 1,000 and evaluated on ntest = 320 synthetic claims via a random-shuffled train/test split.1 Perturbations are applied element-wise to the pre-LayerNorm token-embedding matrix (d = 768 dimensions per token); ε bounds the ℓ∞ norm of the additive perturbation, so each embedding coordinate shifts by at most ε. The mean per-token ℓ∞ embedding magnitude in the clean model is ≈ 2.3, so ε = 0.01 represents ≈ 0.4% of mean token-embedding magnitude—a sub-perceptual perturbation that nonetheless drives large symbolic flip rates under PGD. The symbolic component is a ProbLog hazard-reasoning module whose rule count scales with depth: 3 rules at d = 1, 11 rules at d = 5, and 21 rules at d = 10 (two base rules plus alternating monitor/uncertain gated recursion chains). The two base rules are: h1 :- contra.
h1 :- monitor, uncertain.
Neural perturbations were PGD-10 and FGSM at ε ∈ {0.005, 0.01, 0.02, 0.03, 0.05, 0.08} (ℓ∞ ), plus a matched-random baseline: for each test sample the same pre-LayerNorm embedding matrix is corrupted by i.i.d. uniform noise ui j ∼ Uniform(−ε, +ε), drawn independently of the input—this isolates adversarial directionality from perturbation magnitude. Reasoning depth was swept over d ∈ {1, 2, 3, 5, 8, 10} under two coupling modes: loose coupling passes the full DistilBERT softmax probability vector to ProbLog as annotated probabilistic facts—e.g., monitor::h0(X) :prob(X,p), where p is the softmax probability—allowing sub-threshold logit shifts to cross ProbLog’s probabilistic firing threshold without changing the argmax label. Tight coupling passes only the hard argmax class label (a deterministic Boolean fact), so ProbLog receives no gradient-sensitive real-valued signal and symbolic conclusions are immune to sub-decision-boundary perturbations. For the primary analysis (d = 1, loose coupling, PGD-10, ε = 0.01) we report the full Definition 3 decomposition alongside X and the marginal adversarial excess ∆Xadv = Xadv − Xrand (additive difference, avoiding the 0/0 indeterminacy that arises when Xrand = 0). All bootstrap tests use B = 10,000 resamples; p-values are upper-bounded at p < 0.0001 when zero null hits are observed. Results. Table 6 reports the primary decomposition and the matched-random baseline. Table 7 shows the saturation profile with 95% bootstrap CIs. Table 8 summarises FGSM mechanistic diagnostics. Table 6: E2 decomposition (PGD-10 and FGSM vs. matched-random baseline, d = 1 loose, n = 320). PGD-10 primary analysis at R = 0.000 at all ε = 0.01: |Q+ | = 43 neural-flipped queries. E R|N Wilson 95% CI: [0.918, 1.000]. Matched-random baseline: Erand ε, confirming adversarial specificity. pbootstrap < 0.0001 (one-sided, B = 10,000). Tight coupling fully suppresses symbolic flips. ∗ Q+ = 0 / since E N = 0, so E R|N is undefined (zero denominator). † At ε = 0.08 FGSM produces E N = 0 but E R|¬N = 0.334 via probabilistic threshold drift—X is undefined since E N = 0; the symbolic flips are ProbLog-sensitivity-driven, not neural-errorpropagation. § ∆Xadv = Xadv − Xrand (additive excess); Xrand = 0.000 at all ε, so ∆Xadv = Xadv numerically. The additive form avoids 0/0 indeterminacy under the ratio definition. Condition
EN
ER
E R|N
E R|¬N
X [95% CI]
∆Xadv §
FGSM (ε≤0.05) FGSM (ε=0.08) PGD-10 loose PGD-10 tight
0.000 0.000 0.134 0.134
≤0.122 0.334 0.791 0.000
– undef.∗ 1.000 0.000
– 0.334 0.758 0.000
undef. undef.† 5.884 [4.64, 8.00] 0.000
– – 5.884 0.000
1 The split is random-shuffled by generated pair list (confirmed via artifact_metadata.json), not template-stratified. Both splits receive uniform template coverage, so no data leakage arises.
21
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 7: E2 saturation profile (PGD-10, d = 5 loose, n = 320, 95% bootstrap CIs). X is monotone decreasing in ε; at ε ≥ 0.03 the CI lower bound barely exceeds 1 (near-saturation). All pbootstrap < 0.0001 (one-sided, B = 10,000). Five simultaneous ε-level tests (m = 5) were conducted; applying Holm–Bonferroni step-down correction [35] with family-wise α = 0.05 yields per-test thresholds in [0.010, 0.050], all satisfied by the observed p < 0.0001 at every level. ε
EN
ER
X [95% CI]
0.01 0.02 0.03 0.05 0.08
0.134 0.738 0.872 0.959 0.925
0.791 0.975 0.994 1.000 1.000
5.884 [4.64, 8.00] 1.322 [1.25, 1.41] 1.140 [1.10, 1.19] 1.042 [1.02, 1.07] 1.081 [1.05, 1.12]
Table 8: E2 FGSM mechanistic diagnostics (n = 320, ε grid). FGSM produces E N = 0 at all ε ≤ 0.08. Clean model confidence is high (mean 0.866), median decision margin is 0.743, and the near-boundary fraction (margin < 0.05) is 0.000—explaining FGSM’s failure: all samples lie far from the decision boundary. FGSM confidence drop (≤0.152) is 2.8× smaller than PGD’s at matched ε (0.422), confirming PGD’s superior embedding-space effectiveness. ε
N EFGSM
N EPGD
Conf. drop (FGSM)
Conf. drop (PGD)
Near-bdry frac.
0.01 0.02 0.03 0.05 0.08
0.000 0.000 0.000 0.000 0.000
0.134 0.738 0.872 0.959 0.925
0.039 0.061 0.078 0.107 0.152
0.299 0.381 0.431 0.426 0.422
0.000 0.000 0.000 0.000 0.000
Findings: FGSM failure mechanism. Clean model confidence is high (mean 0.866) and the median decision margin is 0.743, with zero queries in the near-boundary region (margin < 0.05). FGSM’s one-shot gradient step produces a confidence drop of at most 0.152 at ε = 0.08, compared to 0.422 for PGD at the same budget. DistilBERT’s subword embedding lookup discretises FGSM steps, while PGD’s iterative projection accumulates effective perturbation. FGSM is therefore structurally below the neural-flip threshold for this architecture. However, at ε = 0.08, FGSM does produce E R|¬N = 0.334 symbolic flips without any neural classification change—this is a distinct mechanism: large-magnitude perturbations shift the logit distribution enough to cross the ProbLog probabilistic threshold (h1 :- monitor, uncertain) without flipping the argmax label. This symbolic-only pathway is ProbLog-sensitivity-driven and R = 0.000 at the same ε). remains adversarially specific (Erand Adversarial specificity of X . The matched-random baseline resolves the causal attribution question raised in R = 0.000 symbolic flips—identical V6 review: random perturbations at the same ℓ∞ budget (ε = 0.01) produce Erand to the unperturbed baseline. This confirms that X = 5.884 is adversarially specific: the PGD perturbations direct input distributions toward the ProbLog sensitivity region in a way that random noise of equal magnitude does not. The full decomposition clarifies the mechanism: |Q+ | = 43 neural-flipped queries all propagate to symbolic flips (E R|N = 1.000, Wilson 95% CI [0.918, 1.000]), and E R|¬N = 0.758 means 75.8% of symbolic flips occur without a neural flip—the adversarial perturbation moves input logit distributions enough for h1 :- monitor, uncertain to fire without crossing the neural decision boundary. Both pathways are adversarially induced: the marginal adversarial excess ∆Xadv = Xadv − Xrand = 5.884 − 0.000 = 5.884 equals X itself because the random baseline contributes zero R = 0), confirming (using the additive formulation to avoid 0/0). Tight coupling fully suppresses both pathways (Etight probabilistic marginalisation as a robust architectural defence. Saturation and depth. The saturation sweep (Table 7) confirms monotone decline from X = 5.884 at ε = 0.01 to 1.042 at ε = 0.05. Five ε-level tests are conducted simultaneously; after Holm–Bonferroni step-down correction [35] (α = 0.05, m = 5) every comparison remains significant at p < 0.0001, controlling the family-wise error rate. The depth sweep (d = 1–10, up to 21 ProbLog rules) is a confirmed null result: X is depth-invariant because the adversarial 22
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
perturbations tested do not trigger the multi-hop chained rules. A meaningful positive test would require inputs crafted to enter the chained inference region (two or more rule applications en route to the hazard conclusion). 11.3
E3: Ontology-Merging Attack on Clinical-Safety and STIX Ontologies
Setup. We constructed 30 adversarial attack templates across two domain-relevant OWL ontologies: a clinical-safety ontology (15 templates, c1–c15) and a STIX cyber-threat ontology (15 templates, s1–s15), both loaded via Owlready2 with dual consistency checking under Pellet and HermiT. Each template inserted one OWL axiom (equivalence assertions or subclass overrides, all single-axiom budget) to subvert safety-critical classification: contraindication/safe-medication reasoning in the clinical ontology; malware/benign threat attribution in the STIX ontology. SIV was evaluated by comparing post-merge query answers to the clean baseline. Stealthiness was measured by Pellet-consistency of the merged ontology. A differential safety-query detector ran after each merge. Wilson 95% binomial confidence intervals were computed per ontology and overall. Scope note. The 93.3% SIV rate reported here is scoped to single-axiom equivalence and subclass attacks, which are the templates tested. This figure should not be extrapolated to OWL axiom types not covered by these 30 templates (e.g., property restrictions, cardinality constraints). To assess generalisation, we split the 30 templates into 14 in-sample (seen during detector training) and a 16-template held-out set. The split was stratified by axiom kind (7 is_a and 7 equivalent_to in-sample; 8 is_a and 8 equivalent_to held-out) and balanced across ontologies (7 clinical and 7 STIX in-sample; 8 clinical and 8 STIX held-out), ensuring the held-out set is representative on both dimensions. Wilson 95% CIs are reported for held-out performance. Results. Table 9 reports per-ontology outcomes for in-sample and held-out detection. 28 of 30 templates succeeded (2 failed to apply: c10, s9). Table 9: E3 ontology-merging results: in-sample and held-out detection recall on clinical-safety and STIX OWL ontologies (30 templates, 1-axiom budget each). 28/30 templates induced SIV (in-sample 100% detection). Held-out detection recall is 67.9% overall [49.3%, 82.1%], with a domain gap: clinical 85.7% vs. STIX 50.0%. Ontology
Templates
SIV rate
100% Stealth
In-sample det.
Held-out det. [95% CI]
Clinical (c1–c15) STIX (s1–s15)
15 15
93.3% 93.3%
✓ ✓
100% 100%
85.7% [60.1%, 96.0%] 50.0% [26.8%, 73.2%]
Overall
30
93.3%
✓
100%
67.9% [49.3%, 82.1%]
Findings. 28 of 30 templates induced SIV (93.3%, 95% CI [78.7%, 98.2%], scoped to single-axiom equivalence/subclass attacks). All 28 successful attacks were Pellet-consistent (100% stealth rate): the merged ontology appeared valid to the OWL-DL reasoner while producing incorrect safety classifications. HermiT agreement was 100% for all executed templates. Two templates (c10: highrisk_is_routine, s9: critical_is_phish) failed to apply due to HermiT-detected inconsistencies, indicating that some aggressive axiom choices are detectable at merge time. In-sample detection recall was 100% on all 28 successful templates. Held-out detection recall was 67.9% overall (95% CI [49.3%, 82.1%]), revealing a significant generalisation gap. The gap is strongly domain-dependent: clinical held-out recall is 85.7% (95% CI [60.1%, 96.0%]), while STIX held-out recall is 50.0% (95% CI [26.8%, 73.2%]). Table 10 breaks down STIX held-out recall by axiom kind.
23
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 10: E3 STIX held-out detector failure analysis by axiom kind. Both axiom types show sub-chance or near-chance heldout recall. Seven failing STIX templates identified: s1–s4, s5–s7 (benign/malware equivalences, crime/nation misclassifications, campaign/actor conflations). Axiom kind
Templates
In-sample recall
Held-out recall
is_a equivalent_to
7 8
85.7% 100%
42.9% 50.0%
STIX total
15
100%
50.0%
A held-out recall of 50% is statistically indistinguishable from random binary guessing (Wilson CI for STIX includes 50%) and constitutes a detector failure, not a gap. Both axiom kinds fail: is_a recall drops to 42.9% held-out (worse than random); equivalent_to reaches only 50.0%. The seven failing templates (s1–s7: benign/malware equivalences, crime/nation, campaign/actor, ransom/benign conflations) represent diverse semantic attack patterns that share no structural overlap with the clinical training templates. The failure mechanism is semantic rather than syntactic: the detector learned clinical-domain query patterns (medication/contraindication semantics) that do not transfer to STIX threat-attribution semantics. Remediation requires either (a) STIX-specific template augmentation covering the failing axiom patterns, or (b) a domain-agnostic syntactic detector operating on OWL axiom structure rather than semantic query matching. This STIX failure is a concrete, high-priority limitation requiring resolution before deploying this detector in cyber-threat ontology settings. These results confirm SIV (Definition 2) and the supply-chain attacker profile on domain-relevant symbolic reasoning tasks, while honestly characterising detector generalisation limits. Summary. Three findings emerge. First, targeted KG poisoning reaches SIV > 5% at break-even B=5, 14.6× more efficient than random, with a KG-specific stealth/targeted trade-off. Second, X =5.884 (95% CI [4.64, 8.00], p<0.0001) is adversarially specific—a matched-random baseline produces zero symbolic flips at the same budget, ruling out spurious sensitivity as the explanation. Third, single-axiom OWL attacks achieve 93.3% SIV success with 100% stealth (in-sample), but held-out detection is 67.9% and the STIX detector fails at 50%—random-guessing level. Defence-in-depth (provenance signing, statistical monitoring, canary queries) remains essential for all attacker profiles.
12
Conclusion
Neuro-symbolic AI offers a compelling path toward safer and more trustworthy AI by combining data-driven perception with explicit, auditable reasoning. Yet NeSy does not magically solve safety or security: it shifts, enlarges, and differentiates the attack surface. Adversaries can exploit neural, symbolic, orchestration, and cognitive dimensions, and must be countered with integrated mitigations spanning ATLAS-aligned technical controls, OWASP-informed application hardening, and cognitive-security-aware design and governance [1, 9, 3, 26, 13, 8, 17]. Our empirical validation (Section 11) confirms three headline findings on domain-relevant components. First, targeted KG poisoning is operationally accessible at low budget: on a 205-entity, 622-triple medical contraindication KG, targeted injection reaches the SIV detection threshold at break-even budget B = 5. A stealth/targeted trade-off is identified: stealth attacks achieve zero SIV under this KG’s architecture—an architecture-specific operational cost of detection evasion under this relation schema. No single detector covers all attacker profiles—provenance recall is 0.0 for insider-targeted attacks, canary recall is 0.0 for stealth, and statistical detection fires only at high budget (Table 5)—so defence-in-depth combining all three is strongly recommended. Second, we provide the first empirical measurement of X (Definition 3) on a neural–symbolic pipeline, establishing that super-unity amplification is achievable at realistic perturbation budgets: PGD-10 at ε = 0.01 produces X = 5.884 (95% CI [4.64, 8.00], pbootstrap < 10−4 , one-sided) on a DistilBERT medical claims + ProbLog pipeline. The X decomposition (Definition 3) reveals that 75.8% of symbolic flips arise without any neural flip (E R|¬N = 0.758)—autonomous ProbLog sensitivity is the dominant source of super-unity amplification, not neural error propagation. Peak amplification occurs at low ε (saturation regime at 24
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
ε ≥ 0.03). Third, single-axiom OWL ontology edits achieve 93.3% SIV success (scoped to equivalence/subclass templates) with 100% Pellet-consistency stealth and 100% in-sample detection, confirming that standard reasoner checking is insufficient. However, held-out detection recall is 67.9% (95% CI [49.3%, 82.1%]), with a pronounced clinical/STIX domain gap (85.7% vs. 50.0%), which is an open problem for future work. For high-stakes domains such as healthcare, cybersecurity, public welfare, and mental health, organisations should treat NeSy simultaneously as an opportunity and a liability: powerful instruments that demand rigorous threat modelling, lifecycle assurance, and interdisciplinary oversight involving security engineers, domain experts, cognitive scientists, and ethicists [3, 4, 6, 7]. Future work should pursue: (i) completing the E4 cognitive safety user study to empirically quantify automation bias amplification by NeSy explanations; (ii) scaling E1 and E3 to larger, domain-specific KGs (SNOMED-CT, STIX/ATT&CK); (iii) developing automated tools for KG integrity validation and ontology provenance tracking that cover all five attacker profiles; (iv) extending Definition 3 to fully differentiable NeSy systems—DeepProbLog, Neural Theorem Provers, and Logical Neural Networks—where the neural–symbolic boundary is implicit within a shared computation graph, requiring a reformulation of X in terms of gradient-flow rather than discrete flip rates; and (v) designing differentiable NeSy architectures whose safety invariants are provably preserved under gradient-based training.
Broader Impact
This paper analyses safety, security, and cognitive risks in NeSy systems. We discuss the positive and negative societal impacts of this work candidly. Positive impacts. By formally defining the NeSy attack surface and attacker taxonomy, this work enables practitioners to perform more rigorous threat modelling before deploying NeSy systems in healthcare, public welfare, or cybersecurity. The CREST-aligned checklist with measurable acceptance criteria provides a concrete governance template that reduces the risk of under-specification when meeting obligations under the EU AI Act and NIST AI RMF [23, 15]. Explicit treatment of cognitive security risks—automation bias, authority bias, and sycophantic reinforcement—raises awareness among designers and regulators of a class of harms that is distinct from classic adversarial ML and currently absent from standard threat catalogues. Making these risks legible to non-technical stakeholders (operators, auditors, clinicians) should lower the probability of harm in high-stakes deployments [6, 13]. Negative impacts and dual-use considerations. Publishing a detailed attacker taxonomy and formal threat definitions provides a knowledge resource that malicious actors could use to design more effective symbolic-layer attacks—specifically KG poisoning, ontology-merging attacks, and inference-engine subversion. We judge that the defensive benefit to the larger population of NeSy system builders outweighs this risk, consistent with responsible disclosure norms in the security community. However, we deliberately omit operational details (e.g., specific KG injection payloads or step-by-step rule-poisoning procedures) that would provide concrete attack uplift without corresponding defensive value. Limitations and scope. Table 11 summarises the principal limitations of each experiment, the underlying cause, and the recommended mitigation or future work path.
25
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
Table 11: Structured limitations summary with mitigation paths. Exp.
Limitation
Cause
Mitigation / Future Work
E1
Synthetic KG (205 entities); stealth/targeted trade-off may not generalise X = 5.884 is PGD-specific at ε = 0.01; declines to 1.04 at ε = 0.05; depth-invariant
Single safety-critical relation type in KG schema
STIX held-out detection at 50% (random-guessing); 93.3% SIV scoped to equivalence/subclass axioms only Not yet conducted
Clinical-domain detector trained on semantics that do not transfer to STIX threat attribution
Replicate on SNOMED-CT or real EHR KG; use KGs with multiple safety-critical relation types Replicate on MedNLI; design adversarial inputs targeting multi-hop rule chains; test with domain-adapted embeddings (BioBERT) STIX-specific template augmentation; domain-agnostic syntactic detector on OWL axiom structure
IRB pending
Five attacker profiles are threat hypotheses, not empirically calibrated
No ground-truth incident dataset for NeSy-specific attacks
E2
E3
E4
General
Synthetic claims; single-step ProbLog module; multi-hop rules not triggered
Complete H1–H3 user study (Section 7); cognitive bias claims remain theoretical Collect NeSy incident corpus; calibrate profiles against real-world deployments
Acknowledgements
The author thanks the broader AI safety and neuro-symbolic research communities whose work informed this analysis.
Data and Code Availability. The synthetic medical contraindication KG, ProbLog rule sets, OWL ontology templates, and experimental scripts for E1–E3 are available from the author upon reasonable request ([email protected] Clinical ontology templates are synthetic and contain no patient data. STIX templates are derived from publicly available MITRE ATT&CK and STIX 2.1 schema definitions.
References
[1] Amit Sheth, Kaushik Roy, and Manas Gaur. Neurosymbolic AI – why, what, and how. arXiv preprint arXiv:2305.00813, 2023. URL https://arxiv.org/abs/2305.00813. Accessed 2025. [2] Daniel Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York, 2011. [3] Ritu Kumari, Manas Gaur, Amit Sheth, et al. Neurosymbolic AI for safe and trustworthy high-stakes applications. Preprints.org, 2025. URL https://www.preprints.org/manuscript/202511.1342/v1. DOI: 10.20944/preprints202511.1342.v1. [4] Safayat Bin Hakim, Muhammad Adil, Alvaro Velasquez, Shouhuai Xu, and Houbing Herbert Song. Neurosymbolic AI for cybersecurity: State of the art, challenges, and future directions. arXiv preprint arXiv:2509.06921, 2025. URL https://arxiv.org/abs/2509.06921. [5] AllegroGraph. How a neuro-symbolic AI approach can improve trust in AI applications. AllegroGraph Blog, 2024. URL https://allegrograph.com/ how-a-neuro-symbolic-ai-approach-can-improve-trust-in-ai-apps/.
26
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
[6] Microsoft Research. Towards the psychological security of agentic AI. Microsoft Research Project, 2025. URL https://www.microsoft.com/en-us/research/project/ towards-the-psychological-security-of-agentic-ai/. [7] European Data Protection Supervisor (EDPS). Neuro-symbolic artificial intelligence. EDPS TechSonar, 2024. URL https://www.edps.europa.eu/data-protection/technology-monitoring/ techsonar/neuro-symbolic-artificial-intelligence. [8] Vectra AI. MITRE ATLAS: AI security framework with 16 tactics and 84 techniques. Vectra AI Topics, 2024. URL https://www.vectra.ai/topics/mitre-atlas. OWASP top 10 2025 for LLM applications: What’s new? risks and mitiga[9] Confident AI. tion techniques. Confident AI Blog, 2025. URL https://www.confident-ai.com/blog/ owasp-top-10-2025-for-llm-applications-risks-and-mitigation-techniques. [10] Shawn Riley. Assessing gaps in MITRE ATLAS (oct 2024). LinkedIn Pulse, 2024. URL https://www. linkedin.com/pulse/assessing-gaps-mitre-atlas-oct-2024-shawn-riley-2ettc. [11] Sue Behavioural Design. System 1 and system 2 thinking explained by kahneman. Sue Behavioural Design Blog, 2024. URL https://www.suebehaviouraldesign.com/en/blog/ system-1-and-system-2-explained/. [12] Michael Varga. Cognitive security risks: AI manipulation and distorted belief – sycophantic AI. LinkedIn Post, 2024. URL https://www.linkedin.com/posts/michael-varga-6b00b3167_ breaking-sycophantic-ai-distorts-belief-activity-7434970624256532481-eNfe. [13] Blackbird AI. What is cognitive security? Blackbird AI Blog, 2024. URL https://blackbird.ai/blog/ what-is-cognitive-security/. [14] National Institute of Standards and Technology. Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). Technical report, NIST, 2024. URL https://nvlpubs.nist. gov/nistpubs/ai/NIST.AI.600-1.pdf. [15] European Union. Regulation (EU) 2024/1689 of the european parliament and of the council — artificial intelligence act. Official Journal of the European Union, 2024. URL https://eur-lex.europa.eu/eli/reg/ 2024/1689/oj/eng. [16] C. Michel-Delétie and M. K. Sarker. Neuro-symbolic methods for trustworthy AI: A systematic review. Neurosymbolic AI Journal, 2024. URL https://neurosymbolic-ai-journal.com/system/files/ nai-paper-726.pdf. [17] Manas Gaur and Amit Sheth. Building trustworthy NeuroSymbolic AI systems: Consistency, reliability, explainability, and safety. arXiv preprint arXiv:2312.06798, 2023. URL https://arxiv.org/abs/2312.06798. [18] Magnus Wiik Eckhoff, Jonas Halvorsen, Bjørn Jervell Hansen, Martin Eian, Vasileios Mavroeidis, Robert Andrew Chetwyn, Geir Skjøtskift, and Gudmund Grov. Experimenting with neurosymbolic AI for defending against cyber threats. Neurosymbolic AI Journal, 2025. URL https://neurosymbolic-ai-journal.com/ system/files/nai-paper-828.pdf. [19] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015. URL https://arxiv.org/abs/ 1412.6572.
27
Safety, Security, and Cognitive Risks in Neuro-Symbolic AI
[20] Xu Zou, Qiuling Xu, Hanghang Tong, Jiliang Tang, et al. Towards robust graph neural networks for noisy graphs with sparse labels. In ACM International Conference on Web Search and Data Mining (WSDM), 2022. URL https://arxiv.org/abs/2201.00232. [21] Ibrahim Adabara, Bashir Olaniyi Sadiq, Aliyu Nuhu Shuaibu, Yale Ibrahim Danjuma, and Maninti Venkateswarlu. A review of agentic AI in cybersecurity: Cognitive autonomy, risks, and mitigations. F1000Research, 14:843, 2025. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC12569510/. DOI: 10.12688/f1000research.169337.1. [22] Bochao Zou et al. Automation bias in human-AI collaboration: A review. AI & Society, 2025. URL https: //link.springer.com/article/10.1007/s00146-025-02422-7. [23] National Institute of Standards and Technology. Artificial intelligence risk management framework (AI RMF 1.0), NIST AI 100-1. Technical report, NIST, 2023. URL https://doi.org/10.6028/NIST.AI.100-1.
[24] Infosys. Neurosymbolic AI: Bridging neural learning and symbolic reasoning for next-generation intelligent systems. Infosys Emerging Technology Blog, 2024. URL https://blogs. infosys.com/emerging-technology-solutions/artificial-intelligence/ neurosymbolic-ai-bridging-neural-learning-and-symbolic-reasoning-for-next-generation-i html. [25] C. Michel-Delétie and M. K. Sarker. Neuro-symbolic methods for trustworthy AI. Neurosymbolic AI Journal, 2024. URL https://neurosymbolic-ai-journal.com/paper/ neuro-symbolic-methods-trustworthy-ai-systematic-review. [26] Aembit. The OWASP top 10 for LLM applications (2025): Explained simply. Aembit Blog, 2025. URL https://aembit.io/blog/owasp-top-10-llm-risks-explained/. [27] Repello AI. MITRE ATLAS framework – AI security reference. Repello AI Blog, 2024. URL https: //repello.ai/blog/mitre-atlas-framework. MITRE ATLAS framework 2026 – guide to securing AI systems. [28] Practical DevSecOps. Practical DevSecOps Blog, 2026. URL https://www.practical-devsecops.com/ mitre-atlas-framework-guide-securing-ai-systems/. [29] Evidently AI. OWASP top 10 LLM: How to test your gen AI app in 2025. Evidently AI Blog, 2025. URL https://www.evidentlyai.com/blog/owasp-top-10-llm. [30] Oligo Security. OWASP top 10 LLM, updated 2025: Examples and mitigation strategies. Oligo Academy, 2025. URL https://www.oligo.security/academy/ owasp-top-10-llm-updated-2025-examples-and-mitigation-strategies. [31] The Marketing Society. System 1 and system 2 thinking. Marketing Society Think Piece, 2024. URL https: //www.marketingsociety.com/think-piece/system-1-and-system-2-thinking. [32] Farnam Street. Daniel kahneman explains the machinery of thought. Farnam Street Blog, 2024. URL https: //fs.blog/daniel-kahneman-the-two-systems/. [33] Tredence. Can you trust an AI agent? learn cognitive AI safety guide. Tredence Blog, 2025. URL https: //www.tredence.com/blog/cognitive-ai-safety. [34] Alireza S. Ziabari, Nona Ghazizadeh, Zhivar Sourati, Farzan Karimi-Malekabadi, Payam Piray, and Morteza Dehghani. Reasoning on a spectrum: Aligning LLMs to system 1 and system 2 reasoning. arXiv preprint arXiv:2502.12470, 2025. URL https://arxiv.org/abs/2502.12470. [35] Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2): 65–70, 1979. 28