arXiv:2609.19472v1 [cs.AI] 16 Sep 2026
Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models Alizishaan Khatri
Chiquita Prabhu
Omkar Neogi
Wrynx Inc. [email protected]
Independent Researcher
Independent Researcher
Abstract—Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model’s internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000×+ larger guard models while cutting latency and compute costs. Index Terms—large language models, AI safety, activation probing, jailbreak detection, guardrails, latent representations, interpretability
which can be used to make safety determinations during inference. II. R ELATED W ORK A. External Guardrails Most production systems handle safety by adding separate guard models that check inputs and outputs [10], [11], [13]. Fig. 1 shows how this typically works.
I. I NTRODUCTION As LLMs are embedded into autonomous systems, from UAV mission planners [1] and natural language command interfaces to space operations decision-support tools [2], their capacity to produce harmful or misaligned outputs becomes a dependability concern, not merely a quality issue. Sometimes users deliberately try to elicit dangerous outputs [3] [4] [5]; other times the model stumbles into them on its own [6], [7]. Researchers have tackled this in two ways. Build-time approaches like RLHF based alignment training [8] bake safety into the model during training. These are brittle [9]. Run-time approaches monitor the model during the inference loop. The dominant run-time strategy uses external guardrails: separate models like LLaMA Guard [10], ShieldGemma [11], and WildGuard [12] that screen prompts and responses of the main LLM [13]. But these guardrails have real drawbacks (Section V-A). We ask: What if the LLM already knows when a prompt is harmful? Recent work suggests it does [14]–[16]. If that’s true, we should be able to read that signal straight from the model’s internals, minimizing the reliance on expensive external safety infrastructure. We leverage the improved understanding of the geometry of an LLM’s internal representations to train MLP probes on select activations from LLaMA-3.1-8B and evaluate on three safety benchmarks: WildJailbreak [17], Beavertails [18], and AEGIS 2.0 [19]. Our results support the notion that the model’s hidden states already encode useful safety signals
Fig. 1. External guardrail architecture. User prompts pass through a prompt filter on their way to the primary model, and responses pass through a response filter before reaching the user. Each filter requires a separate inference pass through the guard model.
This approach is the best one can do with black box access to the model. However, this has serious limitations (see Section V-A). Perhaps most importantly, external guards are blind to what happens inside the model, overlooking a very critical and information rich source of signal. Prompt injection works because injected instructions hijack the model’s attention patterns [20] before any filter even sees the output. By the time an external filter checks the response, the attack has already succeeded, and the filter has limited visibility into what happened [21]. B. Safety Representations and Probes Recent interpretability research has isolated safety information within LLMs: [15], [21] identified “safety neurons,” Zou et al. [16] showed that safety concepts appear as linear directions in hidden states, and Saglam et al. [14] achieved 96.7% precision on jailbreak detection using MLP probes. Constitutional Classifiers++ [22] demonstrated that lightweight probes on activations can match external classifiers at 40× lower cost.
C. Direct Threat Model: Adversarial Prompt Injection
Algorithm 1: Safety Probe Training
We focus on the adversarial prompt injection threat as the primary attack surface. In this threat model, an external attacker who has no access to model weights, training data, or system internals constructs malicious natural language inputs designed to cause the LLM to generate harmful, policyviolating, or operationally dangerous outputs. The attacker’s goal is to bypass the model’s alignment training, either by directly requesting harmful content or by embedding jailbreak constructions that reframe, roleplay, or obfuscate the harmful intent [17] [4] [5]. Formally, we define the threat as follows. Let M be an LLM deployed as a reasoning or command component in an autonomous system, and let A be an external adversary capable of injecting arbitrary text into M’s input context. A seeks to find a prompt p∗ ∈ P such that M(p∗ ) produces output o∗ satisfying a harmful objective H(o∗ ) = 1, where H is a harm classifier. The adversary is constrained to the input interface and has no knowledge of M’s weights or intermediate activations, also referred to as a black-box attacker. This is the most operationally realistic threat model for deployed autonomous systems, where physical or network access to model internals is restricted. The adversarial objective can be stated as:
Input: Dataset D = {(xi , yi )}, LLM M, epochs E, learning rate η Output: Trained safety probe fθ 1: for each prompt xi ∈ D do 2: hi ← M(xi )[−1] // Last-token pooling 3: Store (hi , yi ) where hi ∈ R4096 4: end for 5: Initialize MLP fθ : R4096 → {0, 1} 6: for epoch = 1, . . . , E do 7: for each batch (H, Y ) do 8: Ŷ ← fθ (H) 9: L ← CrossEntropy(Ŷ , Y ) 10: θ ← θ − η∇θ L 11: end for 12: end for 13: return fθ
p∗ = arg max H(M(p))
subject to
A. System Architecture Figure 2 shows the pipeline. Each prompt goes through LLaMA-3.1-8B, and we pull out the 4096-dimensional activation from the final layer using last-token pooling. From there, a 6-layer MLP classifies the embedding as harmful or benign.
G(p, M(p)) = 0
p ∈ PA
(1) where PA ⊆ P is the set of prompts accessible to the black-box adversary, H : O → {0, 1} is a harm classifier over the output space O, and G represents the interface-level guardrail constraint that the adversary must satisfy to avoid pre-generation interception. The constraint G(·) = 0 captures the adversary’s requirement to evade existing surface-level filters while still inducing a harmful output. Existing external guardrail approaches attempt to intercept p∗ or o∗ at the interface boundary. As we have demonstrated, a sufficiently sophisticated A can construct inputs that satisfy surface-level safety filters while still inducing harmful internal model states. Our core claim is that we can detect harmful p∗ using the model’s latent activations during processing of p∗ at a fraction of the cost of interface level filters leveraging a detection signal that is structurally inaccessible to the adversary.
Fig. 2. Probe pipeline. Stage 1 extracts 4096-dim activations from final layer of frozen LLaMA-3.1-8B. Stage 2 trains a 6-layer MLP for binary classification.
III. M ETHODOLOGY We extract hidden states from LLaMA-3.1-8B and train MLP classifiers to predict whether prompts are harmful. Algorithm 1 summarizes the two-stage training procedure. Stage 1 extracts activations: each prompt is passed through the primary LLM, and we store the final hidden state using lasttoken pooling. Stage 2 trains the classifier: the MLP learns to map stored activations to binary safety labels using crossentropy loss.
Fig. 3. Deployment architecture. Safety probes run alongside the primary model, producing per-policy risk scores without external infrastructure. Compare with external guardrails (Fig. 1).
B. Dataset Preprocessing We evaluated our probes on three separate safety datasets, each requiring different preprocessing for binary classification.
WildJailbreak [17] provides 261K prompts spanning four categories: vanilla harmful, vanilla benign, adversarial harmful, and adversarial benign. We focus on the vanilla subset (roughly 100K examples) because our goal is classifying prompt intent, not detecting adversarial jailbreak attacks. We label harmful prompts as 1 and benign as 0, then split 80/10/10 for training, validation, and testing, stratified by label. Beavertails [18] contains ∼364K English question-answer pairs labeled with 14 harm categories (e.g., violence, hate speech, discrimination) and a binary is_safe flag. The dataset includes structured train/test splits and a curated evaluation set of ∼700 prompts for benchmarking. AEGIS 2.0 [19] includes 26K human-LLM interactions labeled across 12 hazard categories such as violence, hate speech, and self-harm. Some sensitive prompts are redacted in the public release but can be reconstructed using IDs linked to the Kaggle Suicide Watch dataset; we reconstruct these to ensure complete coverage. Unsafe prompts are labeled 1, safe prompts 0, using the provided 80/10/10 train/validation/test splits. C. Classifier Architecture TABLE I MLP ARCHITECTURE (12.6M PARAMETERS , 0.16% OF LL A MA-8B). Layer
In
Out
Activation
1 2 3 4 5 6
4,096 2,048 2,048 512 512 64
2,048 2,048 512 512 64 2
GELU + Dropout GELU + Dropout GELU + Dropout GELU + Dropout GELU + Dropout Softmax
Following Saglam et al. [14], we use a 6-layer MLP with progressively decreasing dimensions (Table I). The classifier contains 12.6M parameters, only 0.16% of LLaMA’s 8B. We train with AdamW [23], learning rate 2.5 × 10−4 , batch size 1024, for 50 epochs.
TABLE II P ROBE PERFORMANCE ACROSS DATASETS .
WildJailbreak Beavertails AEGIS
Model
Params
BeaverT
AEGIS 2.0
BeaverDam MD-Judge GPT-4 WildGuard LlamaGuard3 LlamaGuard2 Aegis-Guard
7B 7B ∼1.8T 7B 8B 8B 7B
89.9 86.7 86.1 84.4 – 71.8 74.7
– – – 81.9 77.3 76.8 –
Our Probe
12.6M
82.7
83.5
Table III compares our probe against established guard models. On BeaverTails, our 12.6M-parameter probe achieves 82.7% F1, competitive with 7B models like WildGuard (84.4%) and MD-Judge (86.7%), while substantially outperforming LlamaGuard2 (71.8%). On AEGIS 2.0, we achieve 83.5% F1, surpassing WildGuard (81.9%), LlamaGuard3 (77.3%), and LlamaGuard2 (76.8%). Top performers like BeaverDam (89.9%) and GPT-4 (86.1%) score higher on BeaverTails but they are also over 1000× larger. We note that BeaverTails evaluates response harmfulness while AEGIS 2.0 evaluates prompt classification. We omit WildJailbreak because Han et al. [12] and others evaluate on WildGuardTest, a different test set than the vanilla split we use. V. D ISCUSSION Our results demonstrate that lightweight probes on LLM hidden states can detect harmful content with high accuracy across diverse safety benchmarks. To contextualize these findings, we first examine the limitations of external guardrails, then discuss how probe-based approaches address them. A. Limitations of External Guardrails Latency and Cost. Guardrails require dedicated GPUs and add latency to every request. Total latency includes both compute and network overhead: Lguard = Tpf + Tllm + Trf + 2Tnet
IV. R ESULTS
Dataset
TABLE III C OMPARISON WITH GUARD MODELS . B EAVERTAILS RESULTS FROM H AN ET AL . [12]; AEGIS 2.0 RESULTS FROM G HOSH ET AL . [19].
Precision
Recall
F1 (%)
0.997 0.841 0.858
0.985 0.814 0.814
99.1 82.7 83.5
Table II shows the performance of our probes on LLaMA3.1-8B. We achieve 99.1% F1 on WildJailbreak, 82.7% F1 on Beavertails and 83.5% F1 on AEGIS datasets. The precision and recall numbers for each one of these datasets are also within a 5% range of each other, highlighting stable detection performance.
(2)
where Tpf , Tllm , Trf are the response times of the prompt filter, the LLM, and the response filter respectively. These operations introduce sequential dependencies: the LLM must finish before the response filter can evaluate its output, and both prompt and response filters must finish running before the result can be sent to the user. Each filter also incurs network latency when deployed as an external service. Compute overhead alone can add significant latency. Albrethsen et al. [24] report latency numbers between 4–1430 ms per turn for different guard models. The network latencies are implementation specific and grow with payload size. The latency overhead has motivated a growing line of research on shrinking guard models [25], [26]. Zheng et al. [25] highlight an approach to shrink the guard model from 7B to 67M parameters to achieve acceptable latency while trading off on safety.
Adversarial Vulnerability. Guardrails remain vulnerable despite years of development. Adversarial techniques like character injection achieve up to 100% evasion rates against commercial systems including Microsoft Azure Prompt Shield and Meta Prompt Guard [27]–[31]. Opacity to Internal Mechanisms. Guard models see inputs and outputs, nothing in between. During a Prompt injection attack, injected instructions hijack the model’s internal representations including the attention layer. [20], and hidden states encode malicious intent before any tokens are generated [14], [32]. By the time an external filter sees the response, the attack has already succeeded. This creates a fundamental asymmetry of information. The attacker operates entirely in the input space — crafting prompts that satisfy surface-level filters while steering internal representations toward harmful generation — and never needs to observe the model’s internals to succeed. The defender, relying solely on external guardrails, is constrained to the same input-output interface as the attacker and so holds no informational advantage.
probes have 12.6M parameters, less than 0.2% of the base model, and need no external infrastructure. The bottom line is that safety-relevant information already lives in LLM hidden states, organized in linearly separable form. Simple classifiers can tap this structure directly, providing a fast and lightweight alternative to external guardrails by re-using safety signals within the primary model.
B. Advantages of Probe-Based Detection
Large language models are increasingly deployed in realworld applications where failures can produce harmful or policy-violating outputs. This work introduces latent space probes, a class of model-native runtime defenses that analyze internal model activations to detect safety-relevant concepts during generation. By operating directly on internal representations, such defenses may enable faster and more scalable detection of unsafe behaviors, including prompt injection attacks and harmful content, while avoiding the latency and computational costs of external moderation systems. This approach could help improve the reliability and safety of AI systems deployed in production. However, activation-based detection systems may inherit biases present in underlying models or datasets, which could lead to disproportionate moderation of certain linguistic styles or topics. In addition, such mechanisms could be misused for overly restrictive filtering if deployed without transparency or appropriate governance. Careful evaluation across diverse datasets and responsible deployment practices are therefore important to mitigate these risks.
Our approach (Fig. 3) addresses each of these limitations directly. Minimal Latency. Probes integrate into the forward pass rather than running separately: Lprobe = max(Tllm , Tprobe )
(3)
While guardrail latency is additive ((2)), probe latency uses max because the probe executes concurrently with generation on the same host. Since the number of probe parameters (12.6M) ≪ number of primary model parameters (8B), it follows that Tprobe ≪ Tllm . The probe is effectively “free.” In practice, we observed that the probes ran in under 1 ms while typical generation took 50–500 ms. Our probes do not require separate GPUs for running guard models and eliminate network transport costs entirely. Early Termination. Unlike external guardrails that must wait for complete responses, probes can flag harmful content during generation and terminate immediately, saving compute and limiting user exposure to unsafe content. Access to Internal State Probes operate on hidden representations where harmful intent is encoded. The defender gains access to the model’s internal activations, a signal that is both richer than the input-output interface and structurally inaccessible to a black-box adversary. Under this information symmetry driven framing, probing hidden states is not merely an efficiency improvement over external guardrails — it is a shift in the defender’s observability that the attacker cannot trivially counter without white-box access to the model. VI. C ONCLUSION We showed that lightweight probes trained on frozen LLM activations detect harmful content effectively. On three benchmarks, our approach achieved F1 scores of 99% (WildJailbreak), 83% (Beavertails), and 84% (AEGIS 2.0). These
L IMITATIONS Our experiments focus exclusively on LLaMA-3.1-8B; we have not validated cross-model transfer. We extract only final-layer activations. The earlier layers may encode safety information differently. Our probes predict binary labels; real deployments may require finer-grained risk scores. Future directions include analyzing activations at different layers and tokens to trace how harmful intent emerges, evaluating adversarial robustness, and expanding to other model architectures and modalities. I MPACT S TATEMENT
R EFERENCES [1] W. Xiao, C. Shi, M. Chen, A. V. Vasilakos, M. Chen, and A. Farouk, “Llm-based uav path planning for autonomous and adaptive industry systems,” ACM Transactions on Autonomous and Adaptive Systems, 2025. [2] L. Yuan, C. Deng, D.-J. Han, I. Hwang, S. Brunswicker, and C. G. Brinton, “Next-generation llm for uav: From natural language to autonomous flight,” arXiv preprint arXiv:2510.21739, 2025. [3] Z. Niu, H. Ren, X. Gao, G. Hua, and R. Jin, “Jailbreaking attack against multimodal large language model,” arXiv preprint arXiv:2402.02309, 2024. [4] M. Andriushchenko, F. Croce, and N. Flammarion, “Jailbreaking leading safety-aligned llms with simple adaptive attacks,” arXiv preprint arXiv:2404.02151, 2024. [5] P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer et al., “Jailbreakbench: An open robustness benchmark for jailbreaking large language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 55 005–55 029, 2024.
[6] A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043, 2023. [7] A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?” in Advances in Neural Information Processing Systems, vol. 36, 2024. [8] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 27 730– 27 744. [9] R. Greenblatt, C. Denison, B. Wright, F. Roger, M. MacDiarmid, S. Marks, J. Treutlein, T. Belonax, J. Chen, D. Duvenaud et al., “Alignment faking in large language models,” arXiv preprint arXiv:2412.14093, 2024. [10] H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa, “Llama guard: Llmbased input-output safeguard for human-ai conversations,” arXiv preprint arXiv:2312.06674, 2023. [11] W. Zeng, Y. Liu, R. Mullins, L. Peran, J. Fernandez, H. Harkous, K. Narasimhan, D. Proud, P. Kumar, B. Radharapu et al., “Shieldgemma: Generative ai content moderation based on gemma,” arXiv preprint arXiv:2407.21772, 2024. [12] S. Han, K. Rao, A. Ettinger, L. Jiang, B. Y. Lin, N. Lambert, Y. Choi, and N. Dziri, “Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms,” in Advances in Neural Information Processing Systems, 2024. [13] Y. Dong, R. Mu, G. Jin, Y. Qi, J. Hu, X. Zhao, J. Meng, W. Ruan, and X. Huang, “Building guardrails for large language models,” arXiv preprint arXiv:2402.01822, 2024. [14] B. Saglam, P. Kassianik, B. Nelson, S. Weerawardhena, Y. Singer, and A. Karbasi, “Large language models encode semantics and alignment in linearly separable representations,” in Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, K. Inui, S. Sakti, H. Wang, D. F. Wong, P. Bhattacharyya, B. Banerjee, A. Ekbal, T. Chakraborty, and D. P. Singh, Eds. Mumbai, India: The Asian Federation of Natural Language Processing and The Association for Computational Linguistics, Dec. 2025, pp. 2282–2303. [Online]. Available: https://aclanthology.org/2025.ijcnlp-long.124/ [15] J. Chen, X. Wang, Z. Yao, Y. Bai, L. Hou, and J. Li, “Towards understanding safety alignment: A mechanistic perspective from safety neurons,” in The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [Online]. Available: https://openreview.net/forum?id=AAXMcAyNF6 [16] A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski et al., “Representation engineering: A top-down approach to ai transparency,” arXiv preprint arXiv:2310.01405, 2023. [17] L. Jiang, K. Rao, S. Han, A. Ettinger, F. Brahman, S. Kumar, N. Mireshghallah, X. Lu, M. Sap, Y. Choi et al., “Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 47 094–47 165, 2024. [18] J. Ji, M. Liu, J. Dai, X. Pan, C. Zhang, C. Bian, B. Chen, R. Sun, Y. Wang, and Y. Yang, “Beavertails: Towards improved safety alignment of llm via a human-preference dataset,” in Advances in Neural Information Processing Systems, 2023. [19] S. Ghosh, P. Varshney, M. N. Sreedhar, A. Padmakumar, T. Rebedea, J. R. Varghese, and C. Parisien, “Aegis2. 0: A diverse ai safety dataset and risks taxonomy for alignment of llm guardrails,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2025, pp. 5992–6026. [20] K.-H. Hung, C.-Y. Ko, A. Rawat, I.-H. Chung, W. H. Hsu, and P.-Y. Chen, “Attention tracker: Detecting prompt injection attacks in llms,” in Findings of the Association for Computational Linguistics: NAACL 2025, 2025, pp. 2309–2322. [21] M. Zhang, K. K. Goh, P. Zhang, J. Sun, R. L. Xin, and H. Zhang, “Llmscan: Causal scan for llm misbehavior detection,” arXiv preprint arXiv:2410.16638, 2024. [22] H. Cunningham, J. Wei, Z. Wang, A. Persic, A. Peng, J. Abderrachid, R. Agarwal, B. Chen, A. Cohen, A. Dau et al., “Constitutional
classifiers++: Efficient production-grade defenses against universal jailbreaks,” arXiv preprint arXiv:2601.04603, 2026. [23] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019. [24] J. Albrethsen, Y. Datta, K. Kumar, and S. Rajasekar, “Deepcontext: Stateful real-time detection of multi-turn adversarial intent drift in llms,” arXiv preprint arXiv:2602.16935, 2026. [25] A. Zheng, M. Rana, and A. Stolcke, “Lightweight safety guardrails using fine-tuned bert embeddings,” arXiv preprint arXiv:2411.14398, 2024. [26] I. Fedorov, K. Plawiak, L. Wu, T. Elgamal, N. Suda, E. Smith, H. Zhan, J. Chi et al., “Llama guard 3-1b-int4: Compact and efficient safeguard for human-ai conversations,” arXiv preprint arXiv:2411.17713, 2024. [27] W. Hackett, L. Birch, S. Trawicki, N. Suri, and P. Garraghan, “Bypassing llm guardrails: An empirical analysis of evasion attacks against prompt injection and jailbreak detection systems,” in Proceedings of the The First Workshop on LLM Security (LLMSEC), 2025, pp. 101–114. [28] J. Yang, Z. Zhang, S. Cui, H. Wang, and M. Huang, “Guiding not forcing: Enhancing the transferability of jailbreaking attacks on llms via removing superfluous constraints,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 19 643–19 655. [29] J. Geng, B. Yi, Z. Fei, T. Wu, L. Nie, and Z. Liu, “When safety detectors aren’t enough: A stealthy and effective jailbreak attack on llms via steganographic techniques,” arXiv preprint arXiv:2505.16765, 2025. [30] J. Fairoze, S. Garg, K. Lee, and M. Wang, “Bypassing prompt guards in production with controlled-release prompting,” arXiv preprint arXiv:2510.01529, 2025. [31] A. Reddy, A. Zagula, and N. Saban, “Autoadv: Automated adversarial prompting for multi-turn jailbreaking of large language models,” arXiv preprint arXiv:2511.02376, 2025. [32] X. W. Chia, S. L. Wong, and J. Pan, “Probing latent subspaces in llm for ai security: Identifying and manipulating adversarial states,” arXiv preprint arXiv:2503.09066, 2025.