Conceptio › Archive › arXiv CS

arXiv CSopen access

Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models

· arxiv_cs

arXiv CS · Papers · License: Open Access

Open Source ↗Direct PDF ↓

knowledge-representation reasoning

artificial intelligence, reasoning, knowledge representation

arXiv:2609.20722v1 [cs.AI] 17 Sep 2026

Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models

Frank E. Bobe III, Gregory D. Vetaw, Darshan W. Bryner, Matthew G. Cook, and Jose L. Salas-Vernis Naval Surface Warfare Center Panama City Division {frank.e.bobe, gregory.d.vetaw, darshan.w.bryner, matthew.g.cook12, jose.l.salasvernis}[email protected]

Abstract Activation steering modifies LLM behavior at inference time, but identifying where and how strongly to steer remains manual. We introduce Deep Noir, a framework that uses Logit Lens convergence and causal head-level attribution to autonomously discover optimal steering parameters. Across three scales (1B×3, 2–3B×2, 7–9B×4), our engine achieves +16.7 pts on spam at 1B (±4.7, 39 runs), with gains increasing to +21–42 pts at 7–9B across four architectures; on SST-2 sentiment, +13.1 pts with zero code changes. Mechanistic grounding enables automated discovery of intervention points that generalize across tasks and architectures: on sentiment, RepE (no head masking) fails to improve over baseline while Deep Noir improves all models (p<0.01). We further show that steering creates a predictable prompt injection attack surface whose vulnerability scales monotonically with steering magnitude. A critical finding for agent systems deploying steered classifiers.

Introduction

Activation steering modifies LLM behavior by adding direction vectors to the residual stream at inference time. While effective, current methods require manual selection of intervention layers, head subsets, and magnitudes – a process that does not scale across architectures or tasks. Recent work has automated layer selection via statistical scoring [1] or per-input adaptation [2], but these approaches lack causal mechanistic grounding for their choices. We propose Deep Noir (DN), a framework that replaces heuristic parameter tuning with diagnosticfirst engineering. The core idea is Architectural Chronometry: measuring when a model resolves a semantic concept from uncertainty to commitment by tracking token probabilities through the Logit Lens [3]. By combining this layer-wise diagnostic with causal head-level attribution patching [4], we ⃗ steering parameter search. While individual components exist in prior automate the full (L, K, M, d) work, our contribution is showing that their composition enables cross-task generalization where each component alone fails: RepE (without head masking) achieves zero improvement on sentiment, while the full pipeline succeeds (Section 4.4). Our approach aligns with the field’s pragmatic turn toward practical interpretability tools [5] and the priority of latent topology mapping [6]. DN targets a different objective than supervised classifiers: enabling training-free, model-internal discovery and control of representations rather than maximizing accuracy. A key finding motivating this work: weight-space corrections via SVD are ineffective in our experiments, likely due to LayerNorm attenuation (Section 9), making activation-space hooks the only intervention point. Scope and motivation. We evaluate Deep Noir primarily on binary classification (spam, sentiment) with extensions to reasoning and generation (Appendix J). As LLMs are increasingly deployed as Preprint.

decision-making components within autonomous agents, understanding the security properties of steering interventions becomes urgent. Our contributions: (1) A five-phase Steering Discovery Engine with recursive refinement that autonomously discovers steering parameters (+16.7 pts spam at 1B, scaling to +21–42 pts at 7–9B across 4 architectures) without manual tuning. (2) Evidence that mechanistic grounding enables cross-task generalization: DN transfers to sentiment with zero code changes (p<0.01 all models), while RepE which lacks head masking fails to improve over baseline on all 15 folds in this setting. (3) Characterization of containment boundaries via MMLU: containment depends on semantic distance between the steered concept and the evaluation domain. (4) Discovery that steering creates a predictable prompt injection attack surface whose vulnerability scales monotonically with magnitude and is architecture-dependent directly relevant to agent security.

Related Work

Representation Engineering. RepE established contrastive activation steering. Subsequent work diversified the approach: CAA , ActAdd , and SAE-guided steering [7]. Surveys [8] identify capability preservation and automated parameter selection as open challenges. DeepMind’s negative results on SAEs for downstream safety tasks [9] validate our use of contrastive centroid directions over more complex feature decomposition. Automated Layer Selection. LayerNavigator [1] scores layer steerability via discriminability metrics (NeurIPS 2025). SADI [2] constructs steering vectors via adaptive binary masks (ICLR 2025). FASB [10] introduces per-token steering with backtracking. ITI [11] performs head-level intervention but requires manual identification of “truthful heads.” Deep Noir automates this: we use causal mechanistic tools (logit lens + head attribution) to explain why a layer and head are chosen. InterLoRA [12] uses mech-interp for LoRA architecture design; we optimize adapter placement. Safety and Containment. Refusal is mediated by a single direction [13]. Steering can increase vulnerability [14]. Systematic evaluation [15] confirms reasoning robustness under steering (<2% entanglement). Surgical Containment analysis extends this with domain-isolation measurements.

Method: The Steering Discovery Engine

Given a pre-trained model M with L layers and a labeled probe set D = {(xi , yi )}N i=1 , we seek steering parameters (l∗ , K ∗ , M ∗ , d⃗∗ ) that maximize classification accuracy when a scaled, masked direction vector is added to the residual stream at layer l. The engine searches a discrete-continuous parameter space over layer (l ∈ {1, . . . , L}), head subset (K ∈ {1, 2, 4}), and magnitude (M ∈ [0.01, 20]). Algorithm 1 summarizes the pipeline. Phase 1: Layer ranking. For each layer l, compute two scores. First, the logit-lens differentiation: SLL (l) = |P (t | l) − P (c | l)|, where P (t | l) is the probability of the target token when projecting layer-l hidden states through the unembedding matrix. Second, the antagonist head strength: for (h) each head h in layer l, compute ah = −⟨WO h(h) , ⃗u⟩ where ⃗u = WU [t] − WU [c] is the target (h) direction in unembedding space, WO is the output projection for head h, and h(h) is the head’s output. The layer antagonist score is Sant (l) = maxh ah . Both scores are min-max normalized across layers to [0, 1] before combining: S(l) = 0.4 · ŜLL (l) + 0.6 · Ŝant (l). An ablation over 7 ratios from (0,1) to (1,0) on Llama and Gemma shows the ranking is fully invariant to the weight choice: all ratios select the same top layer with accuracy (Appendix E). The top-5 layers by S(l) advance. Phase 2: Head isolation. For each candidate layer, compute ∇h(h) L where L = − log P (yi | xi ) across 10 randomly sampled probes (5 per class). Select the top-K heads by gradient magnitude, voting across samples (K ∈ {1, 2, 4}). Construct binary mask m ∈ {0, 1}d that zeros out all dimensions except those corresponding to the selected heads. Phase 3: Direction computation. Collect hidden states at layer l+1, the representation space immediately downstream of the intervention point. This ensures the contrastive direction is expressed in the same space that the hook at layer l will modify via residual addition. Compute ⃗ d∥. ⃗ d⃗ = h̄counter − h̄target from mean states across 5–10 samples per class. Normalize: dˆ = d/∥ 2

Phase 4: Magnitude calibration. The steering intervention is ⃗v = α · M · dˆ · m, where α = 450 is a base scale factor chosen so that α · M · ∥m∥ falls within the residual stream’s ℓ2 norm range (∼200–800 at 1B; Appendix B) and M is the per-configuration magnitude. Golden-section search over M ∈ [0.01, 20.0] with coarse grid pre-scan at M ∈ {0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0}. Each evaluation applies ⃗v as a forward hook at layer l and measures labeled accuracy. Phase 5: Selection. Choose (l, K, M ) maximizing accuracy; prefer fewer heads and lower M at equal performance. Algorithm 1: Steering Discovery Engine Input: Model M, probe set D, target token t, counter token c Output: Steering config (l∗ , K ∗ , M ∗ , d⃗∗ , m∗ ) 1 for each layer l ∈ {1, . . . , L} do 2 S(l) ← 0.4 · SLL (l) + 0.6 · Sant (l); 3 C ← top-5 layers by S(l); 4 for l ∈ C, K ∈ {1, 2, 4} do 5 m ← head mask from gradient attribution on 10 probes; 6 dˆ ← normalized contrastive direction at layer l + 1; ˆ m); 7 M ∗ ← golden-section search maximizing Acc(D, l, K, M, d, ∗ ˆ 8 Store (l, K, M , d, m, Acc); 9 return config with highest Acc (ties broken by lowest K, then lowest M ); Recursive refinement. We extend single-shot discovery to an iterative loop. After the initial discovery, the engine (1) identifies remaining misclassified samples, (2) re-runs discovery with probe weights biased 2× toward errors and the previously selected layer excluded, (3) applies the new correction as an additional hook, (4) evaluates on the full probe set. If accuracy drops, the correction is rolled back. This repeats until convergence (accuracy stable for 2 iterations) or a maximum of 5 iterations, discovering complementary corrections across different layers. LayerNorm washout. Our initial approach attempted closed-form weight-space corrections: compute the desired activation delta, then derive a rank-r update to Wo proj via SVD such that ∆W · h ≈ ⃗v . This produced zero accuracy improvement across all configurations. The root cause: LayerNorm normalizes activations after the residual addition, attenuating small weight perturbations by ∼1000×. Specifically, the SVD corrections produced weight deltas of 0.01–0.1% of ∥Wo proj ∥F , corresponding to <0.001 logit shifts after LayerNorm—3–4 orders of magnitude below the 2–4 logit shifts required to flip decisions. This failure mode is confirmed independently by [16], who show that removing LayerNorm enables direct weight editing. Activation-space hooks bypass this bottleneck entirely because they inject perturbations after the attention output but before LayerNorm normalization at the next layer.

Experiments

Setup. Four architectures spanning distinct design choices (Table 8): Llama-3.2-1B (GQA, 16L/32H), OLMo-1B (Full attention, 16L/16H), Gemma-3-1B-IT (GQA, 26L/8H), Mistral-7B (32L,32H). Five spam datasets: Enron (30K corporate emails), SMS Spam Collection (5.6K messages [17]), Phishing (10K phishing/legitimate), SpamAssassin (6K public corpus), and Ultimate (combined multi-source, 11K). For cross-task evaluation: SST-2 binary sentiment [18] (872 validation samples). For Enron and Ultimate we run 5-fold CV; for SMS, Phishing, and SpamAssassin we run fold-0 only due to compute constraints, yielding 3 × (2 × 5 + 3 × 1) = 39 total discovery runs. Each fold uses 50 labeled probes for discovery; comparison experiments use 100 probes with strict separation between discovery and evaluation sets (held-out fold evaluation in Section 5). Evaluation metric. we classify by comparing the model’s next-token log-probability for the target vs. counter tokens (“spam”/“ham” for spam, “positive”/“negative” for sentiment) given a taskappropriate prompt; accuracy is the fraction of correctly classified probe samples. Single Quadro RTX 5000 (16GB VRAM). Total compute: ∼210 GPU-hours including scaling experiments and preliminary work (Appendix M). 3

Prompting baselines. On Enron fold-0, standard prompting achieves 49–52%, chain-of-thought 44–56%, and 5-shot 40–78% across models (Appendix C). Gemma’s 78% 5-shot result is competitive, but prompting requires per-model prompt engineering and is not interpretable. Deep Noir’s value is in the autonomous, mechanistically grounded discovery process. 4.1

Discovery Results

Table 1: Deep Noir Autonomous Steering Discovery across tasks and scales (5-fold CV, 50 probes per fold). 1B: 39 spam runs (3 models × 13 dataset-folds). 2–9B: 5-fold per model. 7B+ models use 4-bit quantization. 95% CIs via bootstrap. Task

Model

Base

Steered

Gain

Impr.

Spam (1B)

Llama-3.2-1B Gemma-3-1B OLMo-1B Overall

57.7 64.0 58.5 60.1

78.6 79.8 71.7 76.7

+20.9 +15.8 +13.2 +16.7

±8.1 ±8.9 ±7.7 ±4.7

12/13 11/13 10/13 33/39

Sentiment (1B)

Llama-3.2-1B OLMo-1B Gemma-3-1B Overall

77.6 55.2 75.6 69.5

89.2 72.4 86.0 82.5

+11.6 +17.2 +10.4 +13.1

±3.0 ±3.9 ±5.8 ±3.0

5/5 5/5 5/5 15/15

Spam (2B) Spam (3B) Sentiment (2B) Sentiment (3B)

Gemma-2-2B Llama-3.2-3B Gemma-2-2B Llama-3.2-3B

53.2 55.6 92.0 82.4

83.6 83.2 93.6 92.4

+30.4 +25.6 +1.6 +10.0

±9.4 ±8.5 ±1.3 ±1.9

5/5 5/5 5/5 5/5

Spam (7–9B)

Gemma-2-9B Llama-3.1-8B Mistral-7B OLMo-7B Llama-3.1-8B Mistral-7B OLMo-7B Gemma-2-9B

52.0 53.6 54.4 41.2 92.0 87.6 84.8 94.8

94.4 82.8 76.4 62.4 94.0 91.6 92.0 95.2

+42.4 +29.2 +22.0 +21.2 +2.0 +4.0 +7.2 +0.4

±3.4 ±6.7 ±5.2 ±6.3 ±1.6 ±1.6 ±1.8 ±0.7

5/5 5/5 5/5 5/5 5/5 5/5 5/5 5/5

Sentiment (7–9B)

Example. On Gemma-3-1B/Enron fold-0, Phase 1 ranks layer 20 highest (Ŝant =0.82, ŜLL =0.31). Phase 2 isolates head 3 (K=1, dh =256). Phase 3 computes a contrastive direction from 10 samples. Phase 4’s golden-section search finds M ∗ =1.0. The perturbation is applied only within the K=1 head’s 256 dimensions (22% of Gemma’s 1152-dim residual stream); the remaining 896 dimensions are unmodified. Result: 50%→86% (+36 pts) in 17 minutes. The discovered parameters differ markedly from Llama on the same dataset (L=4, K=1, M =0.2), illustrating how the engine adapts to each architecture’s topology. Each architecture discovers distinct preferred layers (Figure 1): Llama 2–14 (6 unique layers), OLMo 9–15 (7 unique), Gemma 9–24 (8 unique). This “correction topology” is stable: seed sensitivity (5 seeds) shows 4/5 select the same layer; accuracy gain is 14.0% ± 7.2% (Appendix E). 4.2

Comparison with RepE and CAA

Table 2: Enron Spam: DN vs. RepE and CAA (5-fold CV, 100 probes, accuracy %). CAA is singlerun with 50 probes (not directly comparable). We included the single-run CAA numbers to provide a familiar anchor point representing prior steering attempts, rather than as a strict 1:1 baseline. Model Base CAA RepE Deep Noir ∆RepE p Llama-3.2-1B OLMo-1B Gemma-3-1B-IT

56.8±3.9 51.6±4.1 51.2±4.1

64.0 56.0 52.0

67.6±3.4 59.0±3.4 55.6±4.9

70.4±4.5 66.4±5.7 76.4±5.2

+2.8 +7.4 +20.8

0.37 0.028 0.004

Deep Noir outperforms CAA on all 3 architectures (+6 to +24 pts) and produces higher point estimates than RepE on all 3 (Table 2). Paired t-tests confirm statistical significance for OLMo 4

Figure 1: Layer selection frequency during autonomous discovery. Each architecture exhibits a distinct correction topology. Llama favors early/mid layers, OLMo late layers, Gemma deep layers.

Figure 2: Left: per-model accuracy (baseline vs. auto-steered). Right: gain distribution by dataset. SMS shows largest gains; Phishing/SpamAssassin near ceiling.

(p=0.028, Cohen’s d=1.52) and Gemma (p=0.004, d=2.66). Llama’s advantage over RepE (+2.8 pts) is not significant (p=0.37) due to high fold-level variance. On SMS with 5-fold crossvalidation (Table 3), Deep Noir significantly outperforms RepE on all 3 architectures: Llama +6.2 (p<0.001), OLMo +14.6 (p=0.006), Gemma +20.8 (p=0.019). Table 3: SMS Spam: DN vs. RepE (5-fold CV, 100 probes, accuracy %). All 3 models significant. Model

Base

RepE

∆

Llama-3.2-1B OLMo-1B Gemma-3-1B-IT

46.0 49.4 43.8

59.8 52.8 43.8

66.0 67.4 64.6

+6.2±1.0 +14.6±4.7 +20.8±9.6

0.0004 0.006 0.019

Deep Noir exhibits higher per-fold variance (±10–11 vs. ±5–7 for RepE) because per-fold discovery produces different layer/head configurations. The key advantage over RepE is interpretability: discovered (L, K, M ) parameters have causal mechanistic meaning, enabling the containment analysis (Section 4.5) and injection analysis (Section 4.6) that RepE’s opaque vectors cannot provide. 4.3

Recursive Correction

The recursive loop diagnoses remaining errors, weights discovery toward misclassified samples, and rolls back corrections that hurt overall accuracy (Table 4). 5

Table 4: Recursive correction trajectory (Llama-3.2-1B, Enron fold-0) Step

Errors

Accuracy

Action

Layers

0 1 2 3 4

22/50 13/50 13/50 14/50 6/50

56%→74% 74%→64% 74%→72% 72%→88% 88%→74%

Added L=4 Rolled back Added L=3 Added L=9 Rolled back

[4] [4] [3, 4] [3, 4, 9] [3, 4, 9]

The recursive process discovers 3 complementary layers and reaches 88% (+32 pts from baseline). The initial single-layer correction (Step 0) reaches 74%; subsequent iterations find complementary layers that push to 88%, a +14 pt gain from multi-layer refinement. Across 33 iterative runs (Appendix F), rollback triggers in 40% of attempts, preventing an average of 12 pts accuracy loss per rolled-back step. Multi-layer steering adds +4 pts on average when single-layer is insufficient. Cross-fold generalization averages +6.6% on held-out folds, confirming the discovered steering captures genuine model properties rather than overfitting to the probe set. 4.4

Cross-Task Generalization: Sentiment Analysis

To test cross-task generalization, we run the identical engine on SST-2 sentiment with zero code changes, only token IDs differ. Table 1 (bottom) reports 5-fold results: +13.1 pts with 100% success rate (15/15 improved). In a comparison with 100 probes (Table 5), Deep Noir significantly outperforms RepE on all 3 architectures. Critically, RepE achieves zero improvement scoring exactly baseline on all 15 folds. RepE’s failure has a clear mechanistic explanation: sentiment representations are distributed across layers (L=12–19) rather than concentrated at the mid-layer where RepE applies its direction. DN’s layer ranking identifies where each task’s decision occurs.

Table 5: SST-2 Sentiment: DN vs. RepE (5-fold CV, 100 probes, accuracy %). RepE = baseline on all 15 folds. Model

Base

RepE

∆

Llama-3.2-1B OLMo-1B Gemma-3-1B-IT

77.2 55.6 76.0

88.0 69.2 83.8

+10.8 +13.6 +7.8

0.007 <0.001 0.006

The discovered sentiment layers (Llama: 12–15, OLMo: 12–14, Gemma: 18–19) differ from the spam layers (Llama: 2–14, OLMo: 9–15, Gemma: 9–24), showing the engine adapts per-task while maintaining the same architectural preference pattern (Llama mid, OLMo late, Gemma deep). We also apply the engine to multiple-choice reasoning (AQuA-RAT, MMLU) and toxicity reduction (Appendix J). Gains are limited (≤3%) because reasoning baselines are near-random at 1B–9B (22– 31% on 4–5 choice), leaving insufficient contrastive signal for the direction computation to exploit. The framework requires that the model partially encodes the target distinction; when the model cannot distinguish correct from incorrect reasoning steps, steering has nothing to amplify. Why does the composition generalize where components alone fail? RepE computes a contrastive direction and applies it at a fixed mid-layer without head masking. A strategy that works when the target concept is concentrated in the mid-layer residual stream (as spam often is). Sentiment, however, resolves later and more diffusely across layers. DN’s logit-lens scan detects where each concept resolves; head attribution identifies which heads encode it; magnitude calibration determines how strongly to intervene. This is supported by causal ablation: removing head masking drops accuracy below baseline (−5% Llama), proving the selected heads carry task-specific signal that global steering destroys; removing layer ranking yields only +9% vs. +21% with ranking; removing calibration yields −11% (Table 7). Each component has a necessary causal role that random search cannot replicate. The cross-task transfer works precisely because each phase adapts to the task’s representational structure rather than relying on a fixed heuristic. 6

4.5

Surgical Containment and Reasoning Degradation

Containment. We evaluate containment using both our custom coherence metric (117 comparisons, δ = 0.000) and MMLU (200 questions across 4 subjects). On MMLU, sentiment steering has minimal impact: ≤4% change across all models. Spam steering shows larger but mixed effects: Llama −9.5%, OLMo +8.5%, Gemma −2.5%. At 1B scale, MMLU baselines are near chance (22–29%), so deltas are noisy; the consistent finding is that sentiment steering is well-contained while spam steering shows architecture-dependent leakage on MMLU. This adds nuance to the prior custom-metric result and reconciles with reports of MMLU penalties from steering [13]: containment depends on the distance between the steered concept and the evaluation domain. At 7B (Mistral), the pattern holds: spam steering reduces MMLU by −10%, comparable to the 1B Llama result (−9.5%). Degradation. Circuit-level heuristics degrade rapidly at the 1B scale: halflives of 0–2 steps across all architectures (Llama 1.6–2.0, OLMo 1.0, Gemma 0.0). This establishes a minimum capability threshold for mechanistic reasoning correction. Per-step resolution traces in Appendix. 4.6

The Steering Injection Attack Surface

As LLMs are deployed as decision-making components in autonomous agents, classifying inputs, routing requests, authorizing actions, and steering interventions become attack surfaces. We characterize this threat across 3 models, 2 tasks, and 3 public injection benchmarks. Setup. We evaluate with (a) our 24 OWASP-aligned templates on 50 emails (3,600 trials), (b) sentiment-adapted (300 trials/model), and (c) three public benchmarks: deepset/prompt-injections [19] (116 test samples), SafeGuard (500 samples), and a jailbreak classification (500 samples). Table 6: Prompt injection: steering effect on spam classification (3,600 trials). Model

Base%

+Steer%

∆

Top Attack

Llama OLMo Gemma

21.2 13.7 15.3

35.0 16.9 35.6

+13.8 +3.2 +20.3

Delimiter escape (36%) Instr. override (13%) Encoding obfusc. (34%)

Vulnerability scales with magnitude. Define the vulnerability function V (M ) = E[1[inject succeeds | M ]] over the injection template distribution. Figure 3 shows V (M ) is monotonically non-decreasing in M for all architectures. OLMo rises from V (0)=0.34 to V (4M ∗ )=1.0; Gemma follows a similar curve (0.25 → 0.99). Combined with the accuracy function A(M ) from Phase 4, this defines a Pareto frontier {(A(M ), V (M ))} enabling principled M selection. Cross-task: architecture-dependent. On sentiment, the effect reverses for 2/3 models: Llama (−14.7%) and Gemma (−18.0%) become harder to inject under steering, while OLMo (+38.7%) becomes far more vulnerable. The critical factor is alignment between d⃗ and the attacker’s goal. External benchmarks. On the deepset injection benchmark, spam steering has mixed effects: Gemma’s injection detection improves (+4.3% accuracy), while OLMo’s collapses (−2.6%). On SafeGuard (10K samples), Gemma improves (+16.4%) while Llama degrades (−10.4%). These results confirm the architecture-dependent pattern is robust across injection datasets. Detection via activation norms. At the steering layer, injected inputs produce significantly different activation norms on Gemma (p<0.001, norm difference of −660), suggesting a viable detection signal. Llama and OLMo show no significant norm difference (p>0.27), indicating detection requires architecture-specific strategies. Scale dependence. At 7B (Mistral), steering reduces injection vulnerability from 97.5% to 61.1% (−36.4 pts) the opposite of the 1B pattern. Larger models have stronger baseline classification, and steering reinforces rather than undermines this. This suggests the injection attack surface is primarily a small-model phenomenon that diminishes with scale. Implications for agent security. For steered LLMs gating access to tools or sensitive operations: (a) vulnerability is predictable from the steering configuration, enabling targeted threat modeling before deployment, (b) the magnitude–vulnerability curve enables principled M selection that bal7

Figure 3: Injection vulnerability scales monotonically with steering magnitude. Practitioners can select M to balance accuracy gain against injection risk.

ances accuracy against injection risk, (c) activation-norm monitoring at the steering layer provides a detection signal on some architectures, and (d) defenses should operate before the steered layer.

Discussion and Limitations

Architectural determinism. Each model has a distinct “correction topology”preferred layers, head counts, and magnitudes that are stable across seeds (4/5 consistency) but vary across architectures. The relative ordering is consistent across tasks: Llama uses earlier layers (spam: 2–14, sentiment: 12–15), OLMo uses late layers (spam: 9–15, sentiment: 12–14), Gemma uses deep layers (spam: 9–24, sentiment: 18–19). However, the absolute layer ranges shift per-task, indicating that steering parameters are determined by the model’s topology and the task’s representational structure. Component ablation. Every component of Deep Noir is essential across all 3 architectures (Table 7). Removing head masking drops accuracy below baseline on Llama (−5%) and Gemma (−1%), confirming that global steering introduces destructive interference; OLMo gains only +3% without masking vs. +20% with it. Fixed magnitude (M =0.2) also fails (−11% Llama, +3% OLMo vs. +20% calibrated), validating per-layer calibration. Using the middle layer instead of ranked selection achieves only +9% (Llama) vs. +21% with ranking, though OLMo’s middle layer (+14%) is closer to its ranked result (+20%), suggesting its correction topology is more centralized. Table 7: Component Ablation and Computational Cost (Enron, fold-0, 100 probes) Condition

Llama

OLMo

Gemma

Time

Baseline Full Deep Noir − Head masking − Magnitude cal. − Layer ranking

58.0% 79.0% (+21) 53.0% (−5) 47.0% (−11) 67.0% (+9)

50.0% 70.0% (+20) 53.0% (+3) 53.0% (+3) 64.0% (+14)

54.0% 80.0% (+26) 53.0% (−1) 54.0% (0) 55.0% (+1)

— 8 / 10 / 27 min — — —

RepE (grid search)

67.6% (+10)

59.0% (+9)

55.6% (+2)

0.3 / 0.3 / 0.4 min

Discovery takes 4–31 minutes depending on architecture (Table 7), which is 50–80× slower than RepE’s grid search (∼15s). However, this is a one-time setup cost. The discovered configuration is reused across all subsequent inferences with zero overhead. Future work could reduce discovery time by caching layer rankings across tasks (since architectural topology is partially shared) or pruning the K× layer search space via early stopping. 8

Comparison with supervised baselines. A logistic regression probe trained on mid-layer hidden states (50 samples) achieves 90–93% on held-out folds higher than DN’s 70–80% at 1B. However, the probe (a) requires gradient-based training, (b) provides no mechanistic insight into which layers or heads encode the decision, (c) does not transfer cross-task, and (d) cannot be analyzed for injection vulnerability or containment. Deep Noir does not aim to outperform supervised classifiers; it enables interpretable, training-free diagnosis and control of model internals. A fundamentally different goal that supervised probes cannot address. Mechanistic insight vs. search. To verify the advantage comes from mechanistic grounding rather than better hyperparameter search, we compare DN against 30 random (layer, heads, M ) configurations using the same contrastive direction. DN outperforms random search by +14% (Llama), +20% (OLMo), and +33% (Gemma), confirming that logit-lens layer ranking and head attribution provide signal that random search cannot match. Scaling from 1B to 9B. The discovery engine scales across three model sizes (Table 1). Spam gains increase monotonically with scale within families: Gemma +15.8→+30.4→+42.4 and Llama +20.9→+25.6→+29.2. At 7–9B, four architectures confirm the pattern: Gemma-9B (+42.4), Llama-8B (+29.2), Mistral-7B (+22.0), and OLMo-7B (+21.2) on spam all >+20 pts. Sentiment gain decreases with scale (+13.1→+10.0→+4.0) as baselines rise toward 95%. Critically, RepE fails at every scale: on sentiment, RepE=baseline at 1B, 3B (p=0.0004), and 7B (p=0.0004), while DN consistently improves. At 7B, steering also reduces injection vulnerability by −36.4 pts (Section 4.6). Discovery takes 4–10 min (1B), ∼15 min (3B), ∼30 min (7B). Held-out generalization. Discovering on fold-0 and evaluating on completely held-out fold-1 yields +8% gain (Llama/Enron). On the full SST-2 validation set (872 samples, not just probes), a config discovered from 50 probes achieves +7.3% (76.9%→84.3%), confirming the steering generalizes beyond the discovery set. The in-fold vs. full-dataset gap is modest (11.6% vs. 7.3%), indicating the discovered direction captures genuine model properties. Limitations. (1) Strongest on binary classification; reasoning steering is limited by near-random baselines at tested scales (Appendix J). (2) Scaling uses one model per family-size combination; 7B models use 4-bit quantization. (3) 10% LoRA fidelity gap (Appendix G). (4) Linear direction assumption may miss non-linear decision boundaries. (5) Higher per-fold variance than RepE (±10– 11 vs. ±5–7), a cost of per-fold optimization. (6) The injection vulnerability is inherent to any steering approach with inferable directions. (7) Testing scalability and generality on 70B+ models across complex reasoning, math, and instruction following remains future work.

Conclusion

Deep Noir demonstrates that mechanistic interpretability can automate activation steering, achieving +16.7 pts on spam at 1B and scaling to +21–42 pts at 7–9B across four architectures, with +13.1 pts cross-task on sentiment, all without manual tuning. The mechanistic grounding is essential: DN outperforms random search by +14–33% and outperforms RepE where RepE fails entirely (p<0.01 all models on sentiment, p=0.0004 at 7B). Steering containment is task-dependent: sentiment steering preserves MMLU (≤4%) while spam steering leaks (−9.5%). Steering also creates a quantifiable injection attack surface: vulnerability scales monotonically with steering magnitude, and is partially detectable via activation-norm monitoring. Our results suggest that mechanistic composition is not merely additive but enables qualitatively new intervention capabilities particularly relevant for agent systems where interpretability is required for safety-critical decisions.

Broader Impact Positive impacts. Automated steering discovery democratizes mechanistic interpretability, enabling practitioners to correct model behaviors without expert manual tuning. Our containment analysis (Section 4.5) shows that steering can preserve general capabilities. Agent security risks. Our injection analysis (Section 4.6) shows that activation steering creates a predictable attack surface relevant to agent systems where steered LLMs gate access to tools or sensitive operations. We disclose this vulnerability and our injection templates to encourage defensive research. 9

Acknowledgments This research is supported by the Office of Naval Research (ONR).

References [1] Kyle Arrot et al. Layernavigator: Navigating layers for effective activation steering. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [2] Zhihao Wang et al. SADI: Semantics-adaptive activation intervention for LLMs via dynamic steering vectors. In International Conference on Learning Representations (ICLR), 2025. [3] nostalgebraist. interpreting gpt: the logit lens. LessWrong, 2020. [4] Neel Nanda. Attribution patching: Activation patching at industrial scale. Neel Nanda’s Blog, 2023. [5] Neel Nanda. A pragmatic vision for interpretability. Alignment Forum, 2025. [6] Lee Sharkey et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025. [7] Anonymous. Feature guided activation additions: A lens into targeted steering of LLMs. In International Conference on Learning Representations (ICLR), 2025. [8] Roee Wehner, Omer Barak, and Yonatan Belinkov. Taxonomy, opportunities, and challenges of representation engineering for large language models. Transactions on Machine Learning Research, 2025. [9] DeepMind Safety Research. Negative results for sparse autoencoders on downstream tasks and deprioritising SAE research. DeepMind Safety Research Blog, 2025. [10] Yuxin Cheng et al. FASB: Flexible activation steering with backtracking. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [11] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inferencetime intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024. [12] Anonymous. InterLoRA: Adaptive LoRA design via mechanistic interpretability. In International Conference on Machine Learning (ICML), 2025. [13] Andy Arditi et al. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems, 2024. [14] Anonymous. Analysing the safety pitfalls of steering vectors. arXiv preprint arXiv:2603.24543, 2026. [15] Chi Hong Siu et al. Steeringsafety: A systematic safety evaluation framework for steering methods. arXiv preprint arXiv:2509.13450, 2025. [16] Anonymous. Transformers don’t need LayerNorm at inference time: Scaling LayerNorm removal to GPT-2 XL. arXiv preprint arXiv:2507.02559, 2025. [17] Tiago A Almeida, José Marı́a Gómez Hidalgo, and Akira Yamakami. Contributions to the study of SMS spam filtering: new collection and results. In Proceedings of the 11th ACM Symposium on Document Engineering, pages 259–262, 2011. [18] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, 2013. [19] deepset. Prompt injections dataset. https://huggingface.co/datasets/deepset/ prompt-injections, 2023. Apache 2.0 License, 662 samples.

Architectural Specifications

Table 8 details the architectural specifications for the models evaluated in this study, spanning parameter scales from 1B to 9B. The selected models represent a diverse set of structural design choices, including variations in depth, hidden dimensions, and attention mechanisms (e.g., grouped-query versus full attention).

Table 8: Model architectures. 7B+ models use 4-bit NF4 quantization.

Model

Params

Layers

Heads

Hidden

Attn

Llama-3.2-1B OLMo-1B Gemma-3-1B-IT Gemma-2-2B-IT Llama-3.2-3B OLMo-7B Mistral-7B-Inst. Llama-3.1-8B Gemma-2-9B-IT

1.2B 1.2B 1.0B 2.6B 3.2B 6.9B 7.2B 8.0B 9.2B

16 16 26 26 28 32 32 32 42

32 16 8 8 24 32 32 32 16

2048 2048 1152 2304 3072 4096 4096 4096 3584

GQA Full GQA GQA GQA Full GQA GQA GQA

Base Scale Factor Derivation

The steering intervention is ⃗v = α · M · dˆ· m, where α = 450 is the base scale factor. This constant was calibrated empirically to produce activation perturbations that shift output logits by 2–4 units, sufficient to flip classification decisions at the 1B scale. Rationale. At the 1B scale, residual stream activations at intermediate layers have ℓ2 norms of ∼200–800 depending on architecture and layer. A unit-norm direction vector dˆ therefore needs to be scaled by O(102 ) to produce a perturbation of the same order √ as the residual stream. The effective perturbation magnitude is α · M · ∥m∥, where ∥m∥ = K · dh (number of active head dimensions). For typical configurations (K=1, dh =64, M =0.2), this gives 450 × 0.2 × 8 = 720, within the residual stream’s norm range. Invariance. The golden-section search over M absorbs architecture-specific variation: the same α=450 works across all three architectures because the search finds the appropriate M for each (Llama/OLMo: M ≈ 0.2; Gemma: M ≈ 1.0). Ablating α by 2× shifts optimal M by ∼2× with no accuracy change, confirming α and M are interchangeable degrees of freedom.

Prompting Baselines

Note: Gemma’s 5-shot baseline (78%) outperforms Deep Noir’s single-fold result (72%) on this particular fold. However, across 5-fold cross-validation, Deep Noir achieves 76.4% ± 5.2 (Table 2), and the 5-shot result lacks cross-validation. More importantly, prompting baselines require permodel prompt engineering, while Deep Noir’s discovery is fully autonomous.

Table 9: Prompting baselines on Enron fold-0 (accuracy %). Deep Noir uses 50 labeled probes; few-shot uses 5 examples in-context. Model Llama-3.2-1B OLMo-1B Gemma-3-1B-IT

Standard

Chain-of-Thought

5-Shot

Deep Noir

52% 46% 50%

48% 56% 44%

44% 40% 78%

79% 66% 72%

Table 10: Per-fold discovery results for Enron and Ultimate datasets (accuracy %)

Model

Dataset

Fold 0

Fold 1

Fold 2

Fold 3

Fold 4

Llama Llama OLMo OLMo Gemma Gemma

Enron Ultimate Enron Ultimate Enron Ultimate

56→84 46→80 46→66 62→78 50→86 70→84

56→64 52→86 52→72 56→88 56→78 74→84

58→86 44→64 58→58 44→72 68→72 74→80

60→68 42→86 60→72 56→70 60→68 58→68

58→80 66→78 50→62 64→84 68→86 66→76

Per-Fold Discovery Results

Notable observations: OLMo/Enron fold-2 shows 0% gain (58%→58%), one of 6 non-improving runs. Gemma/Enron fold-0 shows the largest single gain (+36 pts). Per-fold variance is highest for Llama/Ultimate (range: +12 to +44 pts).

Robustness Analysis

Seed sensitivity. 5 random seeds (42, 123, 456, 789, 1337) on Llama/Enron fold-0: accuracy gains range from +4% to +24%, with 4/5 seeds selecting the same target layer. Mean gain: 14.0% ± 7.2%. Ordering sensitivity. 5 probe orderings on Llama/Enron fold-0: accuracy gains range from +19% to +24%, with layer selection varying (L=9 to L=15) but gains remaining stable. Maximum variance: 5 pts. Phase 1 weight ablation. Testing 7 ratios from (LL=0, Ant=1) to (LL=1, Ant=0) on Llama/Enron and Gemma/Enron (fold-0, 50 probes): all 7 ratios select the same top layer and produce identical accuracy for both models (Llama: L=15, 76%; Gemma: L=25, 80%). The ranking is dominated by whichever signal is strongest per-layer, making the linear combination weight irrelevant. Probe balance sensitivity. On balanced probes (50/50 spam/ham), gain = +16%. On imbalanced probes (80/20 spam-heavy): gain = +4%. On ham-heavy (20/80): gain = +47%. Discovery is robust to moderate imbalance but benefits from diverse representation.

Figure 4: Probe sample sensitivity: accuracy and compute cost vs. number of labeled probes. Discovery works with as few as 5 samples, stabilizing at N ≥ 20.

Iterative Multi-Layer Results

Across 33 iterative runs (3 models × 3 datasets × variable folds): single-layer steering is sufficient in 24/33 cases. Multi-layer adds +4 pts average when triggered. Maximum layers discovered: 3 (OLMo/Enron, Gemma/Ultimate). All models converge within 5 iterations. 12

Table 11: Single-shot vs. iterative discovery (best single-fold results per model/dataset)

Model

Dataset

Baseline

Single

Iterative

Layers

Llama Llama OLMo Gemma Gemma

Enron SMS SMS SMS Enron

56% 46% 36% 28% 50%

84% 78% 82% 86% 86%

88% 78% 82% 86% 86%

3 1 1 1 1

LoRA Conversion

33 steering configurations converted to LoRA adapters (rank 16, AdamW lr=10−3 , 100 steps). Mean fidelity gap (hook accuracy − LoRA accuracy): Table 12: LoRA conversion fidelity by model Model Llama-3.2-1B Gemma-3-1B-IT OLMo-1B

Mean Gap

Std

Conversion Time

7.7% 10.8% 13.6%

8.6% 7.0% 11.7%

155s 349s 164s

Rank-64 does not improve over rank-16 (mean gap: 10.7% vs. 10.7%), indicating the bottleneck is the distillation approach (matching hook deltas), not LoRA capacity. Future work could apply latent class separability techniques to enforce better class separation in the adapter’s latent space.

Prompt Injection Category Breakdown

Architecture-specific vulnerability profiles: Llama is most susceptible to delimiter escapes (36%), suggesting weaker boundary detection in its tokenizer/attention. OLMo resists most attacks but is vulnerable to encoding obfuscation (33.5%). Gemma shows similar encoding vulnerability (34%) plus high instruction override susceptibility (31%). Table 13: Injection success rate (%) by OWASP category and model (1,200 trials per model)