Conceptio › Archive › arXiv CS

arXiv CSopen access

Steer Like the LLM: Activation Steering that Mimics Prompting

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

neural-networks

machine learning, deep learning, neural networks

Steer Like the LLM: Activation Steering that Mimics Prompting

Geert Heyman 1 Frederik Vandeputte 1

arXiv:2605.03907v1 [cs.CL] 5 May 2026

Abstract

ble to prompt injection attacks that override the intended behavior (Anwar et al., 2024) and constructing prompts that consistently steer the target behavior can be challenging or may not be feasible altogether (Turner et al., 2023).

Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform compared to prompt-based approaches. We propose a framework that formulates prompt steering as a form of activation steering and investigates whether distilling successful prompt steering behavior into simpler, interpretable models can close this gap. Our analysis reveals that popular activation steering methods are not faithful to the mechanics of prompt steering, which applies strong interventions on some tokens while barely affecting others. Based on these insights, we introduce Prompt Steering Replacement (PSR) models that estimate token-specific steering coefficients from the activations themselves and are trained to imitate prompt-based interventions. Experiments on three steering benchmarks across multiple language models show that PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.2

Activation steering (Dathathri et al. (2020); Subramani et al. (2022); Turner et al. (2023); Zou et al. (2023); Li et al. (2023); Rimsky et al. (2024); inter alia) has been explored as an alternative with the promise of offering more finegrained control, while being lightweight and more robust to adversarial attacks (Wang et al., 2025a). Because activation steering relies on (often simple) interventions that target specific parts of the model, it is also appealing from a mechanistic interpretability perspective (Geiger et al., 2025). Unfortunately, activation steering methods still struggle to outperform prompting (Wu et al., 2025a; Chen et al., 2025; Wu et al., 2025b). This raises the question: “Can we learn from prompt steering to create better activation steering methods?” Recent work from Dherin et al. (2025) indicates that the effects of prompt steering, activation steering and parameter-efficient finetuning can all be represented as lowrank updates to the model weights. Expanding upon this perspective, we frame in-context learning as the form of (uninterpretable) activation steering that is implemented by the LLM itself. From this angle, this paper explores the benefits of distilling how prompting intervenes on the LLM’s activations in an interpretable activation steering module. We make the following key contributions:

1. Introduction As large language models (LLMs) become more prominent in real-world applications, so does the need to reliably control their behavior. Finetuning and prompting are common approaches to align LLMs to preferences and constraints. However, alignment-through-finetuning is computationally expensive and typically requires a significant amount of human-annotated data, making the approach less flexible. Moreover, finetuning on downstream tasks can inadvertently override guardrails that were implemented by alignment finetuning (Qi et al., 2023).

(1) We propose a new framework for studying prompting and activation steering by formulating prompt steering as activation steering and distilling it into simpler, more interpretable interventions. (2) We analyze the prompt steering interventions, and show that activation steering methods that are popular in the literature are not faithful to the mechanics of prompt steering, which tend to apply strong interventions on some token positions and barely intervene on others.

While prompting is more flexible to deploy, it is suscepti-

(3) Within our framework, we propose new rank-1 activation steering methods, and lay out the assumptions under which they can represent prompt steering. These Prompt Steering Replacement (PSR) models apply token-specific steering coefficients estimated from the activations themselves, relaxing the common design choice to intervene

1 Nokia Bell Labs, Belgium. Correspondence to: Geert Heyman <[email protected]>.

Preprint. May 6, 2026. 2 https://github.com/Nokia-Bell-Labs/stee r-like-the-llm

Steer Like the LLM: Activation Steering that Mimics Prompting

equally across all positions. While these assumptions do not hold for every prompt instruction, our analyses suggest that token-specific steering coefficients are a likely ingredient of more general theories.

function controls the strength of the intervention at each token position. In their work, the steering architecture and training objective were aimed at learning interventions that steer only on tokens whose activations are inconsistent with a desired attribute. Our purpose is different, we explore token-specific steering coefficients to better approximate prompt steering, for which we require a different training objective and setup.

(4) We evaluate the effectiveness of these PSR models for steering long-form generation on three benchmarks and across multiple language models, and find that the best configurations compare favorably to strong steering baselines, especially when controlling for high-coherence completions.

Prompt steering. Radford et al. (2019) and Brown et al. (2020) demonstrated that the behavior of LLMs can be customized by adding instruction and/or examples to the prompt. Modifying model behavior by engineering good prompts has become a common practice and methods have since been proposed to automate prompt engineering (Shin et al. (2020); Zhou et al. (2023b); inter alia).

2. Related Work Activation steering. Before activation steering became popular for transformer LLMs, it had been explored in the context of other architectures and modalities (Giulianelli et al., 2018; Bau et al., 2019; Soulos et al., 2020; Besserve et al., 2020). Early works that applied activation steering to LLMs inferred a different steering vector for every query (Dathathri et al., 2020; Subramani et al., 2022), an approach that has been recently revisited by Oozeer et al. and Wang et al. (2025b). While this makes steering methods more expressive, they also become more difficult to interpret.

Connecting activation steering and prompt steering. A few works used steered prompts to construct steering vectors. Zou et al. (2023), Stolfo et al. (2025) and Chen et al. (2025) use difference-in-means on activations of prompts that steer the LLM to express/suppress the target attribute; whereas Liu et al. (2024) leverage the activations corresponding to the final tokens of in-context examples demonstrating the target behavior. Wu et al. (2024) proposed finetuning lowrank activation interventions, and Wu et al. (2025a) used this approach for their ReFT-R1 activation steering method, training the intervention parameters to maximize the loglikelihood of responses generated through prompt steering. However, none of these methods aim to be faithful to prompt steering at inference time: they either apply the intervention equally on the different positions, apply it only on the last prompt token, or clip the steering vector to a set value. In this paper, we replicate prompt steering mechanics at a more fine-grained level: we allow for different steering coefficients at different token positions, and propose to minimize the difference between the activations from prompt steering and those from the activation steering method.

A large body of work uses the same steering vector for different inputs (Turner et al. (2023); Zou et al. (2023); Rimsky et al. (2024); Li et al. (2023); Liu et al. (2024); Marks & Tegmark (2023); Wu et al. (2025a); inter alia). Common to these works is that they either steer on a single token position (e.g., on the activations of the last input token) or apply the same steering coefficient at every token position on which they intervene. To construct the steering vector, these works rely on computing the difference between the mean activations from inputs that express the target attribute and those that do not (difference-in-means), or use the weight vector from a probe that predicts attribute presence (Li et al., 2023; Marks & Tegmark, 2023).

From a theoretical perspective, Bigelow et al. (2025) argue that prompt steering and constant activation steering can be seen as dual techniques to influence the belief in a latent concept given the prompt. We offer the complementary insight that prompt steering can itself be seen as a type of activation steering that applies token-specific interventions.

To better model the mechanics triggered by prompting, we move beyond such constant steering approaches and explore methods that compute a different steering coefficient for each steered activation. Recent works have proposed per-token steering coefficients to make the presence of the steering vector in the steered activations (i.e., the projection of the steering vector onto the steered activations) uniform across token positions (Stolfo et al., 2025; Hedström et al., 2025; Vogels et al., 2025). While this may help mitigate oversteering, our paper finds that such approaches are not faithful to the mechanics of prompt steering, which can exert strong interventions on some token positions and barely intervene on others. We therefore propose to learn the tokenspecific steering coefficients from the activations themselves. This type of intervention was recently explored by Nguyen et al. (2025) for multi-attribute steering, where a gating

3. Connecting Prompt and Activation Steering 3.1. Preliminaries The goal of steering is to elicit a certain attribute attr in an LLM’s response without changing the model weights. In prompt steering, this is achieved by adding instructions and/or in-context examples to the original prompt. In activation steering the LLM behavior is influenced by

Steer Like the LLM: Activation Steering that Mimics Prompting

Figure 1. Illustration of how prompt steering interventions ∆P S can be computed by subtracting prompt-steered activations from the corresponding unsteered activations (left and center). Prompt Steering Replacement (PSR) models approximate these interventions, but only on cases where prompt steering successfully elicits the target attribute (right).

methods from Zou et al. (2023); Chen et al. (2025) all rely on Equation 2 during inference. We will refer to this steering family as constant activation steering (Const). 3.2. Prompt Steering as Activation Steering Without loss of generality, we can write the activations of the LLM response tokens y ′ that were generated from the steered prompt x′ as an intervention on the activations of y ′ computed with the original prompt x (refer to the left and center sections of Figure 1):5

Figure 2. Strength of prompt steering interventions on Llama-3.23B, layer 16, across token positions (x-axis) for randomly sampled completions that are prompt-steered towards sycophancy (y-axis).

′ ′ Al,yi′ |P S = Al,yi′ + ∆P S (x′ y≤i , xy≤i )

The nature of ∆P S depends on how the baseline activations Al,yi′ are defined: (1) If Al,yi′ are the activations from a fully unsteered forward pass (i.e., using the original prompt x), then ∆P S ≜ ∆P Sacc captures the total effect of prompt steering accumulated across layers 1 through l. (2) If instead Al,yi′ are obtained by feeding prompt-steered activations from layer l−1 through layer l, but replacing the steering prompt token activations with those from an unsteered forward pass, then ∆P S ≜ ∆P Sloc isolates the local steering contribution made at layer l.

intervening on the LLM’s internal activations. Formally, if Ayi denotes the activations for the ith response token at a given layer l,3 then we can write single-attribute activation steering as follows:4 Ayi |AS = Ayi + ∆AS (xy≤i , attr)

(1)

Where xy≤i denotes the concatenation of the prompt x and the sequence of response tokens y≤i up to and including the ith token, and ∆AS is the steering intervention function that modifies the original activations Ayi to produce the steered activations Ayi |AS . A common choice for ∆AS is α zattr : Ayi |AS = Ayi + α zattr

(3)

Because Al,yi′ and Al,yi′ |P S are computable in both cases, the analytical forms of the prompt steering interventions ∆P Sacc and ∆P Sloc are known. However they are complex functions of the model weights, the steered prompt x′ , the ′ original prompt x, and the response tokens y≤i . In this paper, we study the nature of the prompt steering interventions and explore if we can mimic them with simpler, more interpretable functions. We refer to the latter as prompt steering replacement (PSR).6

(2)

where zattr is a steering vector in the activation space whose presence is correlated with an increase in likelihood of predictions with the target attribute, and where α is a scalar known as the steering coefficient. ActAdd (Turner et al., 2023), CAA (Rimsky et al., 2024), ITI (Li et al., 2023), ReFT-R1 (Wu et al., 2025a), and the activation steering

If x′ is constructed by prepending a steering prompt xattr to the original prompt x, prompt steering also intervenes the activations of the original prompt tokens xi . In general, the activations of any token ti in the shared suffix between xy ′ and x′ y ′ can be seen as subject to the prompt steering interventions. 6 This term is inspired by the “replacement model” in Ameisen et al. (2025). 5

We omit the layer index l in our notation when it is clear from the context. 4 Some methods apply interventions on input tokens. To keep notation simple, we will write all intervention equations in this paper in terms of the activations of a response token.

Steer Like the LLM: Activation Steering that Mimics Prompting

from token yi′ at layer l:

3.3. Prompt Steering as Constant Activation Steering In this subsection, we lay out two assumptions under which the accumulative effect of prompt steering in layers 1 to l is reduced to constant activation steering in layer l (Equation 2).

′ ′ ∥∆P Sacc (x′ y≤i , xy≤i )∥ = f (Ayi′ ; θ attr )

Where θ attr are attribute-specific intervention parameters. In transformer architectures, this assumption is also motivated by the fact that the activations access the information in the steered prompt only through self-attention, where attention weights are computed as a dot product between a query (a linear transform of the activations that consume the attention output) and a key (a representation of the incoming information, which in our case could be captured by θ attr ). Assumptions 3.1 and 3.2a define a family of steering architectures:

Assumption 3.1. The interventions ∆P Sacc that capture the accumulative effect of prompt steering across layers 1 to l operate along a single direction. ′ ′ ′ ′ ∆P Sacc (x′ y≤i , xy≤i ) = c(x′ y≤i , xy≤i ) zattr

(6)

(4)

Here c(·) is a scalar function that computes the steering coefficient for token position i in layer l and zattr is the steering vector that corresponds to attribute attr in layer l.

Al,yi′ |AS = Al,yi′ + α λ(Al,yi′ ; θ attr,l ) zattr,l

Prior work on the linear representation hypothesis and activation steering has found evidence for the existence of linear representations that steer model behavior (Turner et al., 2023; Zou et al., 2023; Park et al., 2024). It therefore seems plausible that the LLM is leveraging these representations when implementing prompt steering. If, in the activation space of a given layer, there exists a direction zattr that is associated with the presence of attribute attr, then Assumption 3.1 would be a reasonable approximation.

(7)

Here λ(·; θ attr,l ) is a steering coefficient function that estimates the token-specific steering coefficient from the activations at token position i in layer l, and zattr,l is the steering vector for attribute attr in layer l. The scalar α models the presence of the attribute in the steered response y ′ . During training, α should be set to a value that reflects the degree to which the attribute is expressed in y ′ (e.g., to 0 or 1 for binary attributes). During inference, α can be treated as a hyperparameter that controls the strength of the intervention, similar to other activation steering methods. We will refer to α as the global steering coefficient. In the remainder of this section, we introduce concrete architectures and detail how the parameters θ attr,l and zattr,l are optimized to replicate prompt steering behavior.

Assumption 3.2. (from prior art) The interventions ∆P Sacc that capture the accumulative effect of prompt steering across layers 1 to l have the same magnitude across all token positions. For all i, j: ′ ′ ′ ′ ∥∆P Sacc (x′ y≤i , xy≤i )∥ = ∥∆P Sacc (x′ y≤j , xy≤j )∥ (5)

For our experiments, we estimate λ(·) using a single-layer probe with ReLU activation:

When combining Assumptions 3.1 and 3.2, Equation 3 simplifies to the constant activation steering method defined in Equation 2. However, empirical analysis of prompt steering interventions shows that the strength of prompt steering interventions varies significantly across token positions. Figure 2 illustrates this for LLama-3.2-3B activations computed on randomly sampled completions that were prompt-steered towards sycophancy. We observed similar behavior for other attributes and language models.

λ(Al,yi′ ; θ attr,l ) = ReLU (Al,yi′ · wattr,l + battr,l ) (8) Where θ attr,l = {wattr,l , battr,l } are the parameters of the steering function at layer l. S-PSR. When the intervention in Equation 7 is applied only in a single layer l, we refer to the steered model as a S-PSR. Even under Assumptions 3.1 and 3.2a, this model can only truly replicate prompt steering if no further prompt steering is implemented in subsequent layers.

3.4. Towards a More Faithful PSR Architecture When analyzing on which tokens prompt steering exerts strong interventions, we observe distinct patterns (see Appendix A.2 for more analysis). This suggests that the prompt steering’s intervention strength could be decoded from the activations themselves. We therefore propose to relax Assumption 3.2 as follows:

A-PSR. So far, our analysis has connected the accumulated effect of prompt steering ∆P Sacc to single-layer activation steering in layer l. Assumptions analogous to 3.1 and 3.2a can also be formulated for the local prompt steering intervention ∆P Sloc , connecting it to activation steering methods that intervene on all layers. This motivates A-PSR models that iteratively apply the intervention in Equation 7 at all layers of the LLM. That is, the activations Al,yi′ at layer l are computed based on the steered activations from the previous layer l-1, approximating prompt steering throughout

Assumption 3.2a. (proposed relaxation) The magnitude of the intervention ∆P Sacc that captures the accumulative effect of prompt steering across layers 1 to l on token yi′ can be expressed as a function of the activations-before-steering 4

Steer Like the LLM: Activation Steering that Mimics Prompting

the entire forward pass. It is unlikely that Assumptions 3.1 and 3.2a are good prompt-steering approximations for all layers, though. Therefore interventions might add noise that propagates through the rest of the forward pass. However, we find that by choosing an appropriate end-to-end training objective that jointly optimizes the parameters θ attr,l and zattr,l for all layers this risk can be mitigated (see Sections 3.5 and 5).

using the LLM (Chen et al., 2025). Optionally, we can filter the resulting triplets (x, x′ , y ′ ) based on the quality of the steered response. This ensures that we are training a replacement model for successful prompt steering and may enable PSR models to surpass the performance of prompt steering. In our experiments, following Chen et al. (2025), we use two judge components Jattr and Jcoher to assess whether the response y ′ contains the target attribute and is coherent. Details about the judges we used in our experiments will be described in Section 4.

From the perspective of mechanistic interpretability, S-PSR and A-PSR seek answers to two different questions: The S-PSR model on layer l aims to uncover how the target attribute is represented at the output of layer l, whereas A-PSR sheds more light on how this representation was computed throughout layers 1-l.

From the triplets (x, x′ , y ′ ), we can compute the activations Al,yi′ |AS (by feeding xy ′ to the LLM augmented with a PSR model) and Al,yi′ |P S (by feeding x′ y ′ to the LLM). During training, we set the global steering coefficient in Equation 7 to the judge score: α = Jattr . For non-binary attributes, in a setting with access to a judge Jattr that outputs a continuous score, this ensures that λ(·) only estimates whether the activations Al,yi′ are appropriate for steering, while the steering strength is determined by the judge scores. At inference time, α remains a hyperparameter that controls attribute presence.

3.5. PSR Training Objectives Mean-Squared Error (MSE). The most direct way to train a PSR model is to minimize the difference between the activations from prompt steering and those from the PSR intervention. As a difference measure, we propose meansquared error (MSE). For S-PSR we minimize the sum of the MSEs of layer l and all the subsequent layers; for A-PSR we jointly optimize all the interventions to minimize the sum of the MSEs for all layers. Because the intervention in layer l also optimizes for the MSEs of all subsequent layers, we expect that when Assumptions 1 and 2 do not hold for l, the intervention will not negatively impact the overall performance of the replacement model. That is, the MSE loss of the subsequent layers discourages learning interventions that make it harder to approximate the activations in subsequent layers.

In our experiments, we found it to be sufficient to train PSR models on positive examples only (i.e., triplets for which x′ steers the prediction y ′ to contain the target attribute). To leverage both positive and negative examples, it is important to adjust Jattr with a bias parameter bm,l initialized at −0.5. This ensures that negative examples (i.e., Jattr < 0.5) get negative steering coefficients at the start of training, and gives the PSR model the option to fit the bias according to the LLM’s default behavior.

Loglikelihood (LL). As an alternative to MSE, we also consider maximizing the loglikelihood of the steered response y ′ when predicted from the steered activations Al,yi′ |AS . Although this objective does not enforce that the intermediate activations are faithful to prompt steering, the experiments in Section 5.1 demonstrate that, for controlling attributes where Assumptions 3.1 and 3.2a are bad approximations, this lack of faithfulness can be beneficial.

4. Experimental Setup 4.1. Benchmarks We carry out experiments on three benchmarks that evaluate steering long-form text generation. Persona Steering: Persona Vectors. To assess the effectiveness of the PSR models in steering LLMs towards a personality trait, we use the framework from Chen et al. (2025), which provides an automated pipeline to create datasets (x, x′ , y ′ ) for a given trait and LLM. Specifically, the framework generates 20 training and 20 evaluation questions per trait along with five ‘positive’ instructions, that elicit behavior that is aligned with the target persona, and five ‘negative’ instructions that elicit the opposite behavior. By prepending each instruction to each generated question, 100 positive and 100 negative prompt pairs are obtained. For each prompt pair in the training set, 10 responses are sampled from the target LLM using temperature 1.0 and topp 1.0, resulting in 1000 positive and 1000 negative triplets (x, x′ , y ′ ) per trait for training.

Regularization. To avoid that the steering coefficient function ends up in the dead region of the ReLU activation for all token positions, we add a regularization term to the loss that punishes cases where the sum of the λ outputs across P all token positions is less than 1: Lreg = max(0, 1 − i λ(Al,yi′ ; θ attr,l )) 3.6. Training Pipeline To create the training data for a given attribute and LLM, we assume access to a collection of prompt pairs (x, x′ ) that only differ w.r.t. the target attribute. For each prompt pair, we generate a response y ′ from the steered prompt x′ 5

Steer Like the LLM: Activation Steering that Mimics Prompting

In addition, the framework comes with an LLM judge Jattr to assess whether the response y ′ contains the target personality trait, and implements a judge Jcoher by prompting gpt-4.1-mini-2025-04-14 to score response coherence. Instances with coherence < 0.5 are filtered out from the training set, positive instances with Jattr < 0.5 or negative instances with Jattr > 0.5 are filtered out as well. This way the interventions learn from successful prompt steering.

generated using greedy decoding for all methods. For the steering method of Stolfo et al. (2025), we reproduce their results with their code such that we can evaluate the coherence of the steered predictions. For all other activation steering methods, we filter out training instances for which Jattr (y ′ ) = 0. From this pool we further remove training instances for which Jcoher (y ′ ) < 0.5, except when this would result in less than 20 training instances. In this case, we select the 20 instances with Jattr = 1 that have the highest coherence. Because in this dataset x′ is not constructed by prepending a steering prompt to x (i.e., there is no consistent separation x′ = xattr x), we steer only on the hidden states that decode the response tokens.

To evaluate a steering method, 10 responses y ′ are sampled (again with temperature 1.0 and top-p 1.0) from the LLM augmented with the steering method for each of the 20 questions in the evaluation set. Except for prompt steering, the unsteered prompts x are used as input when sampling y ′ . The attribute presence and coherence of the responses are assessed using the judges Jattr and Jcoher .

AxBench. Following Sun et al. (2025), we use the Gemma-2-2B-L20 and Gemma-2-9B-L20 subsets of AxBench (Wu et al., 2025a) to validate steering methods across a wider range of target attributes. Each subset evaluates steering across 500 target concepts that were selected from a random sample of Sparse Autoencoder (SAE) features of the residual stream in layer 20 of a subject LLM (Gemma-2-2B and Gemma-2-9B, respectively). Each concept is associated with (x, x′ , y ′ ) triplets: 72 for training, 5 for validation, and 5 for testing. Different from the other two datasets, the steered response y ′ was generated by a frontier LLM, rather than by the subject LLM itself. We train and evaluate using the same setup as Wu et al. (2025a) and reuse their evaluation script to obtain our results. Based on tuning experiments, we disable the regularization term from Section 3.5 for the AxBench experiments, as it degraded performance, particularly for MSE-trained models.

We reuse the traits from Chen et al. (2025): the apathetic and humorous traits were used for hyperparameter tuning, the evil, sycophantic, and hallucinating traits were used for evaluation.7 We ran persona steering experiments for three different LLMs: Llama-3.2-3B-Instruct, Llama3.1-8B-Instruct (Grattafiori et al., 2024), and Qwen2.5-7BInstruct (Yang et al., 2024). All activation steering methods steer on the residual stream. For the S-PSR models and baselines that steer activations in a single layer, steering is done in the layers that were found to be most effective in Chen et al. (2025) (see Appendix C). For Llama-3.2-3B-Instruct (not evaluated in Chen et al. (2025)) we steer in layer 16. Instruction Following: IFEval. To evaluate steering for instruction following, we use the augmented version of IFEval (Zhou et al., 2023a) that was created by Stolfo et al. (2025). Specifically, we evaluate on format instructions of 12 different types and on ‘answer-in-language-X’ instructions for 14 languages.8

4.2. Steering Models We study different PSR ablations and other steering baselines by varying the intervention architecture, training objective, and intervention positions.

We follow the training and evaluation setup from Stolfo et al. (2025), with the same LLMs and steering the single-layer methods on the layers that worked best for their steering method (see Appendix C). Instruction-following is evaluated using the evaluation script from IFEval (Zhou et al., 2023a), i.e., for this dataset the judge Jattr is a script that outputs a binary score, not an LLM judge. The coherence judge Jcoher is the same as for the persona vectors dataset, except for a small adaptation to the prompt instructions to allow for non-English completions (the judge prompt templates are provided in Appendix B).9 Steered responses y ′ are

Intervention architectures. We compare S-PSR and APSR with constant activation steering (Equation 2), both in the single-layer (S-Const) and all-layers (A-Const) settings. For all architectures we steer on the residual stream. Training objectives. For the three architectures, we compare MSE (denoted with ·MSE ) and loglikelihood (denoted with ·LL ) as introduced in Section 3.5.10 For the Const architecture, we also compare with the difference-in-means objective (denoted with ·DiM ), which is a popular choice in the literature (Rimsky et al. (2024); Stolfo et al. (2025); Chen et al. (2025); inter alia).

7 We acknowledge that steering language models toward negative traits such as ‘evil’ and ‘hallucinating’ can be potentially harmful. Our motivation is strictly scientific, these datasets were selected because they have been used in prior work. 8 Stolfo et al. (2025) report 19 languages, but only 14 languages have both training and test instances in their augmented dataset. 9 The original judge instructions explicitly stated that the gener-

ated text should be proper English. 10 Note that ConstLL closely resembles the ReFT-R1 method from Wu et al. (2025a), except that during training ReFT-R1 uses a mechanism to apply the intervention only on activations that already express the target attribute.

Steer Like the LLM: Activation Steering that Mimics Prompting

Intervention positions. We compare steering only on the response tokens (denoted with ·R ) with steering on both the question and response tokens (denoted with ·QR ).

trait alignment across all language models. When intervening at all layers, both the constant steering and PSR achieve high trait alignment at high coherence (refer to the TA@Cp columns). A-PSR outperforms A-Const for 2 out of 3 language models, but the differences are small. A-Const does seem more susceptible to oversteering (see Figure 13 in Appendix E.1).

Other baselines. In addition, we report results without steering (no steering) and with prompt steering (prompt). On IFEval we also report results from Stolfo et al. (2025) as a reference, which uses a single-layer intervention, obtains the steering vector with difference-in-means and dynamically steers across token positions by clipping the steering vector to its mean projection in the set of positive training examples. On AxBench, we include results from Wu et al. (2025a), Wu et al. (2025b), and Sun et al. (2025) as references. This includes the best performing LoRA (Hu et al., 2021) and LoReFT (Wu et al., 2024) variants, as well as HyperSteer (Sun et al., 2025), which finetunes a hypernetwork to predict an intervention from a base prompt and steering instruction.

Across training objectives, we find that loglikelihood outperforms difference-in-means, and MSE outperforms loglikelihood. For the intervention positions, we did not observe substantial differences between question-and-response steering and response-only steering. Results for response-only steering (·R ) were therefore moved to Appendix E.1. When comparing activation steering methods to prompt steering, the best all-layer models outperform prompting: APSRMSE outperforms prompt steering for all three language models, A-ConstMSE outperforms prompt steering for 2 out of 3 language models. At coherence 80.0 (the TA@C80 columns), we observe that S-PSRMSE obtains higher trait alignments than prompt steering for Qwen2.5-7b-Instruct and Llama-3.1-8b-Instruct. For a more detailed insight on the trait alignment-coherence trade-off and the performance of the individual personas, we refer to Figures 12-13 in the Appendix.

4.3. Evaluation Metrics To evaluate steering performance on the Persona Vectors dataset, we report the trait alignment at coherence Jcoher = 80.0 (denoted with TA@C80 ) and at the average coherence of prompt steering (denoted with TA@Cp ). To compute trait alignment at a target coherence level, we explore values for the global steering coefficient α that are close to the target coherence using a binary search procedure, and then interpolate between the two coherence levels immediately above and below the target. See Appendix D for details.

The results on IFEval (Table 2) reveal that the rank-1 PSR models are not expressive enough to replicate prompt steering for all instruction types. This is evident from two observations: the loglikelihood objective is superior to MSE in most settings, and we do not consistently outperform prompting. A closer inspection unveiled that there were multiple instruction types for which the MSE loss barely improved, suggesting that rank-1 interventions cannot capture the prompt steering behavior for these instructions. In such cases, optimizing for loglikelihood is more effective as it only targets the model outputs, not the intermediate activations. An interesting path for future research could be to explore PSR variants that generalize Equation 7 to low-rank interventions.

On the IFEval-format dataset, Stolfo et al. (2025) use a special method to set the steering coefficient α as a function of the input x. For the PSR and Const methods, we therefore fix α to 1 instead of tuning it, to allow a fair comparison. We report the instruction-following accuracy computed with the IFEval script (Jattr ) and coherence (Jcoher ). On AxBench, we use the metric that comes with the dataset, which computes an overall steering score for each concept on a scale of 0 to 2 by taking the harmonic mean of the LLM judge scores that assess concept presence (0-2), fluency (02), and answer relevance (0-2).

It is important to note that the IFEval format evaluation setup includes 5 instruction types that require arguments (e.g., the multiple sections type comprises instructions such as “Include two sections” and “Ensure that your response is in 3 sections”). Such instruction types are not modeled in a single-attribute steering setup. We therefore also include results in Appendix E.2 Table 8 where these instruction types are excluded from the evaluation. In this setup, S-PSRLL outperforms prompting for 2 out of 4 language models and A-PSRLL outperforms prompting for 3 out of 4 language models.

5. Experiments We organize our experiments around two questions: (1) Does PSR improve steering performance? (2) Are PSR interventions more faithful to prompt steering? 5.1. Steering Performance The results on the Persona Vectors dataset are summarized in Table 1. When comparing the single-layer architectures, we observe that PSR’s approach of computing different intervention strengths per activation, significantly improves

Furthermore, PSR significantly improves over the activation steering method of Stolfo et al. (2025). PSR consistently out-

Steer Like the LLM: Activation Steering that Mimics Prompting Table 1. Results on the Persona Vectors dataset. We report trait alignment at coherence 80.0 (TA@C80 ) and at prompt steering coherence (TA@Cp ), scores are macro-averaged over the different traits. TA@Cp scores higher than prompting are underlined. Llama-3.1-8b-Instruct results are included in Appendix E.1. ∗ DiM|R results produced with code from Chen et al. (2025). Llama-3.2-3b-Instruct TA@Cp TA@C80

Llama-3.1-8b-Instruct TA@Cp TA@C80

Qwen2.5-7b-Instruct TA@Cp TA@C80

46.1 72.5 79.3 89.6 91.1

28.9 42.9 57.4 52.6 66.8

49.8 88.4 89.0 96.8 98.8

30.2 44.0 50.1 45.0 74.7

74.8 69.5 71.6 83.3 83.3

34.8 51.8 48.8 59.1 60.9

A-ConstLL|QR A-ConstMSE|QR A-PSRLL|QR A-PSRMSE|QR

98.2 98.9 97.5 98.6

85.6 95.8 94.4 92.5

98.8 98.9 98.4 99.2

85.9 91.3 82.3 96.4

96.1 96.1 95.3 96.8

73.6 83.6 65.7 83.9

–

91.5

–

95.7

–

71.6

prompt

Table 2. Results on the IFEval format dataset. We report the instruction-following accuracy IF Acc. and coherence Coher., both are macro-averaged over the different instruction types. The best IF Acc. scores for the activation steering methods (top) and activation steering with prompting (bottom) are in bold. ∗ no activation steering and no instruction in the prompt. Results for ConstMSE settings are excluded for brevity as they significantly underperform ConstLL . They can be found in Appendix E.2. a Results from Stolfo et al. (2025). b Results reproduced with code from Stolfo et al. (2025). Phi-3-mini-instruct IF Acc. Coher.

Gemma-2-2b-it IF Acc. Coher.

Mistral-7B-Instruct IF Acc. Coher.

Gemma-2-9b-it IF Acc. Coher.

no steering∗ Stolfo et al. (2025) a Stolfo et al. (2025) b S-ConstLL S-PSRLL S-PSRMSE

11.9 30.1 29.0 11.6 62.8 29.3

92.4 86.5 91.6 89.1 91.3

10.6 30.1 39.1 10.7 54.9 39.0

94.3 88.8 94.5 89.5 92.9

6.8 14.1 19.8 19.0 62.7 22.3

90.5 89.8 89.2 87.6 89.0

11.4 28.9 30.8 13.4 66.1 47.5

96.6 96.1 96.7 95.5 96.4

A-ConstLL A-PSRLL A-PSRMSE

61.9 69.0 48.8

90.0 85.4 87.7

36.9 68.7 61.2

90.5 90.6 91.2

69.2 61.1 54.1

81.1 84.1 85.7

50.4 71.9 71.3

94.4 82.3 95.1

72.5

84.6

66.8

88.6

61.8

81.5

85.7

94.8

Stolfo et al. (2025) +prompt Stolfo et al. (2025)+prompt b S-ConstLL +prompt S-PSRLL +prompt S-PSRMSE +prompt

78.6 81.7 78.9 89.8 81.6

79.3 83.2 82.2 79.9

76.1 79.0 74.2 83.2 82.6

84.0 87.1 84.5 87.2

63.7 62.5 77.6 85.5 67.8

80.6 77.3 75.5 76.9

86.6 88.7 91.5 93.1 91.1

94.6 94.3 94.6 94.0

A-ConstLL +prompt A-PSRLL +prompt A-PSRMSE +prompt

89.3 82.8 85.2

75.2 80.2 80.6

87.0 86.4 81.3

78.9 84.6 86.1

82.2 82.0 68.9

71.5 76.0 78.1

85.2 87.6 92.4

92.0 80.0 93.5

prompt a

Table 3. Steering scores (scale 0-2 ↑) on the Gemma-2-2B layer 20 and Gemma-2-9B layer 20 subsets of AxBench. We include the best performing methods on AxBench from the literature: a Wu et al. (2025a). b Wu et al. (2025b). c Sun et al. (2025). (a) Rank-1, single-layer interventions a

SAE

ReFT-r1a

ΦSV,r=1 b

S-ConstLL

S-PSRLL

S-ConstMSE

S-PSRMSE

0.178 0.322

0.151 0.191

0.509 0.630

0.606 0.892

0.504 0.633

0.618 0.667

0.311 0.903

0.367 0.900

S-ConstDiM 2BL20 9BL20

(b) Multi-rank and/or multi-layer methods

2BL20 9BL20

prompta

SFTa

LoRAa

LoRARePSb

LoReFTRePSb

HyperSteerc

A-ConstLL

A-PSRLL

A-ConstMSE

A-PSRMSE

0.731 1.075

0.714 –

0.641 0.602

0.793 0.631

0.805 0.757

0.742 1.091

0.792 0.757

0.690 0.827

0.783 1.053

0.871 1.120

Steer Like the LLM: Activation Steering that Mimics Prompting

activations are as faithful as a forward pass without steering. As a reference, we also report the relative RMSE between prompt steering activations using different but equivalent trait-eliciting instructions (denoted as Equivalent prompts). Specifically, we used the five positive instructions per trait from Chen et al. (2025), see Section 4. Figure 3 plots the average relative RMSE in each layer for Llama-3.2-3B-Instruct on the sycophantic trait. As expected the A-PSRMSE activations are most faithful to prompt steering, but it is still surprising that from layer 10 onwards the relative RMSE is significantly lower than that of equivalent prompts. This indicates that A-PSRMSE is able to closely mimic the interventions that prompt steering implements within the model. Also A-ConstMSE achieves low RMSE values, which suggests that, while constant interventions may not be faithful, the model redistributes steering contributions to the appropriate locations in later layers. A similar phenomenon can be seen for the single-layer activation methods, which spike above relative RMSE of 1 in the intervention layer, indicating that they are less faithful in that layer than no steering, but dip below 1 in the later layers. This suggests that the model is able to partially revert from an “unfaithful” regime to its default behavior in the later layers. Other language models and traits exhibit similar trends, see Appendix F.

Figure 3. Relative RMSE between the accumulated interventions of prompt steering (∆P Sacc ) versus those of other steering methods (∆Xacc ), averaged on prompt steering predictions on the Persona Vectors sycophantic evaluation data for Llama-3.2-3B.

performs constant steering in the single-layer setting and for 3 out of 4 language models in the all-layer setting. The bottom part of Table 2 shows that combining prompting and activation steering consistently improves instruction-following accuracy over prompting alone (Stolfo et al., 2025), albeit with a coherence penalty for some models. From the results on AxBench (Table 3), we find that APSRMSE sets a new state-of-the-art on both the 2B layer 20 and 9B layer 20 subsets, outperforming prompt steering, different LoRA variants and strong activation steering baselines. AxBench computes aggregated steering scores that capture coherence, concept alignment and relevance to the base prompt. We break down the results in these different dimensions in Appendix E.3, and find that A-PSRMSE ’s improvements over the prompting are the result of improved concept alignment and for the 2B layer 20 subset come with a noticeable drop in answer relevance. The results of Table 3 (a) confirm that MSE is not always superior to loglikelihood.

6. Conclusion We proposed a framework for studying the connection between prompting and activation steering by formulating prompt steering as a form of activation steering and distilling its behavior on instances where it is successful into simpler, interpretable models. Our analysis revealed that popular activation steering methods are not faithful to the mechanics of prompt steering, and that closing this gap by learning token-specific interventions improves steering performance. As a first instantiation of this framework, we developed rank-1 Prompt Steering Replacement (PSR) models that, under explicit assumptions, can replicate prompt steering with token-specific steering coefficients estimated from the activations themselves. In our experiments, PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.

5.2. Faithfulness of Interventions In this section, we analyze how faithful different activation steering methods are to prompt steering. To this end, we measure the relative root mean squared error (RMSE) between the interventions produced by prompt steering and those produced by other steering meth′ ods. We compute relative RMSE as ∥∆P Sacc (xy≤i )− ′ ′ ′ ∆Xacc (x, y≤i )∥2 /∥∆P Sacc (x, y≤i )∥2 , with ∆P Sacc (xy≤i ) the prompt steering interventions as defined in Equation 3 ′ and ∆Xacc (x, y≤i ) the accumulative steering effect up to a given layer by steering method X (e.g., S-ConstLL|QE ). We average RMSE values over the prompt steering predictions on the Persona Vectors evaluation data. A lower relative RMSE indicates that the steering method produces activations that are more faithful to those produced by prompt steering. A relative RMSE of 1 signifies that the steered

The assumptions underlying the rank-1 PSR models do not hold universally, however: this is particularly evident on IFEval, where more complex instructions exceed what rank1 interventions can represent and prompting remains the stronger approach. Despite these limitations, our framework and analyses shed new light on the mechanics of prompt steering and suggest that token-specific steering coefficients are a key ingredient of faithful activation steering. 9

Steer Like the LLM: Activation Steering that Mimics Prompting

Acknowledgements

Bigelow, E. J., Wurgaft, D., Wang, Y., Goodman, N. D., Ullman, T. D., Tanaka, H., and Lubana, E. S. Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering. CoRR, abs/2511.00617, 2025. doi: 10.48550/ARXIV.2511.00617. URL https: //doi.org/10.48550/arXiv.2511.00617. arXiv: 2511.00617.

We would like to thank Raf Huysegems, Pascal Justen, Haeun Yu, and the anonymous reviewers for their valuable feedback and suggestions.

Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language Models are Few-Shot Learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.-F., and Lin, H.-T. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurip s.cc/paper/2020/hash/1457c0d6bfcb496 7418bfb8ac142f64a-Abstract.html.

References Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., Persic, A., Qi, Z., Ben Thompson, T., Zimmerman, S., Rivoire, K., Conerly, T., Olah, C., and Batson, J. Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/ 2025/attribution-graphs/methods.html.

Chen, R., Arditi, A., Sleight, H., Evans, O., and Lindsey, J. Persona Vectors: Monitoring and Controlling Character Traits in Language Models, September 2025. URL http://arxiv.org/abs/2507.21509. arXiv:2507.21509 [cs].

Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., Edelman, B. L., Zhang, Z., Günther, M., Korinek, A., Hernández-Orallo, J., Hammond, L., Bigelow, E. J., Pan, A., Langosco, L., Korbak, T., Zhang, H. C., Zhong, R., hÉigeartaigh, S. O., Recchia, G., Corsi, G., Chan, A., Anderljung, M., Edwards, L., Petrov, A., Witt, C. S. d., Motwani, S. R., Bengio, Y., Chen, D., Torr, P., Albanie, S., Maharaj, T., Foerster, J. N., Tramèr, F., He, H., Kasirzadeh, A., Choi, Y., and Krueger, D. Foundational Challenges in Assuring Alignment and Safety of Large Language Models. Trans. Mach. Learn. Res., 2024, 2024. URL https://openreview.net/forum?id= oVTkOs8Pka.

Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., and Liu, R. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https: //openreview.net/forum?id=H1edEyBKDS. Dherin, B., Munn, M., Mazzawi, H., Wunder, M., and Gonzalvo, J. Learning without training: The implicit dynamics of in-context learning, December 2025. URL http://arxiv.org/abs/2507.16003. arXiv:2507.16003 [cs].

Bau, D., Zhu, J.-Y., Strobelt, H., Zhou, B., Tenenbaum, J. B., Freeman, W. T., and Torralba, A. Visualizing and Understanding GANs. In Deep Generative Models for Highly Structured Data, ICLR 2019 Workshop, New Orleans, Louisiana, United States, May 6, 2019. OpenReview.net, 2019. URL https://openreview.net/forum ?id=rJgON8ItOV.

Geiger, A., Ibeling, D., Zur, A., Chaudhary, M., Chauhan, S., Huang, J., Arora, A., Wu, Z., Goodman, N., Potts, C., and others. Causal abstraction: A theoretical foundation for mechanistic interpretability. Journal of Machine Learning Research, 26(83):1–64, 2025.

Besserve, M., Mehrjou, A., Sun, R., and Schölkopf, B. Counterfactuals uncover the modular structure of deep generative models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=SJxD DpEKvH.

Giulianelli, M., Harding, J., Mohnert, F., Hupkes, D., and Zuidema, W. H. Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information. In Linzen, T., Chrupala, G., and Alishahi, A. (eds.), Proceedings of the Workshop: Analyzing and Interpreting Neural Networks for NLP, 10

Steer Like the LLM: Activation Steering that Mimics Prompting

geted Intervention. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, July 2025. doi: 10.18653/v1/2025.acl-long. 1007. URL https://aclanthology.org/2025. acl-long.1007/. arXiv:2502.12446 [cs].

BlackboxNLP@EMNLP 2018, Brussels, Belgium, November 1, 2018, pp. 240–248. Association for Computational Linguistics, 2018. doi: 10.18653/V1/W18-5426. URL https://doi.org/10.18653/v1/w18-5426. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., and others. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783, 2024.

Oozeer, N., Marks, L., Barez, F., and Abdullah, A. Beyond Linear Steering: Unified Multi-Attribute Control for Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistic. URL https://aclant hology.org/2025.findings-emnlp.1278/.

Hedström, A., Amoukou, S. I., Bewley, T., Mishra, S., and Veloso, M. To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models. In Singh, A., Fazel, M., Hsu, D., LacosteJulien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., and Zhu, J. (eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. URL https://proceedings.mlr.press/v267/h edstrom25a.html.

Park, K., Choe, Y. J., and Veitch, V. The Linear Representation Hypothesis and the Geometry of Large Language Models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: //openreview.net/forum?id=UGpGkLzwpP. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023.

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., and Chen, W. LoRA: Low-Rank Adaptation of Large Language Models. CoRR, abs/2106.09685, 2021. URL https://arxiv.org/abs/2106.09685. arXiv: 2106.09685.

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multitask Learners. 2019.

Li, K., Patel, O., Viégas, F. B., Pfister, H., and Wattenberg, M. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL http://papers.nips.cc/paper_files/pap er/2023/hash/81b8390039b7302c909cb76 9f8b6cd93-Abstract-Conference.html.

Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering Llama 2 via Contrastive Activation Addition. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pp. 15504– 15522. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.ACL- LONG.828. URL https://doi.org/10.18653/v1/2024.acl -long.828.

Liu, S., Ye, H., Xing, L., and Zou, J. Y. In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openreview.net/forum ?id=dJTChKgv3a.

Shin, T., Razeghi, Y., IV, R. L. L., Wallace, E., and Singh, S. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 4222–4235. Association for Computational Linguistics, 2020. doi: 10.18653/V1/2020.EMN LP-MAIN.346. URL https://doi.org/10.186 53/v1/2020.emnlp-main.346.

Marks, S. and Tegmark, M. The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. CoRR, abs/2310.06824, 2023. doi: 10.48550/ARXIV.2310.06824. URL https: //doi.org/10.48550/arXiv.2310.06824. arXiv: 2310.06824.

Soulos, P., McCoy, R. T., Linzen, T., and Smolensky, P. Discovering the Compositional Structure of Vector Representations with Role Learning Networks. In Alishahi, A., Belinkov, Y., Chrupala, G., Hupkes, D., Pinter, Y., and

Nguyen, D., Prasad, A., Stengel-Eskin, E., and Bansal, M. Multi-Attribute Steering of Language Models via Tar11

Steer Like the LLM: Activation Steering that Mimics Prompting

Sajjad, H. (eds.), Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2020, Online, November 2020, pp. 238–254. Association for Computational Linguistics, 2020. doi: 10.18653/V1/2020.BLACKBO XNLP-1.23. URL https://doi.org/10.18653 /v1/2020.blackboxnlp-1.23.

Wang, W., Yang, J., and Peng, W. Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025b. URL https: //openreview.net/forum?id=8WQ7VTfPTl. Wu, Z., Arora, A., Wang, Z., Geiger, A., Jurafsky, D., Manning, C. D., and Potts, C. ReFT: Representation Finetuning for Language Models. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. URL http://papers.nips.cc/paper_files/pap er/2024/hash/75008a0fba53bf13b0bb3b7 bff986e0e-Abstract-Conference.html.

Stolfo, A., Balachandran, V., Yousefi, S., Horvitz, E., and Nushi, B. Improving Instruction-Following in Language Models through Activation Steering. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https://openreview.n et/forum?id=wozhdnRCtw. Subramani, N., Suresh, N., and Peters, M. E. Extracting Latent Steering Vectors from Pretrained Language Models. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 566–581. Association for Computational Linguistics, 2022. doi: 10.18653/V1/2022.FINDINGS-ACL.48. URL https://doi.org/10.18653/v1/2022.fin dings-acl.48.

Wu, Z., Arora, A., Geiger, A., Wang, Z., Huang, J., Jurafsky, D., Manning, C. D., and Potts, C. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. In Proceedings of the 42nd International Conference on Machine Learning, March 2025a. URL https://openreview.net/forum?id=K2Cc kZjNy0.

Sun, J., Baskaran, S., Wu, Z., Sklar, M., Potts, C., and Geiger, A. HyperSteer: Activation Steering at Scale with Hypernetworks, June 2025. URL http://arxiv.or g/abs/2506.03292. arXiv:2506.03292 [cs].

Wu, Z., Yu, Q., Arora, A., Manning, C. D., and Potts, C. Improved Representation Steering for Language Models. In Proceedings in The 39th Annual Conference on Neural Information Processing Systems, 2025b. URL https: //openreview.net/forum?id=VHb883Gs1u.

Turner, A. M., Thiergart, L., Udell, D., Leech, G., Mini, U., and MacDiarmid, M. Activation Addition: Steering Language Models Without Optimization. CoRR, abs/2308.10248, 2023. doi: 10.48550/ARXIV.2308. 10248. URL https://doi.org/10.48550/arX iv.2308.10248. arXiv: 2308.10248.

Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui, Z., Zhang, Z., and Qiu, Z. Qwen2.5 Technical Report. CoRR, abs/2412.15115, 2024. doi: 10.48550/ARXIV.2412.15115. URL https://do i.org/10.48550/arXiv.2412.15115. arXiv: 2412.15115.

Vogels, A., Wong, B., Choho, Y., Blangero, A., and Bhan, M. In-Distribution Steering: Balancing Control and Coherence in Language Model Generation. CoRR, abs/2510.13285, 2025. doi: 10.48550/ARXIV.2510.13 285. URL https://doi.org/10.48550/arXiv .2510.13285. arXiv: 2510.13285. Wang, H., Wang, G., and Zhang, H. Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pp. 29947–29957. Computer Vision Foundation / IEEE, 2025a. doi: 10.1109/CVPR52734.2025.02787. URL https://openaccess.thecvf.com/conten t/CVPR2025/html/Wang_Steering_Away_f rom_Harm_An_Adaptive_Approach_to_Def ending_Vision_CVPR_2025_paper.html.

Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., and Hou, L. Instruction-Following Evaluation for Large Language Models. CoRR, abs/2311.07911, 2023a. doi: 10.48550/ARXIV.2311.07911. URL http s://doi.org/10.48550/arXiv.2311.07911. arXiv: 2311.07911. Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large Language Models are HumanLevel Prompt Engineers. In The Eleventh International Conference on Learning Representations, 2023b. URL 12

Steer Like the LLM: Activation Steering that Mimics Prompting

https://openreview.net/forum?id=92gv k82DE-. Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., Goel, S., Li, N., Byun, M. J., Wang, Z., Mallen, A., Basart, S., Koyejo, S., Song, D., Fredrikson, M., Kolter, J. Z., and Hendrycks, D. Representation Engineering: A Top-Down Approach to AI Transparency, 2023. URL http://ar xiv.org/abs/2310.01405. arXiv:2310.01405 [cs].

Steer Like the LLM: Activation Steering that Mimics Prompting

A. Analyzing Prompt Steering Interventions A.1. Prompt Intervention Strength Across Layers Figure 4 plots the average magnitudes of the local and accumulative prompt steering interventions ∥∆P Sloc ∥ and ∥∆P Sacc ∥ across layers on the Persona Vectors dataset. For a given language model, the patterns are remarkably consistent across the three target traits. In absolute terms, the prompt intervention magnitude increases in later layers for both the local and accumulative effects. However, the local intervention magnitudes exhibit a dip in the early-to-mid layers for the Llama models and show a downward oscillating trend for Qwen2.5-7b-Instruct. Notably, the layer with the last high local steering contribution, or the layer immediately after it, corresponds to the layer selected by Chen et al. (2025) for single-layer steering in 8 out of 9 model-trait combinations. Similarly, the accumlative prompt steering intervention magnitudes start to plateau around this layer. Figure 5 provides further insight into how these interventions are distributed across tokens at different layers. The heatmaps confirm that prompt steering is not constant across token positions: both local and accumulative intervention strengths vary substantially. For all three models, there are layers where nearly no local steering occurs except on the first few question tokens. These low-contribution layers are interleaved with layers that exhibit broader steering activity, creating an alternating pattern across depth. Outside of these low-contribution layers, the token positions that receive the strongest steering are fairly consistent across layers. A.2. Intervention Strength Examples Figures 6-8 visualize the strengths ∥∆X (·)∥2 of the interventions from different steering methods on two examples from the Persona Vectors sycophantic evaluation set for Llama-3.2-3B-Instruct. Each example contains the intervention strengths at the intervention layer (16) and at a later layer (26) for S-ConstLL|QR , S-PSRLL|QR , and A-PSRMSE|QR . We observe patterns in the locations where prompt steering exerts strong interventions, for instance, prompt steering consistently steers higher on sentence markers and positions that can be seen as branching points in the generation with respect to the sycophantic trait. We also see that these patterns are well-matched by the PSR methods, including the single-layer variant that is trained with the loglikelihood objective. For S-ConstLL|QR we see that the interventions at layer 26, start to resemble the pattern of prompt steering, indicating that the model can recover from the unfaithful regime introduced by constant steering to default behavior in later layers.

Steer Like the LLM: Activation Steering that Mimics Prompting

(a) Local prompt intervention magnitude ∥∆P Sloc ∥.

(b) Local prompt intervention magnitude relative to the norm of the prompt steered activations ∥∆relative P Sloc ∥ = ∥∆P Sloc ∥/∥AP S ∥.

(d) Accumulative prompt intervention magnitude relative to the norm of the prompt steered activations ∥∆relative P Sacc ∥ = ∥∆P Sacc ∥/∥AP S ∥.

Figure 4. Prompt intervention magnitude across layers, measured locally (per-layer) and accumulatively, in both absolute and relative terms. The relative magnitudes are computed by dividing by the norm of the prompt-steered activations ∥AP S ∥.

Steer Like the LLM: Activation Steering that Mimics Prompting (a) Local ∥∆P Sloc ∥ – evil.

(b) Local ∥∆P Sloc ∥ – hallucinating.

(d) Accumulative ∥∆P Sacc ∥ – evil (normalized).

(e) Accumulative ∥∆P Sacc ∥ – hallucinating (normalized).

(f) Accumulative ∥∆P Sacc ∥ – sycophantic (normalized).

Figure 5. Heatmaps of prompt intervention magnitude per token across layers for different traits on randomly selected examples. (a)–(c) show local interventions ∥∆P Sloc ∥; (d)–(f) show accumulative interventions ∥∆P Sacc ∥, normalized per layer by dividing by their total sum to better reveal the effect in early layers.

Steer Like the LLM: Activation Steering that Mimics Prompting

(a) Prompt steering.

(b) S-ConstLL|QR .

(d) A-PSRMSE|QR .

Figure 6. Example 1, Layer 16: Intervention strength ∥∆X (·)∥2 per token for different steering methods on a sycophantic evaluation example (Llama-3.2-3B-Instruct).

Steer Like the LLM: Activation Steering that Mimics Prompting

(a) Prompt steering.

(b) S-ConstLL|QR .

(d) A-PSRMSE|QR .

Figure 7. Example 1, Layer 26: Intervention strength ∥∆X (·)∥2 per token for different steering methods on a sycophantic evaluation example (Llama-3.2-3B-Instruct).

Steer Like the LLM: Activation Steering that Mimics Prompting

(a) Prompt steering.

(b) S-ConstLL|QR .

(d) A-PSRMSE|QR .

Figure 8. Example 2, Layer 16: Intervention strength ∥∆X (·)∥2 per token for different steering methods on a sycophantic evaluation example (Llama-3.2-3B-Instruct).

Steer Like the LLM: Activation Steering that Mimics Prompting

(a) Prompt steering.

(b) S-ConstLL|QR .

(d) A-PSRMSE|QR .

Figure 9. Example 2, Layer 26: Intervention strength ∥∆X (·)∥2 per token for different steering methods on a sycophantic evaluation example (Llama-3.2-3B-Instruct).

Steer Like the LLM: Activation Steering that Mimics Prompting

B. Prompt Templates for Coherence Evaluation Jcoher I am evaluating a different language model. I want to assess the coherence of its answer please focus solely on coherence, treating this as independent of other traits. I asked the model the following evaluation question: [QUESTION START] {question} [QUESTION END] The model answered with: [ANSWER START] {answer} [ANSWER END] You should rate the answer on a scale of 0 to 100, focusing on how coherent the model's answer is in the context of the question. * 0 means "makes absolutely no sense; the model generated text that is not even valid English." * 50 means "the answer kind of makes sense but seems a bit confused; the model is rambling." * 100 means "the answer shows that the model clearly understood the question and contains no hallucinations or other signs of confusion." Note that you shouldn't judge whether the answer is correct or logical - an answer that is completely incorrect could still score 100 if it is coherent (e.g., something a mean person might say). Respond with a number between 0 and 100. Don't say anything else, just the number. Figure 10. Prompt template for coherence judge Jcoher (Chen et al., 2025) that is used for the Persona Vectors benchmark.

I am evaluating a different language model. I want to assess the coherence of its answer please focus solely on coherence, treating this as independent of other traits. I asked the model the following evaluation question: [QUESTION START] {question} [QUESTION END] The model answered with: [ANSWER START] {answer} [ANSWER END] You should rate the answer on a scale of 0 to 100, focusing on how coherent the model's answer is in the context of the question. * 0 means "makes absolutely no sense; the model generated text that is not even using a valid language." * 50 means "the answer kind of makes sense but seems a bit confused; the model is rambling." * 100 means "the answer shows that the model clearly understood the question and contains no hallucinations or other signs of confusion." Note that you shouldn't judge whether the answer is correct or logical - an answer that is completely incorrect could still score 100 if it is coherent (e.g., something a mean person might say). Respond with a number between 0 and 100. Don't say anything else, just the number. Figure 11. Slight variation on the coherence judge prompt template from Chen et al. (2025), which is used on the IFEval benchmark.

Steer Like the LLM: Activation Steering that Mimics Prompting

C. Steered Layers The layer indexes (starting from 1) where interventions are for the single-layer activation steering methods are listed in Tables 4 and 5. Table 4. Steered layer indices (starting from 1) for each trait and model on the Persona Vectors dataset. Trait Evil Sycophantic Hallucinating

meta-llama/Llama-3.2-3B-Instruct

meta-llama/Llama-3.1-8B-Instruct

Qwen/Qwen2.5-7B-Instruct

16 16 16

20 20 16

Table 5. Steered layer indices (starting from 1) for each instruction type and model on the IFEval dataset. Phi-3-mini-instruct

Gemma-2-2b-it

Mistral-7B-Instruct

Gemma-2-9b-it

7 27 19 16 16 16 16 21 16 21 17 21 16 16 16 16 16 16 16 17 21 16 16 21 16 21

10 12 18 13 13 13 16 13 13 13 16 13 13 13 13 13 13 13 13 18 13 13 18 12 13 14

16 29 19 16 16 16 16 25 16 16 17 16 16 16 16 16 16 16 16 16 16 16 16 15 16 16

15 27 24 21 21 21 9 39 21 21 21 21 21 24 21 21 27 21 30 24 24 21 21 24 21 21

instruction id change case:capital word frequency change case:english capital change case:english lowercase detectable format:constrained response detectable format:json format detectable format:multiple sections detectable format:number bullet lists detectable format:number highlighted sections detectable format:title language:response language ar language:response language de language:response language fa language:response language gu language:response language hi language:response language kn language:response language ko language:response language mr language:response language ne language:response language pa language:response language ru language:response language sw language:response language te language:response language ur punctuation:no comma startend:end checker startend:quotation

D. Computing Trait Alignment at Target Coherence First, we explore the steering coefficient α (as defined in Equations 2 and 7) in areas where the steering method achieves coherence close to the target coherence using the binary search procedure outlined in Algorithm 1. Next, we interpolate the trait alignment at the target coherence using the trait alignments obtained at the two coherence levels that are immediately above and below the target coherence.

Steer Like the LLM: Activation Steering that Mimics Prompting

Algorithm 1 Binary search for steering coefficient αmin := 0.0 αmax := 10.0 for i = 1 to Nsteps do α := (αmin + αmax )/2 c := evaluate coherence(α) { Average coherence of the predictions from the steering method using α. } if |αmax − αmin | < 0.01 then break end if if c > target coherence then αmin := α else αmax := α end if end for

Steer Like the LLM: Activation Steering that Mimics Prompting

E. Additional Steering Results This appendix provides steering results and analysis that were omitted from the main text for brevity. E.1. Persona Vectors Additional Results Table 6. Results on the Persona Vectors dataset for Llama-3.2-3b-Instruct, Llama-3.1-8b-Instruct, and Qwen2.5-7b-Instruct. For different steering methods, we report trait alignment at coherence 80.0 (TA@C80 ) and at prompt steering coherence (TA@Cprompt ), both are macro-averaged over the different traits. Llama-3.2-3b TA@C80 TA@Cprompt

Llama-3.1-8b TA@C80 TA@Cprompt

Qwen2.5-7b-instruct TA@C80 TA@Cprompt

46.1 64.8 72.5 78.1 79.3 82.7 89.6 91.8 91.1

28.9 43.8 42.9 55.2 57.4 49.8 52.6 71.2 66.8

49.8 87.7 88.4 87.9 89.0 97.4 96.8 99.1 98.8

30.2 42.9 44.0 51.1 50.1 44.5 45.0 76.5 74.7

74.8 70.9 69.5 73.5 71.6 85.5 83.3 87.2 83.3

34.8 52.3 51.8 50.5 48.8 58.1 59.1 61.5 60.9

A-ConstLL|QR A-ConstMSE|QR A-PSRLL|QR A-PSRMSE|QR

98.2 98.9 97.5 98.6

85.6 95.8 94.4 92.5

98.8 98.9 98.4 99.2

85.9 91.3 82.3 96.4

96.1 96.1 95.3 96.8

73.6 83.6 65.7 83.9

91.5

95.7

71.6

prompt

Figure 12. Trait alignment on the Persona Vectors dataset after steering Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, and Qwen2.5-7bInstruct. Trait alignments for all steering methods are computed at prompt steering coherence.

E.2. IFEval Additional Results Table 7 provides results that were omitted from Table 2 in the main text for brevity. Table 8 contains the results on the IFEval format dataset after filtering out instruction types that require arguments (e.g., the number of sections to include). Specifically, we filtered the following instruction types: multiple sections, number bullet lists, end checker, number highlighted sections, and capital word frequency. E.3. AxBench: Breakdown of Judge Scores Table 9 shows the breakdown of the overall steering scores on AxBench in their average concept alignment, coherence, and relevance to the base prompt.

Steer Like the LLM: Activation Steering that Mimics Prompting

Figure 13. Trait alignment-coherence curves for different steering methods on the Persona Vectors dataset. Some curves have a region where trait alignment and coherence both go down, this points to oversteering (i.e., when α values are set too high for a method).

Steer Like the LLM: Activation Steering that Mimics Prompting Table 7. Complete results on the IFEval format dataset including all configurations. For different steering baselines we report the instruction-following accuracy IF Acc. and coherence Coher., both are macro-averaged over the different instruction types. IF Acc. is computed using the IFEval script as in Stolfo et al. (2025). Coherence scores are computed following Chen et al. (2025) using LLM-as-a-judge. Activation steering results that outperform prompting are underlined. ∗ no activation steering and no instruction in the prompt. b Results reproduced with code from Stolfo et al. (2025). Phi-3-mini-instruct IF Acc. Coher.

Gemma-2-2b-it IF Acc. Coher.

Mistral-7B-Instruct IF Acc. Coher.

Gemma-2-9b-it IF Acc. Coher.

no steering∗ Stolfo et al. (2025) a Stolfo et al. (2025) b S-ConstLL S-ConstMSE S-PSRLL S-PSRMSE

11.9 30.1 29.0 11.6 12.4 62.8 29.3

92.4 86.5 91.6 93.4 89.1 91.3

10.6 30.1 39.1 10.7 10.6 54.9 39.0

94.3 88.8 94.5 94.5 89.5 92.9

6.8 14.1 19.8 19.0 8.7 62.7 22.3

90.5 89.8 89.2 90.1 87.6 89.0

11.4 28.9 30.8 13.4 12.8 66.1 47.5

96.6 96.1 96.7 96.6 95.5 96.4

A-ConstLL A-ConstMSE A-PSRLL A-PSRMSE

61.9 13.4 69.0 48.8

90.0 93.4 85.4 87.7

36.9 18.4 68.7 61.2

90.5 94.0 90.6 91.2

69.2 37.0 61.1 54.1

81.1 83.9 84.1 85.7

50.4 19.0 71.9 71.3

94.4 96.9 82.3 95.1

prompt

72.5

84.6

66.8

88.6

61.8

81.5

85.7

94.8

Stolfo et al. (2025) +prompt a Stolfo et al. (2025)+prompt b S-ConstLL +prompt S-ConstMSE +prompt S-PSRLL +prompt S-PSRMSE +prompt

78.6 81.7 78.9 73.8 89.8 81.6

79.3 83.2 83.9 82.2 79.9

76.1 79.0 74.2 78.1 83.2 82.6

84.0 87.1 87.9 84.5 87.2

63.7 62.5 77.6 66.0 85.5 67.8

80.6 77.3 78.2 75.5 76.9

86.6 88.7 91.5 90.9 93.1 91.1

94.6 94.3 94.8 94.6 94.0

A-ConstLL +prompt A-ConstMSE +prompt A-PSRLL +prompt A-PSRMSE +prompt

86.0 85.7 82.8 85.2

79.6 80.0 80.2 80.6

82.3 82.8 86.4 81.3

83.6 86.3 84.6 86.1

77.9 76.2 82.0 68.9

73.5 76.8 76.0 78.1

84.8 93.6 87.6 92.4

91.0 93.7 80.0 93.5

Table 8. Results on the IFEval format dataset after filtering out instruction types that require arguments (e.g., “Your response must contain 3 sections”). The best IF Acc. scores for activation steering methods (top) and activation steering with prompting (bottom) are in bold, activation steering results that outperform prompting are underlined. Phi-3-mini-instruct IF Acc. Coher.

Gemma-2-2b-it IF Acc. Coher.

Mistral-7B-Instruct IF Acc. Coher.

Gemma-2-9b-it IF Acc. Coher.

no steering∗ Stolfo et al. (2025) b S-ConstLL S-ConstMSE S-PSRLL S-PSRMSE

7.1 34.0 9.0 5.5 69.7 29.3

92.4 82.4 91.6 92.8 86.7 90.3

1.6 39.9 4.8 4.0 60.7 42.9

94.2 87.0 94.2 93.9 88.8 91.2

3.3 22.5 17.7 6.0 72.6 27.2

89.4 88.4 87.9 89.1 87.6 88.7

3.4 32.1 5.8 5.8 72.3 56.7

96.1 95.0 96.5 96.2 94.9 95.5

A-ConstLL A-ConstMSE A-PSRLL A-PSRMSE

64.9 9.4 78.8 58.4

88.1 93.6 81.7 86.6

37.9 13.8 77.6 65.9

89.3 93.7 89.4 89.6

76.6 47.4 68.7 61.0

77.7 80.5 82.2 84.3

59.1 15.2 78.1 78.7

95.0 96.5 82.6 94.7

67.6

79.5

67.8

86.1

63.4

79.4

85.4

93.9

79.3 75.6 71.9 88.8 80.4

75.1 79.7 79.9 76.2 74.3

88.0 72.9 78.5 90.9 86.7

79.2 84.5 85.3 80.1 83.7

65.2 80.2 70.0 86.8 71.7

79.9 75.0 75.9 73.4 75.7

90.0 90.2 92.5 96.6 92.7

93.5 93.5 93.6 94.1 93.0

89.3 86.1 89.8 86.6

75.2 74.4 75.0 75.6

87.0 88.0 92.1 83.1

78.9 84.1 81.7 82.9

82.2 83.4 83.6 81.4

71.5 72.9 73.2 75.1

85.2 95.4 93.3 93.2

92.0 92.9 79.3 92.3

prompt Stolfo et al. (2025)+prompt S-ConstLL +prompt S-ConstMSE +prompt S-PSRLL +prompt S-PSRMSE +prompt A-ConstLL +prompt A-ConstMSE +prompt A-PSRLL +prompt A-PSRMSE +prompt

Steer Like the LLM: Activation Steering that Mimics Prompting

Table 9. Breakdown of AxBench steering scores into concept alignment, relevance to the base prompt, coherence, and the combined score for the 2BL20 and 9BL20 splits. (a) 2BL20 split.

Jconc. Jrelev. Jcoher. Jcomb.

prompt

S-ConstLL

S-PSRLL

S-ConstMSE

S-PSRMSE

A-ConstLL

A-PSRLL

A-ConstMSE

A-PSRMSE

0.719 1.760 1.072 0.720

0.588 1.630 0.944 0.504

0.794 1.466 0.990 0.618

0.350 1.782 1.008 0.311

0.433 1.724 1.028 0.367

1.010 1.435 1.040 0.792

0.779 1.700 1.046 0.690

1.016 1.281 1.022 0.783

1.064 1.427 1.042 0.871

(b) 9BL20 split.

Jconc. Jrelev. Jcoher. Jcomb.

prompt

S-ConstLL

S-PSRLL

S-ConstMSE

S-PSRMSE

A-ConstLL

A-PSRLL

A-ConstMSE

A-PSRMSE

1.055 1.858 1.132 1.054

0.821 1.338 1.023 0.633

0.847 1.541 1.131 0.667

1.043 1.492 1.077 0.903

0.998 1.620 1.096 0.896

0.996 1.444 1.038 0.757

0.947 1.660 1.076 0.827

1.260 1.550 1.044 1.053

1.202 1.743 1.096 1.120

Steer Like the LLM: Activation Steering that Mimics Prompting

F. Faithfulness Additional Results Figure 14 shows the relative RMSE between activations produced by prompt steering versus other steering methods, averaged on prompt steering predictions on the Persona Vectors evaluation data for Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, and Qwen2.5-7b-Instruct, respectively.

(a) Llama-3.2-3B-Instruct.

(b) Llama-3.1-8B-Instruct.

Figure 14. Relative RMSE between activations produced by prompt steering versus other steering methods, averaged on prompt steering predictions on the Persona Vectors evaluation data.

Steer Like the LLM: Activation Steering that Mimics Prompting

G. Steering Vectors Comparison Figure 15 visualizes the cosine similarity between the steering vectors zattr,l produced by different steering methods at the intervention layer l. We observe that the all-layer methods learn steering vectors with low pairwise similarity, even in layers where their respective activations are faithful to prompt steering (A-PSRMSE vs A-ConstMSE from the early middle layers). Contrasting this with the faithfulness results in Figure 14 suggests that activations faithful to prompt steering can be obtained in different ways, and that the steering vector of A-PSRMSE or A-ConstMSE at a given layer does not necessarily reflect how prompt steering operates in that layer. This can be explained by the fact that the all-layer settings jointly optimize the steering vectors, so each layer’s vector is shaped by gradients from later layers rather than reflecting only that layer’s contribution. This is desirable when the goal is to produce the most faithful activations or the best steering performance, but complicates the interpretation of the steering vectors. We conclude that, to understand prompt steering mechanics in a specific layer, it is more suitable to replace and replicate one layer at a time within an otherwise prompt-steered forward pass.11 Another pattern we observe is that single-layer methods learn steering vectors that are more similar to each other than the all-layer methods and the single-layer steering vectors are near-orthogonal to their all-layer counterparts. This is to be expected as the single-layer methods are trained to capture the entire effect of prompt steering in a single layer, while the all-layer methods can distribute the effect across layers. When comparing the traditional ConstDiM method to the other methods, we see most similarity with the PSRMSE variants.

11 Note that this differs from the single-layer + MSE variants in this paper, which aim to capture the entire effect of prompt steering in a single layer.

Steer Like the LLM: Activation Steering that Mimics Prompting (a) Pairwise similarities between S-ConstDiM and the all-layer methods across layers.

(b) Pairwise similarities between all methods at the single intervention layer (layer 16).

Figure 15. Cosine similarity between steering vectors learned by different methods for sycophancy on Llama-3.2-3B-Instruct.

Record · ID 155284 · SHA-256 5f5b280799b4ed5f
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.