IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
1
Towards the Readability of LLM-Generated Codes through Multitask Representation Engineering
arXiv:2606.06214v1 [cs.SE] 4 Jun 2026
Huifan Gao, Liuhua He, Yinghui Pan, Shenbao Yu, Yifeng Zeng, Shengchao Qin, Weidi Sun
Abstract—Correctness and readability are key measures of code quality, respectively ensuring functional fidelity and ease of comprehension. While most existing research focuses on improving the correctness of large language models (LLMs) generated codes, readability remains under-addressed. Enhancing readability through targeted control is challenging due to its subjective nature. In this article, we employ representation engineering (RepE) as the targeted control method given its characteristics of low data dependency and low computational cost. Prior work on RepE has primarily focused on the targeted control for a single task, but improving the code readability requires the control across multiple tasks. Accordingly we proposes the multitask RepE framework and theoretically discuss the impact of the multitask steering method on the tradeoff between the code readability and correctness. We further provide comprehensive experiments in support. All the relevant implementations are open-source and available upon request. Index Terms—Large language models, code readability, code correctness, representation engineering, multitask steering.
I. I NTRODUCTION N software engineering, code correctness and readability are both critical metrics for assessing code quality [1]. The code correctness refers to whether the code executes strictly according to predefined requirements and produces accurate results. The code readability, which is however subjective, refers to whether the code is easy to understand and maintain. What are comprehensive factors impacting the code readability is still an open question in the code quality research [2]. In this article, we mainly focus on the three commonlyrecognized evaluation metric including (a) comment density, (b) naming conventions, and (c) cyclomatic complexity [3]– [5].These three metrics characterize code readability from complementary perspectives: comment density reflects the sufficiency of explanatory information, naming conventions affect whether identifiers convey clear semantics, and cyclomatic complexity measures the structural difficulty of understanding the control flow. Although they cannot cover all readabilityrelated factors, they provide representative and measurable
I
Huifan Gao is with the School of Aerospace Engineering, Xiamen University, Xiamen, China (e-mail:[email protected]). Liuhua He is with the School of Artificial Intelligence, Shenzhen University, Shenzhen, China (e-mail:[email protected]). Yinghui Pan is with the School of Artificial Intelligence, Shenzhen University, Shenzhen, China (e-mail: [email protected]). Shenbao Yu is with the College of Computer and Cyber Security, Fujian Normal University, Fuzhou, China (e-mail:[email protected]). Yifeng Zeng is with the Department of Computer & Information Sciences, Northumbria University, UK (e-mail:[email protected]). Shengchao Qin is with the School of Computer Science and Technology, Xidian University, Xi’an, China (e-mail:[email protected]). Weidi Sun is with Peking University, Beijing, China (email:[email protected]).
dimensions for analyzing the readability of LLM-generated code. As shown in Fig. 1, the evaluation of code readability is inherently subjective, varying across developers and experience levels. For example, for the initially generated code in the left side of Fig. 1, Users 1, 2, and 3 provide different critiques regarding its readability, each focusing on distinct aspects. This variation is closely related to factors such as the users’ development experience and personal coding habits. Even after the code is optimized based on the requirements of the first three users, User 4 still believes that the code’s readability needed further improvement. This clearly demonstrates that, given the inherently subjective nature of code readability, it is nearly impossible to evaluate it through a single, universal set of standards. Different users may expect different readability properties: novice programmers may prefer more explicit comments, whereas experienced developers may focus more on concise naming and simplified logic. This motivates a controllable method that can adapt to multiple readability preferences rather than optimizing a single static readability score. Before the advent of large language models (LLMs), research on code generation faced significant challenges in ensuring the code correctness [6]–[8]. In recent years, with the advancement of LLMs, many developers have been using LLMs to generate codes on demand to support software engineering. In AI-assisted programming, LLM-generated code is often not the final artifact used in isolation. It usually needs to be reviewed, modified, debugged, and maintained by human developers. Thus, code that passes test cases may still be unsatisfactory if it lacks necessary comments, uses obscure identifiers, or contains unnecessarily complex control flow. Extensive study has been conducted on improving the LLMgenerated codes’ quality [9], [10], and have gradually recognized the importance of the code readability [11]. However, research aimed specifically at improving the readability of LLM-generated code remains very limited. This gap indicates that correctness and readability should be considered jointly. On the one hand, readability-oriented intervention can improve the usability and maintainability of generated code. On the other hand, such intervention should not substantially damage functional correctness, because readable but incorrect code is still unacceptable in practical software development. Improving the readability of codes generated by LLMs can be viewed as a form of targeted control over model behavior [12]. Furthermore, targeted control approaches used in real-world production environments must be capable of rapidly adapting to subjective interpretations of code readabil-
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
Please write a function to reverse words in a given string. def rw(a): w=[] cu='' for ch in a: if ch== '': if cu: w.append(cu) cu='' else: cu+=ch if cu: w.append(cu) r=[] for i in range(len(w)-1,-1,-1): r.append(w[i]) return''. join(r)
Code Generation (Round 1)
2
As a beginner in coding, this code has no comments, making User1 it hard for me to understand.
User2
User3
Please optimize your code based on the above feedback.
The naming in this code violates naming conventions, which is detrimental to understanding.
Very good. This code is quite Very This code is quite easygood. to understand. Very This code is quite easygood. to understand. easy to understand.
def reverse_words(input_string): # Split input string into words words = input_string.split() # Reverse the words order reversed_words = words[::-1]
For me, a seasoned programmer, the cyclomatic complexity of this code is so high that creates obstacles for me when trying to understand its overall logic.
# Join words back into a string return ''.join(reversed_words)
Users’ Feedbacks
Code Generation (Round 2)
I believe that the readability of the optimized code still needs to be improved. I think this code has bad comments. It is necessary to add a /*......*/ format comment section at the beginning of a function to User4 introduce its functionality.
Users’ Feedbacks
Fig. 1. For different users with varying experience levels, the format of comments, naming conventions, and cyclomatic complexity can lead to differences in code readability. While User 1 (2 or 3) deems it clear and easy to understand, User 4 insists that its readability is still insufficient, advocating for the addition of a formal docstring.
ity among different users. As an approach for targeted control of model behavior, representation engineering (RepE) [13], [14] achieves precise manipulation by extracting steering vectors associated with specific concepts and directly injecting them into the model’s internal activation layers. Compared to other targeted control approaches (e.g., full-parameter finetuning [15]–[17], parameter-efficient fine-tuning [18]–[20], and preference tuning [21]–[23]), RepE does not adjust model weights. Instead, it requires only a few dozen data samples targeted at a specific objective to quickly derive steering vectors that effectively control model behavior, thereby eliminating the need for large training datasets or extensive computational overhead. Given these advantages, we select RepE as the foundational approach for improving code readability in this paper. This property is suitable for readability control because different readability requirements can be represented as different steering directions, and their strengths can be adjusted during inference. As a result, RepE provides a lightweight way to transform subjective readability preferences into controllable changes in the hidden representations of LLMs, without repeatedly fine-tuning the model for each new preference. Currently, most of the RepE work focuses on improving the model performance in a single task (referred to as a single-task steering). However, using RepE to improve the code readability requires addressing the challenge of coordinated control across multiple metrics that constitute code readability, namely comment density, naming conventions, and cyclomatic complexity (referred to a multitask steering). This multitask setting is more challenging than ordinary single-task steering. If steering vectors for different readability metrics are extracted independently and injected simultaneously, they may interfere with each other in the hidden representation space. For example, encouraging more comments may also affect the generated code structure, while reducing cyclomatic complexity may influence naming or decomposition choices. Therefore, improving code readability through RepE requires a mechanism that coordinates multiple steering vectors rather than simply combining them. Furthermore, when discussing how to improve the code readability, the impact on the code
correctness must also be considered. Although the existing research [14] has explored the trade-off between the singletask steering behavior and the model capability, a unified framework for the multitask steering is still lacking (i.e., the trade-off between the code readability and correctness in our work). In a nutshell, we aim to improve the LLM-generated code readability through RepE by addressing the following two challenges: (a) How to achieve the coordinated control over the three metrics that constitute the code readability? (b) How to theoretically quantify the control impact on the code readability and correctness, and balance the trade-off between them? To address these challenges, we propose a multitask steering RepE framework (MRepE) to improve the code readability, and analyze the limit of its impact on the code correctness. Methodologically, we propose the joint principal component analysis algorithm with multidimensional orthogonal constraints (MOC-JPCA). It introduces multidimensional orthogonal constraints and iteratively estimates the first principal direction for each dimension on the Stiefel manifold [24], thereby explicitly suppressing mutual interference between different code readability steering vectors. This provides stable and complementary steering bases for subsequent joint injection. Compared with independently extracting steering vectors for each metric, MOC-JPCA explicitly considers the geometric relationship among different steering directions, making the extracted vectors more suitable for coordinated multitask injection. Subsequently, we analyze the functional relationship between the injection coefficients and the change in the finallayer representations, establishing two main theoretical results: a lower bound for the improvement in the code readability and an upper bound for the negative impact on the code correctness. The main contributions are: We propose the MRepE framework to improve the code readability through multitask steering vectors. • We propose the corresponding algorithm (MOC-JPCA) to extract steering vectors in MRepE and analyze its convergence in a theoretical way. • We provide both thoeretical and empirical support in the •
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
MRepE performance. Specifically, we theoretically characterize the readability-correctness trade-off and empirically validate the proposed framework on multiple code LLMs. II. R ELATED W ORKS Code readability has long been regarded as an important factor in software comprehension, software maintenance, and long-term code quality. Traditional studies usually attempt to relate developers’ subjective perceptions of readability to measurable code-level features. For example, Buse and Weimer [3] learned a readability metric from human annotations, while Posnett et al. [4] proposed a simpler readability model based on compact textual and structural factors. Scalabrino et al. [5] further showed that code understandability is affected by multiple source-code and documentation-related factors. These studies indicate that code readability is inherently multidimensional rather than determined by a single universal criterion. Therefore, this work focuses on three commonly recognized and practically controllable dimensions of code readability, namely comment density, naming conventions, and cyclomatic complexity. The LLMs’ application in software engineering has become increasingly widespread. Particularly, the use of LLMs for code generation has significantly enhanced productivity. However, the existing research still mainly aims to improve the correctness of LLM-generated codes [9], [10]. For example, Bui et al. [25] proposed the OPENIA framework, which directly evaluates code correctness by analyzing the model’s internal representations (e.g., hidden states). It outperformed traditional black-box methods on multiple benchmarks and supported more efficient quality control. This line of research is important because generated code must first satisfy functional requirements before being adopted in practical software development. Nevertheless, correctness-based evaluation mainly measures whether a generated program solves the given task, but does not fully reflect whether the program can be easily reviewed, modified, and maintained by human developers. A code snippet that passes test cases may still be difficult to understand if it lacks explanatory comments, uses obscure identifiers, or contains unnecessarily complex control flow. This gap has motivated increasing attention to the readability of LLM-generated code. Wannita et al. [11] stated that, from the perspective of software engineers, the code readability remains critical in both the pre-LLMs and LLMs eras, and revealed the potential of LLMs in generating readable codes. Despite this importance, there is still no widely used quantitative method or model to assess the code readability [26]. In particular, the research that explicitly improves the readability of LLM-generated codes remain rather limited. Most existing studies either evaluate the readability of generated code after generation or rely on prompt engineering to encourage readable outputs. However, prompt-level control may be unstable across models and tasks, and it does not directly explain how readability-related behaviors are represented inside the model. This motivates us to actively control readability-related behaviors of LLMs through their internal representations.
3
Representation engineering (RepE) has appeared as an alternative LLM tuning method. Zou et al. [13] introduced a topdown RepE technique, such as linear artificial tomography to analyze and control high-level cognitive representations (e.g., honesty and morality) in deep neural networks. Building on this, Wolf et al. [14] revealed that RepE in LLM alignment methods linearly improves attack resistance and bias control capabilities, but quadratically impairs performance on base tasks. Compared with full-parameter fine-tuning, parameterefficient fine-tuning, and preference optimization, RepE does not update model parameters. Instead, it manipulates model behavior by extracting steering directions associated with target concepts and injecting them into hidden representations during inference. This property makes RepE suitable for code readability control, where collecting large-scale userspecific preference data for every readability style is often impractical. Moreover, the steering strength can be adjusted at inference time, which provides a flexible mechanism for balancing readability improvement and possible side effects on the original code-generation capability. The aforementioned research on RepE primarily focuses on improving model performance on a single task. In contrast, improving code readability requires coordinated control across multiple dimensions, such as comment density, naming conventions, and cyclomatic complexity. Recent studies on activation steering also indicate that steering multiple behaviors is more challenging than steering a single behavior. For example, directly combining multiple steering vectors into one vector may lead to ineffective or unstable control, whereas injecting individual steering vectors separately can be more promising [27]. This observation is closely related to our problem setting, because each readability metric may correspond to a different representation direction in the hidden space of an LLM. If these directions are extracted independently and injected simultaneously, their geometric correlation may cause mutual interference and reduce the controllability of each metric. To address this challenge, we propose a RepE approach for multitask steering and apply it to enhance the readability of LLM-generated codes. Specifically, we formulate the extraction of multiple readability steering vectors as a joint principal component analysis problem with multidimensional orthogonal constraints. In this way, each steering direction is encouraged to capture the dominant representation variation of its corresponding readability metric, while the orthogonality constraint explicitly suppresses interference among different metrics. Furthermore, we examine how this multitask steering influences the code correctness and present a theoretical framework that quantifies the limit of these effects. Therefore, our work differs from existing studies by jointly considering multitask readability control and the readability-correctness tradeoff in LLM-generated code, rather than treating readability improvement as an isolated single-objective intervention. III. I MPROVE C ODE R EADABILITY VIA MR EP E To improve the readability of code generated by LLMs, we propose the MRepE framework as shown in Fig. 2.
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
Low readability representation
Raw dataset Comment density
Hidden state representation
Large language model
Naming conventions
Cyclomatic complexity
Difference & mean-centered 𝑋!
𝑋"
𝑋#
Centered dataset
(a) Data Preparation #
MOC-JPCA
𝑋!
𝑑!
𝑋"
𝑑"
𝑋#
Orthogonality Constraint
# (')
𝑑#
(b) Steering Vector Extraction
# (')!)
(')")
# 𝑐$ 𝑑$
# 𝑐$ 𝑑$
# 𝑐$ 𝑑$
$%!
$%!
$%!
Layer 𝑙
Injection layers
(c) Steering Vector Injection
Fig. 2. The MRepE framework includes three stages. (Top) Datasets with diverse code readability metrics are provided for LLM to compute differences in hidden state representations, followed by a centering procedure. (Bottom left) Orthogonal steering vectors are extracted using MOC-JPCA. (Bottom right) The orthogonalized vectors are injected into selected layers of LLM to modulate and control its outputs.
Within this framework, we mainly focus on solutions to: (a) how to use steering vectors corresponding to multiple code readability metrics to control model output; and (b) how to extract these steering vectors through the MOC-JPCA algorithm. Different from weight-updating methods that require repeated optimization of the model parameters, MRepE treats readability improvement as a representation-level control problem. Specifically, it first identifies latent directions associated with different readability metrics from contrastive examples, and then injects these directions into selected hidden layers during inference. This design is particularly suitable for code readability control because different developers may emphasize different readability aspects, and such preferences should be adjusted without retraining the whole model. Therefore, the proposed framework separates the whole process into three stages: readability-oriented data preparation, orthogonal steering-vector extraction, and multitask steering-vector injection. A. Preliminary We define high comment density, well-established naming conventions, and low cyclomatic complexity as the good characteristics of LLM-generated code readability. These three dimensions correspond to different levels of human code comprehension. Comment density mainly reflects whether the generated code provides sufficient natural-language explanations for important operations. Naming conventions reflect whether variables, functions, and intermediate results are
0.7380
Readable Unreadable
0.7365
0.80
E
0.7350
0.860
0.70
0.7335
0.60
Readable Unreadable
0.75 0.856
0.54
E
0.852
0.7305
0.55
0
1
2
3
C CD
4
5
0.48 0.40
E
0.32
0.42 1.928
0.24 0.36
0.848 0.60
Readable Unreadable
1.936
0.48
0.65
0.7320
1.944
E
Cyclomatic complexity
High readability representation
expressed with meaningful and consistent identifiers. Cyclomatic complexity reflects whether the control-flow structure is simple enough for developers to trace and maintain. These dimensions represent three commonly observed sources of readability feedback: insufficient explanation, obscure naming, and unnecessarily complicated logic. In principle, one may measure the model’s relative preference by comparing the average negative log-likelihoods of code snippets with different levels of readability. However, the average negative log-likelihood of an entire code sequence is essentially a sequence-level aggregated quantity, which is jointly affected by length normalization, the proportion of attribute-irrelevant tokens, autoregressive dependence, and error accumulation. For the readability metrics considered in this work, the intervention effect may also be diluted in sequence-level averaging by a large number of tokens that are irrelevant to the target attribute. Therefore, sequencelevel likelihood is more suitable as supplementary behavioral evidence, rather than as the core object of theoretical analysis in this paper. Based on this consideration, we instead use short question-answering items that can be interpreted as binary judgments (e.g., yes/no) to test whether the model understands the indicators of highly readable code. Similarly, for code correctness, we employ multiple-choice questions to evaluate the model’s ability to identify the correct code snippet. In addition, in our pilot experiments, we observe that after injecting steering vectors that improve the model’s understanding of code readability metrics, the model’s ability to generate readable code also improves, and the two are positively correlated (as shown in Fig. 3). This suggests that the judgment-based task can serve as an effective proxy for actual generation behavior. Therefore, the following analysis focuses on the probability of generating positive answers in the judgment tasks, while the sequence-level generation preference is used as behavioral evidence to verify that the learned steering directions are also reflected in actual code generation. NLLreadable 1 ∑NLLunreadable K
Naming conventions
Low readability
High readability
Comment density
4
0.16
1.920 0.30
0.844
0.50 0.840
0.08
0.24 1.912
0.00
0.18 0
1
2
3
C NC
4
5
0
1
2
3
C CC
4
5
Fig. 3. Injecting steering vectors for different readability metrics into Deepseek R1 14b reveals a positive correlation between the model’s ability to generate readable code and its ability to understand code readability. The solid line shows the ratio of negative log-likelihood for generating readable versus unreadable code, where a lower ratio indicates a stronger tendency to generate readable code. The dashed line represents the expected accuracy of the model’s binary judgment on code readability, where a higher value corresponds to better judgment performance.
B. Steering Vector Injection Given an LLM model with L layers for code generation, (l) where hθ (1≤l≤L) denotes the l-th layer’s hidden state representation. For a given code quality-related query q = (t1 , · · · , tn ), where tn is the n-th token in the query, the
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
5
probability distribution of the next token generated by the model with parameters θ is given by (L) Pθ (tn+1 | t1 , · · · , tn ) = Softmax U hθ (t1 , · · · , tn ) tn+1
where U is the unembedding matrix that maps the hidden state back to the space of the token vocabulary. Here, the hidden (l) state hθ can be regarded as the intermediate representation through which the model encodes the semantic, syntactic, and task-related information contained in the input query. Since the final token probability is obtained by applying the unembedding matrix to the final-layer representation, a small and structured perturbation on intermediate hidden states can change the downstream answer distribution without modifying the model parameters. This is the key intuition behind steeringvector injection. We denote queries related to comment density, naming conventions, cyclomatic complexity, and code correctness as qcd , qnc , qcc , and qtf , respectively. A positive answer a+ indicates high code readability/correctness, and a negative answer a− indicates low code readability/correctness. The model’s performance on the query can be quantified as the probability of generating a positive answer X X Pθ (a+ | q) = 1 − Pθ (a− | q) a+
a−
For binary readability judgments, a+ usually corresponds to affirmative answers such as “yes”, while a− corresponds to negative answers such as “no”. For multiple-choice correctness judgments, a+ corresponds to the option index associated with the correct code snippet, and a− corresponds to the remaining options. We control the output of the model defined above by injecting code readability steering vectors. The set of code readability steering vectors is defined as n o (l) V = vk | k ∈ {1, 2, · · · , K}, l ∈ {1, 2, · · · , L} (1) (l)
(l)
In Eq. (1), vk =ck ·dk , where ck is the coefficient specifying (l) steering strength, and dk denotes steering direction. K is the number of code readability steering vectors (K = 3 in this paper). For layers that are not injected with code (l) readability steering vectors, we set ∥dk ∥ = 0. The coefficient ck determines how strongly the model is encouraged to follow the k-th readability direction. In this paper, the three directions correspond to comment density, naming conventions, and cyclomatic complexity. A larger coefficient imposes a stronger representation-level bias toward the corresponding readability metric, while a smaller coefficient leads to a more conservative intervention. Therefore, the coefficient vector c = (c1 , · · · , cK ) provides a direct interface for balancing the improvement of different readability dimensions. The model injected with the readability steering vectors is denoted as Pθ,V . The steering vectors are applied by modifying the hidden state representation at each layer as (l)
(l)
(l)
hθ,v(l) ← hθ (l) + v1 + · · · + vK
(2)
The additive form in Eq. (2) is simple but important. It implies that different readability preferences are jointly expressed as
a superposition of several steering directions in the hidden space. If these directions are highly correlated, the injected perturbations may reinforce or cancel each other in an unpredictable way. Therefore, simply extracting each steering direction independently is insufficient for stable multitask control, which motivates the orthogonal extraction algorithm introduced later. The probability that the model Pθ,V generates a positive answer is X X Pθ,V (a− | q) Pθ,V (a+ | q) = 1 − a− a+ P P By comparing a+ Pθ,V (a+ | q) and a+ Pθ (a+ | q), we can analyze the impact of the code readability steering vectors on bothP code readability and correctness. Specifically, P an increase in P (a | q ), P (a | qnc ), or θ,V + cd θ,V + a+ a+ P P (a | q ) indicates that the model becomes more θ,V + cc a+ sensitive to thePcorresponding readability metric. In contrast, a decrease in a+ Pθ,V (a+ | qtf ) indicates a possible degradation in the model’s correctness-related judgment ability. Hence, the same injection mechanism provides a unified way to quantify both the desired readability improvement and the possible side effect on correctness. C. Data Preparation for Readability Representations Before extracting steering vectors, MRepE constructs contrastive representation datasets for different readability metrics. For each metric k, we build a set of paired prompts Mk − {(x+ k,i , xk,i )}i=1
(3)
where x+ k,i denotes a code-related input that better satisfies the k-th readability metric, and x− k,i denotes its low-readability counterpart. For example, for the comment-density metric, − x+ k,i contains sufficient explanatory comments, whereas xk,i contains insufficient or missing comments. For the namingconvention metric, the positive example uses meaningful and conventional identifiers, whereas the negative example uses obscure or inconsistent identifiers. For the cyclomaticcomplexity metric, the positive example presents a simpler control-flow structure, whereas the negative example contains unnecessarily complex branches or loops. − For each pair (x+ k,i , xk,i ), we feed the two inputs into the LLM and obtain the hidden representations at a selected layer l. The representation difference is computed as (l)
(l)
(l)
− ∆hk,i = hθ (x+ k,i ) − hθ (xk,i )
(4)
The difference vector in Eq. (4) captures the local representation shift from a low-readability example to a high-readability example under the same metric. We then mean-center these difference vectors to remove the common offset shared by all pairs (l)
(l)
xk,i = ∆hk,i −
Mk 1 X (l) ∆hk,j Mk j=1
(5) (l)
The centered vectors are stacked to form the dataset Xk = (l) (l) [xk,1 , · · · , xk,Mk ]⊤ . In the following optimization problem, we omit the superscript l for notational simplicity, since the same extraction procedure can be independently applied to each selected injection layer.
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
6
D. Steering Vector Extraction To improve the readability of code generated by LLMs, we need to extract steering vectors associated with code readability metrics, including comment density, naming conventions, and cyclomatic complexity. As shown in Eq. 2, these steering vectors are injected into the model’s internal activation layers, enabling targeted control over the code readability metrics by modulating the hidden state representations of the model. A natural baseline is to extract the steering vectors independently for each code readability metric through PCA, and inject them into the model simultaneously. However, the resulting vectors exhibit substantial and irregular mutual interference [27]. The reason is that independent PCA only maximizes the projected variance within each individual dataset and does not consider the geometric relationship among different readability directions. Consequently, two steering vectors extracted from different readability metrics may still point to highly overlapping subspaces. When such correlated vectors are injected together, the actual representation perturbation is no longer metric-specific: one vector may unintentionally affect another readability metric or even amplify the disturbance to code correctness. This phenomenon is especially problematic in multitask readability control, where the purpose is not merely to improve one metric, but to obtain controllable and complementary improvements across several metrics. To address this challenge, we need to solve an optimization problem that jointly estimates the first principal component for each dataset under the constraint that all principal components are pairwise orthogonal. The formal definition of this optimization problem is as follows. Given K groups of centered datasets {X1 , X2 · · · , XK }, where each dataset Xk ∈ RMk ×N corresponds to a code readability metric, Mk is the number of samples in the kth dataset, and N is the feature dimension of each sample, the objective is to find K unit vectors {d1 , d2 , · · · , dK } that maximize the weighted sum of the projected variance
compromise between metric-specific expressiveness and crossmetric disentanglement. This is different from applying GramSchmidt orthogonalization after independent PCA, because post-processing may rotate a direction away from the highvariance subspace of its own dataset, whereas MOC-JPCA optimizes the variance objective and the orthogonality constraint simultaneously. To solve the optimization problem formulated in (6) above, we develop the joint principal component analysis algorithm with multidimensional orthogonal constraints (MOC-JPCA) in Algorithm 1. Specifically, given K groups of centered datasets, we first calculate the covariance matrix Sk for each dataset (lines 1–3). Then, we initialize the unit vector matrix D in an orthogonal space (line 4). After that, we iteratively update D on the Stiefel manifold. In each iteration, we first compute the covariance product matrix B (line 6), and then obtain the Riemannian gradient G by projecting the Euclidean gradient onto the tangent space of the Stiefel manifold (line 7). To ensure a sufficient increase of the objective function, the update step size η is determined by the Armijo backtracking line search (lines 8–11). Finally, the updated matrix is retracted back onto the Stiefel manifold via the QR retraction (line 12). The loop is executed for a maximum of T iterations. More concretely, the matrix B = [S1 d1 , · · · , SK dK ] collects the Euclidean ascent directions for all readability metrics. Since a direct Euclidean update may violate the orthogonality constraint, the algorithm projects this direction onto the tangent space of the Stiefel manifold. The projection term D sym(D ⊤ B) removes the component that would move D outside the feasible manifold. After taking a step along the projected direction, the QR-based mapping qf(·) retracts the updated matrix back to a feasible orthogonal matrix. In this way, every iterate satisfies D ⊤ D = I, and the extracted directions can be directly used as steering directions. The following theorem establishes the convergence of the MOC-JPCA algorithm, and the proof details are provided in Appendix C-A.
(6)
Theorem 1. Let D (0) be randomly generated from an orthogonal space, and let {D (t) } be the iterative sequence produced by the MOC-JPCA algorithm, where D (t) ∈ RN ×K denotes the solution at the t-th iteration. Then, the sequence {D (t) } converges to a first-order stationary point of Problem (3).
where Sk = Mk1−1 Xk⊤ Xk is the covariance matrix of the k-th dataset, D = [d1 , d2 , · · · , dK ] is the matrix of unit vectors, and I is the identity matrix, enforcing orthogonality among the dk vectors. M = D ∈ RN×K | D ⊤ D = I is a Stiefel manifold. The constraint D ⊤ D = I has two effects. First, it normalizes each steering direction, which makes the coefficient ck in Eq. (1) comparable across different readability metrics. Second, it enforces d⊤ i dj = 0 for i ̸= j, which explicitly reduces the overlap among different readability directions. From the perspective of multitask control, the objective term d⊤ k Sk dk encourages dk to preserve the most informative representation variation associated with the k-th readability metric. The orthogonality constraint, on the other hand, prevents the solution from assigning similar semantic effects to multiple steering directions. Therefore, MOC-JPCA can be viewed as a
Theorem 1 provides the theoretical basis for using MOCJPCA as the extraction module of MRepE. Since the objective in (6) is generally non-convex on the Stiefel manifold, the theorem does not claim global optimality. Instead, it ensures that the iterative process converges to a point at which no first-order feasible ascent direction exists. This guarantee is sufficient for the present framework because the extracted steering vectors are used as representation-level control directions rather than as exact global solutions of a convex problem. After Algorithm 1 returns the orthogonal direction matrix D, each column dk is assigned to the corresponding readability metric and scaled by its coefficient ck during inference. In the full MRepE pipeline, the extraction step and the injection step are decoupled: the directions are computed once from the contrastive readability datasets, whereas the coefficients can be adjusted at inference time according to the desired trade-off
max
d1 ,d2 ,··· ,dK
K X
d⊤ k Sk dk
k=1 ⊤
s.t. D D = I
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
Algorithm 1 The joint principal component analysis with multidimensional orthogonal constraints (MOC-JPCA) Input: The centered datasets {X1 , X2 , · · · , XK }; parameters η̄, β, c Output: The set of unit vectors {d1 , d2 , · · · , dK } 1: for k = 1, 2, · · · , K do 2: Sk ← Mk1−1 Xk⊤ Xk ; 3: end for 4: Initialize D = [d1 , d2 , · · · , dK ] such that D ⊤ D = I; 5: for t = 1, 2, · · · , T do 6: B ← [S1 d1 , · · · , SK dK ]; 7: G ← 2 B − D sym(D ⊤ B) ; 8: η ← η̄; 9: while f (qf(D + ηG)) < f (D) + cη∥G∥2F do 10: η ← βη; 11: end while 12: D ← qf(D + ηG); 13: end for
between readability and correctness. This decoupling makes MRepE flexible in practical scenarios. For example, when the user requires more explanatory comments, the coefficient of the comment-density direction can be increased; when correctness is more important, all coefficients can be kept small to reduce the intervention strength. IV. T HEORETICAL R ESULTS As shown in empirical study, the improvement of code readability might come at the cost of reduced code correctness. By controlling the steering strength of the vectors across different metrics, the model can achieve balanced performance in the code readability and correctness. Specifically, we conduct this analysis in a theoretical way. Theorem 2 shows, if a query corresponds to any metric included among the injected code readability metrics, the model’s capability on that metric is guaranteed with a lower bound. Theorem 3 establishes that if the query corresponds to the code correctness, the model’s capability regarding the code correctness is constrained by an upper bound. In other words, the negative impact on the code correctness due to the improved readability is to be bounded. The detailed proof is presented in Appendixes C-B and C-C. Theorem 2. Let Pθ,V (·|q) denote a model that is injected with a set of code readability steering vectors {V1 , · · · , VK }, where the query q is uniquely associated with one of the steering vectors. The model’s performance on the code readability metric corresponding to query q is evaluated by the probaP bility of generating a positive answer P a+ θ,V (a+ |q). Let √P 2 − λ c k k k δV (q) = A 1 − e denote the change in the final hidden layer representation induced by the injected steering vectors. Then, the model’s performance on the code readability metric corresponding to query q is lower-bounded by X −1 (P0 ) + ∆1 · δ V (q) + ∆2 a+ Pθ,V (a+ | q) ≥ σ σ Here, σ(x) = 1+e1−x is the logistic function and P0 is the expected probability of the model generating a positive
7
answer to query q without any steering vector intervention. A and λk are model-dependent and characterize the relationship between ck and the norm of the corresponding final hidden layer representation. ∆1 > 0 indicates the performance improvement through the code readability steering vector, and ∆2 indicates the potential negative impact caused by steering vectors irrelevant to the query q. Since this work does not consider impairments to code readability, all steering strength coefficients are constrained to satisfy ck > 0(1 ≤ k ≤ K). As evident from the mathematical expression, this lower bound is influenced qP by all steering vector coefficients ck (1 ≤ 2 k ≤ K). At 1≤k≤K λk ck → 0, the lower bound reduces to P0 + σ(∆2 ), approximating the expected probability of the model generatingqa positive answer without any steering P 2 vector injection. At 1≤k≤K λk ck → ∞, the lower bound approaches σ(σ −1 (P0 ) + ∆1 · A + ∆2 ). Theorem 3. Let Pθ,V (· | q) denote a model injected with code readability steering vectors V1 , ..., VK , where the query q is associated with code correctness. Assume that, for the representations of positive and negative answers constituting a probability mass of 1 − ϵ, the change in the final hidden layer representation induced by the injected steering vectors follows a random distribution with variance σ 2 > 0. Then, with probability at least 1 − T2 , the probability of the model generating a positive answer to query q is upper-bounded by Pθ,V (a+ | q) ≤
P0 2
P0 + (1 − P0 )α(1 − ϵ)(1 + 21 β 2 σ 2 δ V (q) )
Here, T denotes the number of tokens constituting the probability mass 1 − ϵ, and α and β are parameters dependent on query q. This theorem characterizes the effect of injecting the code readability steering vectors on tasks of code correctness. The parameter α ∈ [0, 1] quantifies the looseness of the bound caused by asymmetry in the probability distribution. qP 2 Specifically, at 1≤k≤K λk ck → 0, the true probability of the model generating a positive answer to a code correctnessrelated query is P0 , while the upper bound given by the P0 theorem becomes P0 +(1−P . Consequently, at α = 1 0 )α(1−ϵ) qP 2 and ϵ = 0, the upper bound at 1≤k≤K λk ck = 0 exactly coincides with the true probability. A small value of α indicates that the bound overestimates the true probability. The coefficients λk capture how the code readability steering vectors influence the code correctness. The product βσ characterizes the probability decay rate, where σ is the standard deviation of random noises imposed on the correctness due to the injection of the code readability steering vectors, and β is the minimum of the two weighted sums of positive and negative random variables parameterized with σ ′ = 1. V. E XPERIMENTAL R ESULTS We evaluate the code readability and correctness by examining how they vary with increasing coefficients of the code readability steering vectors. Our aim is to verify the aforementioned theoretical bounds, specifically to (a) demonstrate
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
8
the performance improvement of the code readability via the injected steering vectors; and (b) discuss the impact of the injected steering vectors on code correctness. A. Experimental Setup For ease of notation, we let comment density, naming conventions, cyclomatic complexity, and code correctness be CD, NC, CC, and TF, respectively. The steering strength associated with a metric is denoted as c(·), and the average probability of generating positive answers related to a metric is denoted as E(·). a) Datasets: We use the mostly basic Python problems (MBPP) [28] as a benchmark dataset to evaluate the quality of LLM-generated codes. The benchmark consists of around 1, 000 crowd-sourced Python programming problems. We extend 300 samples from the MBPP dataset into two new datasets designed to evaluate code readability and code correctness of code generated by LLMs. The extension format for assessing metrics such as comment density, naming conventions, and cyclomatic complexity is illustrated in the following example (referred to as MBPP-CR dataset, Tbl. I). The extension format for evaluating correctness is as folTABLE I A N EXAMPLE FROM THE MBPP-CR DATASET Question
CD LOW
Write a function to reverse words in a given string.
def reverse words(s): return ’ ’.join(reversed(s.split()))
Question Write a function to reverse words in a given string.
CD HIGH def reverse words(s): #Split string into words, reverse their order, and join back with spaces return ’ ’.join(reversed(s.split()))
NC LOW
NC HIGH
def r w(s): return ’ ’.join(reversed(s.split()))
def reverse words(s): return ’ ’.join(reversed(s.split()))
Question
CC LOW
Write a function to reverse words in a given string.
def reverse words(s): return ’ ’.join(reversed(s.split()))
CC HIGH def reverse words(s): words = [] for word in s.split(): words.insert(0, word) return ’ ’.join(words)
B. Steering Vector Extraction
lows (referred to as MBPP-CC dataset, Tbl. II). TABLE II A N EXAMPLE FROM THE MBPP-CC DATASET Question
A
Write a function to reverse words in a given string.
def reverse words(s: str) → str: return s[::-1]
B def reverse words(s: str) → str: words = s.split() reversed word iterator = reversed(words) return ’ ’.join(reversed word iterator)
Template of qnc : “Do you agree that the following code has good/bad naming conventions? {code}” • Template of qcc : “Is the cyclomatic complexity of the following Python code low/high? {code}” • Template of qtf : “{question} For the programming problem above, which one of the following four Python code options (A, B, C, D) is correct? A) {codeA} B) {codeB} C) {codeC} D) {codeD}” c) Foundation Models: We use three foundation language models: (a) Deepseek R1 14b [29] is an efficient reasoning model obtained through knowledge distillation from DeepSeek-R1. (b) Qwen2.5coder 14b Instruct [30] is a 14.7-billion-parameter causal language model instructionaligned for programming tasks by Tongyi Qianwen. (c) Codellama 13b Instruct [31] is a 13-billion-parameter code generation model built upon the Llama2 architecture. d) Experimental Equipments: All the experiments are conducted on two computing servers (S1 and S2). S1 runs on Ubuntu 22.04.3 LTS and is equipped with a dual Intel Xeon Gold 6530 CPU, 512GB RAM, and 4 NVIDIA GeForce RTX 4090 GPUs. S2 operates on Ubuntu 22.04.4 LTS and featured with a dual Intel(R) Xeon(R) Gold 6248R CPU, 1.0TB RAM, and 4 NVIDIA Tesla V100 SXM3 GPUs. •
C
D
Answer
......
......
B
b) Query Templates: For the evaluation queries, we construct four types of query templates corresponding to comment density, naming conventions, cyclomatic complexity, and code correctness, denoted as qcd , qnc , qcc , and qtf , respectively. For qcd , qnc , and qcc , the positive labels correspond to high comment density, good naming conventions, and low cyclomatic complexity, respectively. For qtf , the positive label corresponds to the correct code option. The query templates are as follows • Template of qcd : “Is the comment density of the following Python code high/low? {code}”
We first evaluate the effectiveness of the steering vector extraction based on MOC-JPCA on the MBPP-CR dataset. Fig. 4 shows the probability that, after injecting the corresponding steering vector at varying strengths (c(·)) for a given code readability metric (either CD, NC or CC), we report the ratio of the average negative log-likelihood for generating P N LLreadable 1 readable versus unreadable code ( K N LLunreadable ), and the probability of the model providing a positive answer (E) to queries related to that metric. Details on the specific layers into which each steering vector is injected and the significance analysis of the average negative log-likelihood ratio are provided in Appendix D-A. The model performance improves as the steering strength increases, which validates the effectiveness of the steering vectors extracted by MOC-JPCA . C. Code Readability Measurement To demonstrate the performance on the code readabilityrelated queries, we compute the expected probability that the model produces a positive answer. For instance, when a pair of the readability metrics, such as comment density and naming conventions, are injected, the model is queried with respect to the comment density and the expected probability that the answer aligns with high comment density is estimated.
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
0.852
0.55
0
1
2
3
C CD
4
5
0.8
0.40
0.7 0.6
0.32
0.42 1.928
0.5 0.4
0.24
0.3
0.36
0.848
0.7305
0.48
E
0.2
0.16
1.920 0.30
0.844
0.50 0.840
0.1
2
3
C NC
4
5
0
1
2
3
C CC
4
5
0
c 1 (CD)
2
2 3
1 4
5
0
c 2 (NC)
4
0
3 1
c 1(CD)
2
2 3
1 4
5
0
0
0.5 0.4 0.3 0.2
Readable Unreadable
0.90 0.824
0.60
Readable Unreadable
1.648 0.75
E
0.75
0.9958
0.80
0.45 1.640
0.30
0.818
0.55
1
2
3
C CD
4
2 3
4
5
5
0
0
1
2
3
C NC
4
5
0.50
0
1
2
3
C CC
4
0
c 1(NC)
2
2 3
1 4
5
0
1.0
0.8 0.7 0.6
0.80 0.75 0.70 0.65 0.60 0.55 0.50 0.45 0.40 0.35 5
5
3 1
c 1(NC)
2
2 3
1 4
5
c 2(CC)
0
Readable Unreadable
0.80 0.774
0.60
E
0.772
Readable Unreadable
0.65 1.472
0
0.78
E
0.55 1.470
0.8285
4
0.84
0.72
c 1(CD)
2
2 3
1 4
5
0.6 0.5 0.4 5 4
4
3 1
0.9 0.8 0.7
0
c 2(NC)
0
3 1
c 1(CD)
2
2 3
1 4
5
0
0
3 1
c 2(NC) c 1(CD)
2
2 3
1 4
5
c 2(CC)
0
E
0.66
0.48
0.766 0
1
2
3
C CD
4
5
0
1
2
3
C NC
4
5
0
1
2
3
C CC
4
5
0.42
5 4 0
(c)
0.75 0.70 0.65 0.60 0.55 0.50 0.45 0.40 0.35 5
c1(CD)
2
2 3
1 4
5
c2(CC)
0
0
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2
TABLE III T- TEST R ESULTS AND S TEERING V ECTOR I NJECTION L AYERS OF D IFFERENT M ODELS
c 1(NC)
2
2 3
1 4
5
0
0.95 0.90 0.85 0.80 0.75 0.70 0.65 0.60 0.55 0.50 5
5
Qwen2.5-Coder
CodeLlama
−35 to −41 −22 to −34 −1 to −21 −15 to −43 −13 to −19 −20 to −35 −20 to −24 −30 to −31 −25 to −29
Figs. 5a, 5b, and 5c present the experimental results for the Deepseek R1 14b, Qwen2.5coder 14b Instruct, and Codellama 13b Instruct models, respectively. The curves show how the expected probability of positive answers varies as the coefficients of the steering vectors increase. The fitted surfaces correspond to the lower bound predicted by Theorem 2. Tbl. III presents the injection layers of steering vectors corresponding to different readability metrics in each model, along with the statistical significance tests for model behavior changes. The results indicate that by injecting the code readability steering vectors, all the tested models show a significant increase in the probability of providing positive answers to the readability-related queries, fully validating the positive regulatory effect of the readability steering vectors. Specifically, as
3 1
2
2 3
1 4
5
c 2(CC)
0
c 1(CD)
2
2 3
1 4
5
0
c 2(NC)
0
c 1(CD)
2
2 3
1 4
5
0
c 2(NC)
0
0.4 0.2
0.7 0.6
c 1(CD)
3
1 4
5
0
2
3
1 4
5
c 2(CC)
0
c 2(CC)
1.0 0.8 0.6 0.4 0.2 0.0
5
5
2
c 1(CD)
2
0.5 5
4
4 3 2
3 1
1.0 0.9 0.8
0.0
1
5 4
3 1
1.0 0.8 0.6
0
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2
4
3 1
E(CC)
DeepSeek-R1
CD NC CC CD NC CC CD NC CC
T-test Metrics T-statistic p -5.73 2.85 × 10−8 -5.58 6.13 × 10−8 -5.71 3.16 × 10−8 -4.31 2.33 × 10−5 -3.86 1.46 × 10−4 -3.82 1.66 × 10−4 -4.02 7.60 × 10−5 -3.84 1.52 × 10−4 -6.54 3.23 × 10−10
0
c 2(CC) c 1(NC)
4 0
Metric Injection Layers
4
3 1
(b) Results of Qwen2.5coder 14b Instruct model
Fig. 4. The readability performance for the models with the single-metric varying with steering strength.
Model
0.6 0.5 0.4 5
4
3 1
1.0 0.9 0.8 0.7
0
3 1
c 1(NC)
2
2 3
1 4
5
0
E(CD)
0.54
0.40 1.466
E(CC)
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
0.60
0.45
c 2(CC)
E(CC)
0.768
0.48
E(NC)
0.56
0.8275
E(NC)
0.770
1.468
0.8280
E(NC)
0.50
E(CC)
0.64
E(CD)
NLLreadable 1 ∑NLLunreadable K
1.474 0.88 0.776
0.72
5 4 0
c 2(CC)
0.5
E
0.4 0.2 0.0
3 1
0.4
0.8290
1.0 0.8 0.6
5
(b) Readable Unreadable
0
5
c 2(CC)
0.9
0.8295
5
c 2(CC)
(a) Results of Deepseek R1 14b model
0.15
1.636
1
1 4
0.00
0.15 0
c 1(CD)
2
2 3
4
3 1
0.30
0.60
0.9954
0.3 0.2
5 0 0.60
0.65 0.45 0.820
0.9956
0.5 0.4
0.1 0.0 4
0.75
E
0.70 1.644
0.822
0.90
0.8 0.7 0.6
E(CD)
NLLreadable 1 ∑NLLunreadable K
Readable Unreadable
E
0.9960
0.826
E
1.05
2
0.9
E(CC)
(a)
3 1
c 2(NC) c 1(CD)
0.6
0.9962
0.7 0.6 0.5 5
4
3 1
1.0 0.9 0.8
5
4
0.00
0.18 1
5
0.08
0.24 1.912
0
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
E(CD)
0.60
Readable Unreadable
1.936
0.48
E(CC)
0.65
0.7320
E
1.944
E(CD)
0.7335
0.54
E(NC)
0.70
0.60
Readable Unreadable
E(NC)
E
0.7350
0.860
0.75 0.856
E(NC)
0.80
E(CD)
Readable Unreadable
0.7365
E
NLLreadable 1 ∑NLLunreadable K
0.7380
9
4 0
3 1
c 1(NC)
2
2 3
1 4
5
c 2(CC)
0
(c) Results of Codellama 13b Instruct model Fig. 5. The surface plots illustrate how the code readability performance of the models varies with the coefficients of the injected steering vectors. The green surfaces represent the direct measurement results, and the red surfaces depict the theoretical bound given by Theorem 2.
the coefficients of the steering vectors increase, the models’ recognition of the code readability metrics demonstrate a stable upward trend. The experimental data align with the theoretical lower bound in Theorem 2, indicating that the injection of steering vectors can effectively and consistently improve the code readability. Furthermore, differences in the sensitivity of various models to the steering vectors are observed, reflecting the influence of model architecture on the effectiveness of code readability regulation, thereby providing an empirical basis for subsequent optimization. D. Code Correctness Measurement To show the performance on the code correctness-related queries, a procedure analogous to that for the code readability is adopted. For example, when the injected code readability
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
10
2
3
4
5
0
1
2
3
4
0.0
0.4 0.2 0
c 2(NC)
1
2
c 1(CD)
3
4
5
0
1
2
3
4
5
0.0
0.4 0.2 0
1
2
c 2(CC) c 1(NC)
3
4
5
0
1
2
3
4
5
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
0.0
0.0 0.0 0.5 1.0
c 2(CC)
1.5
2.0
2.5
c 1 (CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
3.0
2.5
2.0
1.5
1.0
4.0
3.5
4.5
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0.0
5.0
0.5
1.0
1.5
2.0
2.5
c 2 (NC) c1(CD)
3.0
3.5
(a) Results of Deepseek R1 14b model
1
2
c 1(CD)
3
4
5
0
1
2
3
4
5
0.0
0.4 0.2 0
c 2 (NC)
1
2
c1(CD)
3
4
5
0
1
2
3
4
5
0.8
E(TF)
0.2 0
E(TF)
0.4
1.0
1.0 0.8 0.6
0.0
0.6 0.4 0.2 0
c2(CC)
1
2
c1(NC)
3
4
5
0
1
2
3
4
5
4.5
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
E(TF)
1.0 0.8 0.6
4.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
c1(CD)
0.0
3.5
4.0
4.5
5.0
0.5
0.0
1.0
1.5
2.0
2.5
3.0
3.5
4.0
5.0
0.0
0.5
1.0
1.5
2.0
2. 5
3.0
3.5
4.0
5.0
4.5
c2(NC)
c3(CC)
1
c 1(CD)
5
0.6
c 3 (CC)
0.4 0.2 0
0.6
E(TF)
0.8 0.6
E(TF)
1.0 0.8
E(TF)
1.0 0.8
c3(CC)
1.0 1.0
5.0
4.5
c2(NC)
(a) Results of Deepseek R1 14b model
c2(CC)
0.6 0.4 0.2 0
1
2
c1(CD)
3
4
5
0
1
2
3
4
5
0.0
c2(NC)
0.6 0.4 0.2 0
1
2
c1(CD)
3
4
5
0
1
2
3
4
5
0.0
c2(CC)
0.6 0.4 0.2 0
1
2
c1(NC)
3
4
5
0
1
2
3
4
5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
c1(CD)
3.5
4.0
4.5
5.0
0.0
0.5
2.0
1.5
1.0
2.5
3.0
3.5
4.0
4.5
5.0
0.0
c2(NC)
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.5
0.0
1.0
1.5
2.0
2.5
3.0
3.5
4.0
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0.0
5.0
4.5
c2(NC)
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
c3(CC)
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
c3(CC)
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
c3(CC)
1.0 0.8
E(TF)
1.0 0.8
E(TF)
1.0 0.8
E(TF)
(b) Results of Qwen2.5coder 14b Instruct model
5.0
4.5
c2(NC)
0.0
(b) Results of Qwen2.5coder 14b Instruct model
c2(CC)
0.0
0.5
1.0
1.5
2.0
2.5
c1(CD)
E. Trade-off between the Readability and Correctness We analyze the impact on the models’ performance after the steering vector injection in terms of the three readability metrics. Fig. 7 shows the probability of
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
0.0
c2(NC)
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
c3(CC)
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
5.0
4.5
0.0
c2(NC)
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
c2(NC)
(c) Results of Codellama 13b Instruct model Fig. 7. The 3D stacked heatmaps illustrate the probability (blue for 0, red for 1) of models, injected with steering vectors corresponding to all three code readability metrics, providing positive answers to various queries under different coefficient values. The queries correspond to comment density, naming conventions, and cyclomatic complexity, respectively, from the first to the third column.
0.5638
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
(a)
2.5
3.0
3.5
4.5
0.5365
1.5
1.0
1.0
1.0
0.5
0.5
0.5278
5.0
c2(NC)
1.5
3.0 2.5 2.0
0.0 0.0
0.5189
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
(b)
2.5
3.0
3.5
4.0
4.5
0.5236
c2(NC)
0.0 0.0
5.0
0.5107
c3(CC)
0.5368
0.5494
0.5 0.0 4.0
3.0 2.5 2.0
c3(CC)
1.5
0.5458
0.5059
4.5 4.0 3.5
3.0
1.0
0.5623
4.5 4.0
2.5
0.5
5.0
5.0
0.5548
3.5
2.0
0.0
0.5132
0.5752
5.0 4.5 4.0 3.5
c3(CC)
metrics are comment density and naming conventions, the model is queried with respect to code correctness and the expected probability that the answer aligns with code correctness is computed. Figs. 6a, 6b, and 6c present the experimental results for the Deepseek R1 14b, Qwen2.5coder 14b Instruct, and Codellama 13b Instruct models, respectively. The figures show how the expected probability of positive answers regarding the code correctness changes as the coefficients of the code readability steering vectors increase. The fitted surfaces correspond to the upper bound predicted by Theorem 3. Details regarding how the parameters are determined can be found in Tables 1, 2, and 3 of the Appendix. Similarly, the results demonstrate that,as the coefficients of the code readability steering vectors increase, the probability of all the tested models generating positive answers to the code correctness-related queries shows a declining trend, which aligns with the upper bound prescribed by Theorem 3. Specifically, when steering vectors corresponding to any two code readability metrics are injected, the correctness performance gradually decreases as the coefficients grow, but the extent of this decline is constrained by the theoretical upper bound. This finding confirms that through controlled steering vector injection, it is possible to improve the code readability while limiting the loss in the code correctness within the theoretical bound, providing a crucial basis for the parameter tuning in practical applications.
3.0
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
c3(CC)
5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
c3(CC)
(c) Results of Codellama 13b Instruct model Fig. 6. The surface plots illustrate how the code correctness performance of the models varies with the coefficients of the injected steering vectors. The green surfaces represent the direct measurement results, and the red surfaces depict the theoretical bound given by Theorem 3.
0.5
1.0
1.5
2.0
2.5
c1(CD)
3.0
3.5
4.0
4.5
5.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
0.4986
0.4913
0.4840
5.0
c2(NC)
(c)
Fig. 8. The 3D stacked heatmaps illustrate the probability (blue for 0, red for 1) of models, injected with steering vectors corresponding to all three code readability metrics, providing positive answers to queries related to code correctness under varying coefficients.
Deepseek R1 14b, Qwen2.5coder 14b Instruct, and Codellama 13b Instruct, which provides positive answers to varied code readability-related queries after injecting steering vectors in terms of the code readability metrics across different coefficient values. We also provide the corresponding probabilities of the three models that generate positive answers to code correctness queries in Fig. 8. It can be observed that for each code readability metric, its corresponding steering vector plays a dominant role in improving the model’s performance . Furthermore, as the coefficients of the code readability steering vectors increase, the model’s performance on code correctness exhibits a declining trend. We can systematically optimize the trade-off between the
0.4767
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
code readability and correctness by collaboratively leveraging Theorems 2 and 3. Theorem 2 provides a clear lower-bound guarantee for improving the readability, ensuring that after injecting steering vectors, the model’s probability of positive answers to the readability-related queries increases at least at the theoretically predicted rate, thereby reliably improving the readability metrics. Theorem 3 constrains potential negative impacts by setting an upper bound for the decline in the code correctness, preventing excessive degradation of correctness. Specifically, by adjusting the coefficients of steering vectors corresponding to different readability metrics, we can achieve a fine-grained balance between the degree of the readability improvement and the tolerable level of the correctness loss, tailored to specific application scenarios. For instance, the experimental results show that for the Deepseek R1 14b model, with the coefficient combination c1 (CD) = 1.0, c2 (NC) = 4.0, and c3 (CC) = 5.0, the readability is significantly improved while the correctness decreases by only 0.79% — a trade-off acceptable in most practical applications. Similarly, the Qwen2.5coder 14b Instruct model achieves readability optimization with c1 (CD) = 3.5, c2 (NC) = 5.0, and c3 (CC) = 3.0, keeping the correctness loss within 0.95%. The Codellama 13b Instruct model exhibits a mere 0.28% the correctness loss with c1 (CD) = 5.0, c2 (NC) = 4.5, and c3 (CC) = 5.0. These results validate the effectiveness of the steering strategy based on the theoretical bounds, enabling MRepE to achieve a controllable balance between the readability and correctness tailored to different model characteristics and application requirements. VI. C ONCLUSION AND F UTURE W ORK This work addresses the control problem of multiple readability metrics in LLM-generated codes by defining a joint principal component analysis problem under multidimensional constraints, proposing a corresponding solution algorithm (MOC-JPCA), and developing a multitask steering representation engineering method (MRepE) to coordinately control multiple readability metrics. By using MOC-JPCA to jointly extract steering vectors for different code readability metrics and applying them in the MRepE, we achieve coordinated control over these metrics and provide theoretical analysis of the trade-off between the code readability and correctness. The experimental results further show that the extracted steering vectors can consistently improve the model’s responses to readability-related queries, while the degradation in code correctness can be limited by properly selecting the steering coefficients. This indicates that MRepE provides not only an effective method for multi-metric readability control, but also a controllable mechanism for balancing human-oriented code quality and functional reliability. We could refine the code generation according to users’ demands. Although we investigate the MRepE framework over three readability metrics, MRepE can be generalized to more code quality control tasks under similar theoretical bounds on its performance of the code readability and correctness. In future work, we plan to extend MRepE to more fine-grained code quality attributes, such as maintainability, modularity, and
11
documentation style, and evaluate its effectiveness in largerscale software engineering scenarios. Future work will focus on practical applications of the MRepE and investigate its synergy with different aspects of LLM-generated code quality. R EFERENCES [1] D. Spinellis, Code quality: the open source perspective. Adobe Press, 2006. [2] S. Fakhoury, D. Roy, A. Hassan, and V. Arnaoudova, “Improving source code readability: Theory and practice,” in 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 2019, pp. 2–12. [3] R. P. Buse and W. R. Weimer, “Learning a metric for code readability,” IEEE Transactions on software engineering, vol. 36, no. 4, pp. 546–558, 2009. [4] D. Posnett, A. Hindle, and P. Devanbu, “A simpler model of software readability,” in Proceedings of the 8th working conference on mining software repositories, 2011, pp. 73–82. [5] S. Scalabrino, G. Bavota, C. Vendome, M. Linares-Vásquez, D. Poshyvanyk, and R. Oliveto, “Automatically assessing code understandability: How far are we?” in 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2017, pp. 417–427. [6] C. Krzysztof and U. W. Eisenecker, Generative Programming: Methods, Tools and Applications. Addison-Wesley, 2000. [7] Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin, “A neural probabilistic language model,” Journal of machine learning research, vol. 3, no. Feb, pp. 1137–1155, 2003. [8] U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “code2vec: Learning distributed representations of code,” Proceedings of the ACM on Programming Languages, vol. 3, no. POPL, pp. 1–29, 2019. [9] J. Wang and Y. Chen, “A review on code generation with llms: Application and evaluation,” in 2023 IEEE International Conference on Medical Artificial Intelligence (MedAI). IEEE, 2023, pp. 284–289. [10] J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” arXiv preprint arXiv:2406.00515, 2024. [11] W. Takerngsaksiri, C. Tantithamthavorn, M. Fu, J. Pasuksmit, K. Chen, and M. Wu, “Code readability in the age of large language models: An industrial case study from atlassian,” in 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2025, pp. 732–742. [12] D. Zhang, N. R. A. Arias, Y. He, and S. Kovalchuk, “Style2code: A style-controllable code generation framework with dual-modal contrastive representation learning,” arXiv preprint arXiv:2505.19442, 2025. [13] A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski et al., “Representation engineering: a top-down approach to ai transparency,” ArXiv Preprint arXiv:2310.01405, 2023. [14] Y. Wolf, N. Wies, D. Shteyman, B. Rothberg, Y. Levine, and A. Shashua, “Tradeoffs between alignment and helpfulness in language models with steering methods,” in ICLR 2025 Workshop on Foundation Models in the Wild, 2025. [15] Q. Luo, H. Yu, and X. Li, “Badam: A memory efficient full parameter optimization method for large language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 24 926–24 958, 2024. [16] Z. Qin, D. Chen, B. Qian, B. Ding, Y. Li, and S. Deng, “Federated fullparameter tuning of billion-sized language models with communication cost under 18 kilobytes,” in International Conference on Machine Learning. PMLR, 2024, pp. 41 473–41 497. [17] K. Lv, Y. Yang, T. Liu, Q. Guo, and X. Qiu, “Full parameter fine-tuning for large language models with limited resources,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 8187–8198. [18] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” Advances in neural information processing systems, vol. 36, pp. 10 088–10 115, 2023. [19] Q. Zhang, M. Chen, A. Bukharin, P. He, Y. Cheng, W. Chen, and T. Zhao, “Adaptive budget allocation for parameter-efficient fine-tuning,” in International Conference on Learning Representations. Openreview, 2023. [20] R. Zhang, J. Han, C. Liu, A. Zhou, P. Lu, Y. Qiao, H. Li, and P. Gao, “Llama-adapter: Efficient fine-tuning of large language models with zero-initialized attention,” in The Twelfth International Conference on Learning Representations, 2024.
IEEE JOURNAL SUBMISSION, VOL. XX, NO. XX, MONTH 2026
[21] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022. [22] H. Yuan, Z. Yuan, C. Tan, W. Wang, S. Huang, and F. Huang, “Rrhf: Rank responses to align language models with human feedback,” Advances in Neural Information Processing Systems, vol. 36, pp. 10 935– 10 950, 2023. [23] J. Hong, N. Lee, and J. Thorne, “Orpo: Monolithic preference optimization without reference model,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 11 170– 11 189. [24] E. Stiefel, “Richtungsfelder und fernparallelismus in n-dimensionalen mannigfaltigkeiten,” Ph.D. dissertation, ETH Zurich, 1935. [25] T.-D. Bui, T. T. Vu, T.-T. Nguyen, S. Nguyen, and H. D. Vo, “Correctness assessment of code generated by large language models using internal representations,” arXiv preprint arXiv:2501.12934, 2025. [26] U. A. Mannan, I. Ahmed, and A. Sarma, “Towards understanding code readability and its impact on design quality,” in Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering, 2018, pp. 18–21. [27] T. van der Weij, M. Poesio, and N. Schoots, “Extending activation steering to broad skills and multiple behaviours,” arXiv preprint arXiv:2403.05767, 2024. [28] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al., “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732, 2021. [29] D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” arXiv preprint arXiv:2501.12948, 2025. [30] B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu et al., “Qwen2. 5-coder technical report,” arXiv preprint arXiv:2409.12186, 2024. [31] B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remez et al., “Code llama: Open foundation models for code,” arXiv preprint arXiv:2308.12950, 2023.
12