arXiv CSopen access

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

arXiv:2605.13280v1 [cs.SE] 13 May 2026

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code Hengzhi Ye

Fengyuan Ran

[email protected] Peking university Beijing, Beijing, China

[email protected] Wuhan University Wuhan, Hubei, China

Weiwei Xu

Minghui Zhou

[email protected] Peking university Beijing, Beijing, China

[email protected] Peking university Beijing, Beijing, China

Abstract As Large Language Models (LLMs) are transforming software development, the functional quality of generated code has become a central focus, leaving readability, one of critical non-functional attributes, understudied. Given that LLM-generated code still needs human review before adoption, it is important to understand its readability especially compared with human-written code and the role of prompt design in shaping it. We therefore set out to conduct a systematic investigation into the code readability of LLM-generated code. To systematically quantify code readability, We establish a comprehensive readability model that synthesizes textual, structural, program, and visual features of code. Based on the model, we evaluate the readability of code generated by the mainstream LLMs under 5,869 scenarios extracted from large code base including World of Code (WoC) and LeetCode. We find that current LLMs produce code with overall readability comparable to human-written code, but displaying distinct readability issue patterns. We further examine how different prompt dimensions affect the readability of LLM-generated code, and find that function signatures, constraints and style descriptions emerge as the most influential factors, while the overall impact of prompt design remains limited. Our findings indicate that, on one hand, LLM-generated code is at least comparable to human-written code in readability, validating its potential for systematic integration into software workflows from a nonfunctional perspective; on the other hand, distinct readability issue patterns and limited effectiveness of prompt engineering reveal a latent technical debt, highlighting the need for future research to improve the readability of LLM-generated code and thus ensure long-term maintainability.

CCS Concepts • Software and its engineering → Software creation and management; • Computing methodologies → Artificial intelligence.

Keywords Code readability, code generation, prompt engineering, LLM

1

Introduction

Software development has evolved far beyond the simple act of writing code. Reading and understanding code constitutes the fundamental first step in any maintenance and incremental task [73]. Clear and readable code significantly facilitates comprehension,

reduces maintenance burden, and mitigates technical debt [11, 21]. Consequently, the software engineering community has long advocated for considering code readability during development [25]. The prevalence of Large Language Models (LLMs) has transformed automated programming by reducing developers’ efforts in from-scratch development [35, 56]. However, LLM-generated code often suffers from hallucinations, misinterpreted requirements, or outdated knowledge, necessitating rigorous human oversight to mitigate vulnerabilities [53, 102]. While existing research has extensively validated the functional correctness of LLM-generated solutions [17, 18], the readability of LLM-generated code remains underexplored. As human review remains important for verifying AI-generated outputs, unreadable code not only impedes comprehension but may also require more audit effort than manual authoring, thereby undermining the primary benefits of AI assistance [103]. Consequently, a comprehensive evaluation of readability is essential to understanding emerging LLM-integrated development paradigm and providing a foundation for future AI-assisted programming. To address this knowledge gap, this paper presents a systematic investigation into the readability of LLM-generated code following three core research questions: • RQ1: Does LLM-generated code exhibit higher or lower readability scores compared to human-written code? • RQ2: Does LLM-generated code exhibit distinct patterns of readability issues compared to human-written code? • RQ3: Does prompt engineering influence the readability of LLM-generated code? We started from establishing a comprehensive readability model by integrating textual, structural, program, and visual features into a unified framework, allowing for a robust assessment of code readability. Based on the model, we first evaluated the readability scores of LLM-generated code in comparison to human-written code across 5,869 scenarios extracted from World of Code (WoC) [57] and LeetCode [1]. To characterize the qualitative nuances of the generated code, the second stage of our analysis investigated whether LLM-generated code exhibits specific readability issue patterns different from human-written code. Furthermore, we explored the potential factors that influence these outcomes, specifically examining how various dimensions of prompt engineering affect readability within the context of single-turn interactions.

Ye et al.

We found that the readability of LLM-generated code is at least comparable to that of human-written code, while a critical divergence exists: LLM-generated code exhibits distinct readability issue patterns compared to human-written code, including unnecessary complex structures, low-information comments, and unknown API usage. Further prompt analysis indicated that while function signatures, constraints and style descriptions influence the readability of generated code, its overall impact is constrained and remains insufficient to resolve the readability challenges. Significantly, our findings not only validated the potential for systematic LLM adoption from an overall readability perspective, but also exposed the “hidden technical debt” in AI-assisted programming, offering insights for managing AI-generated code and advancing better LLM-based development assistants. In summary, this paper provides five main contributions: • A scalable benchmark to compare LLM-generated code and human-written code, consisting of 5,869 standard prompts and corresponding human-implemented code across software engineering practice and programming competitions; • An empirical study showing that LLMs have achieved readability scores at least comparable to those of humans, establishing a quantitative baseline; • Identification of AI-specific readability issue patterns, distinguishing AI-assisted programming technical debt from traditional human coding habits; • An empirical quantification of prompt efficacy, establishing practical design guidelines while revealing the insufficiency of prompting alone to address readability issues; • A comprehensive readability assessment framework that integrates textual, structural, program, and visual features to systematically and quickly quantify code readability. In Section 2, we review related work. We illustrate the overall methodology in Section 3 and explain the readability model in Section 4. Sections 5 to 7 detail the results for each research question. We discuss implications and limitations in Section 8 and conclude in Section 9.

2 Background and Related Work 2.1 LLM Code Generation The application of LLMs to code generation substantially enhanced automated programming capabilities. Not only have general LLMs like ChatGPT been widely deployed for code generation [42], but fine-tuned models such as Codex [17] and StarCoder [50] have also emerged to achieve competitive performance across various programming languages. Integrated code agents like Copilot [28] and Cursor [6] further enhance developer productivity. Surveys indicate that Copilot is behind an average of 46% of developers’ code across all programming languages [74], highlighting the profound impact of LLM-generated code on software development. Researchers have focused on examining whether LLM-generated code is functionally correct, secure, and aligned with developer expectations. For instance, CodeXGLUE [55], a comprehensive evaluation framework comprising 14 tasks including code completion, generation, and translation, established a systematic assessment standard for code generation tasks, while HumanEval

benchmark [17] offered a more targeted approach, utilizing reference solutions and unit tests to precisely quantify the functional correctness of LLM-generated code. Additional evaluation criteria have also examined LLM-generated code in terms of vulnerability risks, privacy protection, and copyright compliance [99, 100]. However, little attention is devoted to the readability of the generated code despite its importance to long-term software lifecycle.

2.2

Code Readability

Since the emergence of programming, code readability has become an implicit code quality criterion. Terms such as “clean”, “good”, and “beautiful” have long served as intuitive descriptors for highly readable code [23, 65]. Despite the early lack of standardized readability metrics, developers have long prioritized aesthetic and readable code, with studies investigating how identifier strategies and comments influence perceived code quality [37, 47]. Buse and Weimer [14, 15] developed the first readability model by mapping 25 structural features, such as indentation and line length, to human-annotated code. Their findings established a positive correlation between readability scores and overall code quality. Subsequent studies refined structural metrics, with Posnett et al. [72] achieving higher accuracy through a simpler three-feature model and Johnson et al. [43] exploring how nesting and looping constructs influenced readability. A significant advancement occurred as Scalabrino et al. [78, 79] integrated textual and semantic features with structural ones to capture code’s dual nature as both logic and language. Most recently, Vitale et al. [88] moved toward personalization by incorporating individual developers’ reading preferences and habits into readability assessments. Considering practical software engineering contexts, Piantadosi et al. [71] studied readability fluctuations during software evolution, proposing guidelines to prevent readability degradation. Parallel researches have also examined code readability within LLM context. Takerngsaksiri et al. [83] explored the practitioners’ perspectives on code readability in the age of LLMs through a case study, while Pan et al. [67] investigated the influence that readability have on the LLM budget when LLM processing code snippets. Furthermore, Hu et al. [36] investigated LLMs’ performance when interpreting poor-readability code, finding that even LLMs struggle with lowreadability code, which mirrors human developers’ difficulties that lead to technical debt accumulation. These findings further emphasizes the detrimental impact of low-readability code in software engineering.

2.3

Prompt Engineering

As an efficient alternative to costly tuning and retraining, prompt engineering has emerged as a primary paradigm for optimizing LLM performance across diverse tasks [59, 93]. Since prompt optimizations directly shape model outputs, they likely influence the readability of the generated code. Consequently, this study investigates the impact of prompting strategies within the context of single-turn interactions and we review related prompt engineering research here. Leveraging their extensive pre-training, LLMs function as capable few-shot learners through zero-shot instructions or in-context

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

examples [69, 91]. By activating latent knowledge without resourceintensive fine-tuning, these low-cost and flexible techniques have gained widespread attention among both researchers and users, coalescing into the formalized discipline of prompt engineering. The adoption of prompt engineering across various domains has established task-specific best practices. Researchers have defined design principles for specialized contexts such as academic writing and business management [13, 27], while practical implementations like “cursor rules” [7] utilize pre-set contexts to optimize code generation. These frameworks improve LLMs’ output by explicitly shaping the input prompt [7, 27, 54], including persona settings, stylistic guidelines, function signature definition, technical contracts, etc. Beyond structural optimization, prompt engineering also incorporates reasoning-oriented methods. Chain-of-Thought (CoT) prompting [92] enhances performance by simulating stepby-step problem decomposition. Furthermore, the development of algorithmic automatization like the Automatic Prompt Engineer (APE) [41] has transitioned the field from heuristic adjustments to a systematic and algorithm-driven discipline. In summary, current research on LLM code generation primarily emphasizes functional capacity, and similarly, prompt engineering literature has primarily prioritized strategies for enhancing functional performance, leaving the crucial non-functional dimension of readability unexplored. Establishing a readability baseline, identifying specific issue patterns, and evaluating the impact of prompt engineering thus remain critical yet unaddressed knowledge gaps.

3

Methodology

As shown in Figure 1, we employed a mixed-method approach to address our research questions. We started by collecting and processing source data to obtain structured prompt and respective human implementation. Then we prompted LLMs to generate code responses, and compared the readability of LLM-generated code with human-written code. After that, we identified readability issues through manual labeling and thematic analysis, and investigated the impact of different prompt dimensions on generated code readability using various statistical methods. In the following sections, we detail the approaches of data collection, prompt crafting, prompting and processing, thematic analysis, and statistical analysis.

3.1

Data Collection

To make LLM application scenarios more representable, we selected both real software engineering development contexts and competitive programming problems as task backgrounds. Specifically, we used World of Code (WoC) [57] and LeetCode [1] as data sources. Note that, to ensure the reliability of the human-written baseline, we selected data snapshots prior to 2022. Given that ChatGPT was released in November 2022 [64], before which LLMs were neither widely adopted nor integrated into programming workflows, this temporal boundary justifies the assumption that the collected code is human-written. WoC is a comprehensive infrastructure for mining version control data across open-source software ecosystem, which aggregates Git objects, including commits, trees and blobs, on platforms like GitHub, Bitbucket, and GitLab [57, 58]. It provides mappings from

project identification to project code, blobs, version control information, and other data for each project entity, facilitating easy querying and management. We selected version U of WoC (released in October 2021) and used the c2fbb database, which maps each commit to its corresponding commit file name, new blob (post-commit), and old blob (pre-commit), filtering for Python project files1 . To align with Buse and Weimer’s definition of readability [15], we utilized functionlevel snippets to ensure evaluators have sufficient context and to avoid spanning multiple methods. This granularity provides adequate volume for the reading process while minimizing confounding factors that could distort judgment. We therefore extracted and segmented Python source files into individual function-level entities for analysis. We further divided each snippet into a function header and body: the header, containing the function signature and docstring description, would serve as the prompt to stimulate LLM code generation, while the body, representing the concrete implementation, would serve as human-written baseline. After filtering and selection, we finally obtained 3,000 prompt-baseline pairs. LeetCode [1] is an online platform that provides programming problems spanning various difficulty levels and topics. It offers a standardized problem set with clear problem descriptions in natural languages, and worldwide users provide a diverse array of humanwritten code in multiple programming languages [18]. We crawled all accessible problems from LeetCode, including their problem descriptions and code solutions, and then filtered for solutions implemented in Python and released before 2022. Subsequently, we extracted the core function implementation from each code solution to serve as the human-written baseline, with the corresponding problem description serving as the prompt. Note that, given that primary training corpora for LLMs are in English [51], we selected English descriptions to maximize LLM capabilities. Ultimately, we obtained 2,869 pairs from LeetCode. Combining with WoC-sourced data, we got 5,869 effective samples in total.

3.2

Prompt Crafting

To support our multi-stage investigation, we crafted two different prompt sets tailored to different research objectives. The first set A should provides a diverse array of real-world scenarios to evaluate the general readability of LLM-generated code and identify specific defect patterns (RQ1 and RQ2), while the second set B should facilitate a controlled experimental environment for RQ3. This strategic separation allows us to maintain ecological validity for problem identification while ensuring the empirical rigor required for variable-controlled analysis of prompt dimensions. 3.2.1 Set A: Real-World Scenario Prompts (RQ1 & RQ2). This set establishes a baseline for general readability and defect identification. Specifically, we used the 5,869 samples previously collected from real software engineering practice as the foundational data, which includes 2,869 natural language problem descriptions from LeetCode and 3,000 function-level docstrings from WoC. To ensure consistency, two authors manually transformed the technical docstrings into coherent natural language instructions. This process involved independent transformation followed by a rigorous 1 To maintain a manageable research scope, in this study, we focused specifically on

Python code readability.

Ye et al.

Figure 1: Overview of the methodology cross-verification phase. Any discrepancies were resolved through iterative discussion to a consensus on all final expressions. We finally got a comprehensive set of 5,869 prompts reflecting diverse real-world development tasks.

Finally, each baseline corresponded to 16 different prompts and vectors representing their crafting dimensions, ultimately yielding 5,248 operational (prompt, vector) pairs in set B.

3.2.2 Set B: Controlled Experimental Prompts (RQ3). Analyzing how specific prompt dimensions affect readability requires a rigorous control of variables. The complexity of real software practice in the first set makes it difficult to decouple specific descriptive elements into isolated variables. Thus, we adopted a from-scratch construction strategy, prioritizing simplicity and modularity to ensure experimental control. We sourced problems from MBPP [8] and HumanEval [17] as they are established benchmarks with concise descriptions that are easily decoupled into base components. To maintain a balanced sample size, we combined 164 samples from HumanEval with a randomly sampled subset of 164 problems from MBPP. Following a similar refinement pipeline, since MBPP has provided direct natural language descriptions, we converted HumanEval docstrings into basic task descriptions, decoupling core logic from additional expressions to facilitate a controlled experiment. This process was followed by author cross-validation to ensure rigor and accuracy. Refering to prior work [27, 93, 101], we limited the dimensions of prompt crafting to 8 types, as shown in Table 1. Minimum task description served as the baseline, consisting of the most basic problem description without any additional information. Function signature, IO contract, few-shot instruction, and task category were extracted from the code implementation and test cases, while style description, persona setting, and constraint were derived from best practices in cursor rules [7]. Starting with each problem’s baseline prompt, we constructed prompts that only added one dimension, omitted only one dimension, or included all dimensions and recorded dimensions that current prompt contained with a seven-element vector:

3.3

𝑉prompt = (I{𝑑})𝑑 ∈ D , where D = {style, signature, IO, few-shot, category, persona, constraint}, and I{𝑑 } = 1 if dimension 𝑑 is included in the prompt, otherwise 0.

Prompting and Data Processing

For the prompt set A (RQ1 & RQ2), we input the 5,869 prompts into a set of prevalent LLMs, including GPT-4o [2], Grok-3 [98], Claude-3.7 [5], DeepSeek-v3 [29], and Llama 3.1 [86], to obtain their responses. We extracted code body from the markdownformatted text returned by LLMs and cleaned non-function parts, such as execution examples. We then applied the readability model to LLM-generated code and the corresponding human-written baseline. After that, we compared the readability between code authored by LLMs and humans, as well as between code generated by different LLMs using the same prompt. Regarding the prompt set B (RQ3), to conserve computing resources, based on the readability assessment results, we selected the LLM that demonstrated the best code readability performance for further experimentation, which turned out to be claude-3.7. We then inputted the 5,248 prompts to obtain the returned markdownformatted text, and used similar methods to extract Python code and remove extraneous information. We then evaluated the readability of the generated code using our readability model.

3.4

Thematic Analysis

To address RQ2, we conducted a comparative and thematic analysis [84, 96] of LLM-generated code and human-written code. We first identified the most readable model outputs and then randomly sampled 500 pairs from both the WoC-sourced and LeetCode-sourced data for subsequent analysis. Based on readability scores, samples were categorized into those where LLMs outperformed humans (405 WoC, 328 LeetCode) and those where humans scored higher (95 WoC, 172 LeetCode). Subsequently, two authors independently read the code pairs and manually annotated them. Specifically, each annotator first read both LLM and humanauthored code in their entirety and identified which of the four basic

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

Table 1: Dimensions of Prompt Crafting

Table 2: Common Readability Issue Patterns

Dimension

Explanation

Issue Pattern

Explanation

Minimum task description

The core instruction or objective of the prompt as brief and clear as possible.

Deficient Comment (DC)

Style description

Instructions specifying the tone, format and overall aesthetic of desired output.

Critical or complex code lacks necessary explanations, purpose descriptions, or non-obvious details.

Redundant Comment (RC)

Function signature

A specific instruction that forces the model’s output to conform to a structure, typically including the name, parameters, and type hint.

Comments merely duplicate information that is already clear from the code itself, offering no added value.

Inconsistent Style (IS)

IO Contract

A precise description of the expected data structure, format, and content for both the LLM input and output.

The code fails to follow an established, uniform standard in naming conventions, indentation, formatting, or idiomatic usage.

Excessive Complexity (EC)

Few-shot instruction

Providing the model with several examples of the input/output pairs for the specific task.

The logic, control flow, or structure of the code is overly convoluted, deeply nested, or uses unnecessarily complicated mechanisms.

Task category

Explicitly naming the general type of problem or task the model is being asked to perform.

Poor Structure (PS)

Code snippets have poor modularity, disorganized logic, or an unclear structural hierarchy.

Persona setting

Assigning the model a specific identity, role, or character to adopt during the task execution.

Poor Naming (PN)

Constraint

Restrictive guidelines specifying what the model must exclude or minimize.

Names for variables, functions, classes, or other identifiers are vague, misleading, or fail to accurately reflect their true purpose or scope.

Magic Values (MV)

The code directly uses raw constant values (like numbers or strings) that lack explanatory context.

Code Duplication (CD)

Highly similar code fragments appear multiple times in different parts of the program, increasing maintenance cost and potential for errors.

High Coupling (HC)

Different functional parts are excessively dependent on one another, where a change in one component requires cascading changes in others.

readability dimensions exhibited poorer readability in the lowerscoring code, which include Textual Features (TF) [78], Posnett’s Features (PF) [72], Buse and Weimer’s Features (BWF) [15], and Dorn’s Features (DF) [24]. We detail these dimensions in Section 4. Subsequently, based on prior literature on code readability [43, 63, 66, 71, 78], we synthesized 9 common readability issue patterns, as shown in Table 2. Annotators identified readability defects in less readable snippets relative to their counterparts. Any issues not captured by the nine predefined patterns were addressed through open coding. Independent annotations yielded Cohen’s Kappa values of 0.87 for dimensionality assessment and 0.81 for issue pattern identification, reflecting substantial inter-rater agreement. All discrepancies were resolved through discussions to reach consensus. For the opencoded segments, we synthesized individually derived codes into novel readability issue patterns distinct from the predefined ones. To ensure analytical rigor, we employed triangulation [34] across different data sources (WoC and LeetCode) and conducted negative case analysis [32] to refine our understanding of nuanced readability differences. This iterative process continued until theoretical saturation was reached, with no further novel insights emerging from the data.

3.5

Statistical Analysis

To address RQ3, we conducted a comprehensive statistical analysis on the state vectors of 5,248 prompts in set B and their corresponding readability scores of LLM-generated code. First, we performed a random forest regression analysis [12] across all samples, with 𝑛_𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟 = 100, 𝑚𝑎𝑥_𝑑𝑒𝑝𝑡ℎ = 15, 𝑚𝑖𝑛_𝑠𝑎𝑚𝑝𝑙𝑒𝑠_𝑠𝑝𝑙𝑖𝑡 = 5. Random Forest’s non-linear modeling capability is well-suited for our analysis, and it provides feature importance rankings [38] that identify which prompt dimensions exert relatively greater influence on LLM-generated code readability scores. Furthermore, we evaluated the statistical significance of each dimension’s impact through methodological triangulation, combining t-tests [61] and permutation importance testing [4]. While

t-tests identify significant readability score differences between baseline and incremental prompts (𝑝 < 0.1), permutation testing quantifies each dimension’s predictive contribution. Specifically, we utilized a random forest model as prediction kernel, and then shuffled the binary labels of the tested dimension across 10, 000 iterations, finally considering a dimension relevant if its predictive performance significantly exceeded random chance (𝑝 < 0.1). By synthesizing these two statistical perspectives, we determined whether each dimension significantly correlates with LLMgenerated code readability. To ensure reliability, we also conducted an ablation study, which compared full-dimension prompts against those lacking a single dimension, to corroborate findings with inverse evidence.

4

Readability Model

To evaluate code readability, we combined established readability metrics from prior work into a unified representation that comprehensively quantifies readability and helps identify its issue patterns. Specifically, we constructed an integrated metric encompassing TF [78], PF [72], BWF [15], and DF [24]. This selection is based on extensive code readability literature [15, 24, 47, 72, 78, 79], representing widely adopted indicators of code comprehension. Notably, these four families capture complementary perspectives on readability: TF focuses on textual and lexical characteristics, PF reflects information-theoretic and distributional properties of programs, BWF models formatting and whitespace cues, and DF captures visual presentation features. By synthesizing these diverse dimensions, our integrated metric provides a broad and complementary coverage of readability-related properties, reducing the bias of relying on any single metric family [78, 79].

Ye et al.

Due to space constraints, the exhaustive technical details of our measurement framework, including mathematical formulations, metric selection criteria, and comprehensive model evaluation, are provided in appendix A for reference and reproducibility. In this section, we focus on the qualitative rationale of the selected metrics and highlight the pivotal aspects of the model’s implementation and evaluation.

of code on the page. In particular, DF reflects whether a snippet exhibits a regular spatial layout, consistent alignment of syntactic elements, and visually meaningful decoration patterns. Such properties influence how efficiently readers can detect visual patterns before fully parsing the text, as prior studies have suggested that spatial regularity and alignment facilitate code navigation [16, 19, 20, 80].

4.1

To realize the metric families defined above, we implemented a unified pipeline that extracts measurable features from the abstract readability dimensions. In this stage, we focused on transforming raw code snippets into standardized readability representations, preparing for subsequent feature selection and evaluation. In particular, our workflow preserves layout-sensitive formatting to maintain the structural and visual indicators of readability, which are then parsed by language-aware extraction routines to derive essential signals for our readability metrics. Building on these extracted signals, we recombined established metrics from prior work into a unified representation [62, 77, 78] to instantiate the four complementary families: TF, BWF, PF, and DF. All metrics are represented as snippet-level scalar features and concatenated in a fixed order to form a unified feature vector x = [TF, BWF, PF, DF]. In our implementation, TF, BWF, PF, and DF yield 16, 26, 4, and 15 features, respectively, resulting in a raw 61-dimensional representation in total. This unified 61-dimensional representation serves as the input space for our downstream readability prediction and family-wise analysis. To keep the main text focused on the overall workflow and its role in constructing the feature space, we defer implementation details to the appendix provided within the replication package. These details include preprocessing rules, language-specific extraction patterns, normalization procedures, and feature aggregation mechanisms.

4.2 Metric

We summarize all the metrics in our comprehensive readability model in Table A.1, which are grouped into TF, BWF, PF and DF. Checkmarks denote the computed statistics. 4.1.1 TF: Semantic and Lexical Perspective. To capture the impact of semantic and lexical features on code readability, we followed Scalabrino et al. [78, 79] to adopt TF features. This family is motivated by the observation that identifiers and comments are primary carriers of developer intent [46, 48]. Accordingly, TF captures whether the wording used in code is natural, semantically precise, and internally consistent with the documented behavior of the snippet. Representative signals in this family describe naming quality, comment–code consistency, lexical ambiguity, and conceptual cohesion. Collectively, these features reflect how easily a reader can correlate program text with the underlying problem domain and recover the intended meaning of the code implementation. 4.1.2 BWF: Structural Formatting Perspective. We employed BWF features following Buse and Weimer [15] to capture the formatting and layout characterization. This family focuses on low-level structural regularities that can be perceived directly from the arrangement of code on the screen without deep semantic analysis. In our setting, BWF summarizes three complementary aspects of structure: the horizontal density of individual lines, the syntactic distribution within lines, and the vertical segmentation induced by blank lines and comments. Collectively, these features reflect whether a snippet is visually crowded, unevenly distributed, or clearly partitioned into manageable units, all of which affect scanning effort and local comprehension [14, 15, 97]. 4.1.3 PF: Information-Theoretic Perspective. To assess readability from an information-theoretic perspective, we incorporated PF features based on Posnett’s entropy-oriented formulation [72]. The underlying rationale is that code becomes harder to process when its information content is high or its token distribution is irregular. This family therefore captures the compactness, diversity, and unpredictability of the textual signal produced by the code. Representative features in this group quantify the overall information volume of a snippet and the degree of lexical disorder between lines. From a cognitive perspective, these measures approximate the amount of information that a reader must organize and retain when reading code [31, 72, 81]. 4.1.4 DF: Visual and Geometric Perspective. To complement the aforementioned non-visual dimensions, we adopted DF features introduced by Dorn [24], which model code as a visual object. This family captures readability from a geometric and perceptual perspective by describing the shape, alignment, and visual distribution

4.3

Implementation

Evaluation

To evaluate the efficacy of our code readability assessment framework, we conducted experiments on the Ddorn dataset [24], a widely used benchmark for code readability evaluation. In particular, the dataset consists of 360 code snippets across three programming languages: Java, Python, and CUDA. Each code snippet is labeled with binary readability scores based on human expert annotations. We adopted a 10-fold stratified cross-validation protocol [45] to obtain a robust evaluation. Within each cross-validation fold, features are standardized, preventing features with larger scales from dominating the model. Given the high dimensionality of our initial 61-feature space, it is crucial to filter out noisy signals across different metric families. Therefore, we applied Sequential Forward Selection (SFS) [30, 94] to identify the discriminative subset of features. We utilized a Logistic Regression classifier with L2 regularization [10] as the base estimator for SFS. For each setting, SFS starts from the full candidate feature set and is performed within the training split of each fold. Accordingly, the Features column in Table 3 reports the optimal number of selected features per setting. We reported two evaluation metrics, Accuracy and Area Under the ROC Curve (AUC) [26]. These metrics capture both classification accuracy and ranking quality. The evaluation covers the four

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

Table 3: Cross-validated performance of our readability assessment framework. 6

TF BWF PF DF All-features

Features

Accuracy

AUC

15 24 3 14 25

62.2% 73.3% 66.4% 66.4% 77.5%

65.3% 79.4% 70.6% 76.2% 83.8%

4 Score

Feature Family

2 0 2

Table 4: Readability Scores of LLM-generated and Humanwritten Code

4

Avg (LLM)

Avg (Human)

Win Rate (%)

Claude-3.7

WoC LeetCode All

0.85 2.15 1.29

0.82 1.45 0.77

65.60 81.00 73.30

GPT-4o

WoC LeetCode All

0.45 1.89 1.00

0.09 1.43 0.61

67.54 73.08 69.67

Grok-3

WoC LeetCode All

0.24 2.13 1.18

0.09 1.45 0.77

57.60 78.80 68.20

DeepSeek-v3

WoC LeetCode All

0.44 1.84 1.14

0.09 1.45 0.77

64.26 65.00 64.63

Llama 3

WoC LeetCode All

0.33 1.80 1.13

0.14 1.45 0.86

57.25 65.40 61.71

All

WoC LeetCode All

0.38 1.97 1.15

0.10 1.45 0.76

62.63 72.62 67.52

*

Avg refers to the average readability score of corresponding code snippets evaluated by the readability model; Win Rate refers to the proportion of samples where LLM-generated code strictly outperforms the human-written counterpart in readability.

individual feature families: TF, BWF, PF, and DF, as well as their concatenation, denoted as All-features. Overall, the results in Table 3 indicate that our unified feature framework captures a richer set of readability signals than any single category. The All-features combination achieves peak performance (77.5% accuracy, 83.8% AUC), consistently outperforming individual feature families and previously proposed standalone perspectives [14, 24, 72, 79]. This highlights the complementary nature of our four dimensions and suggests that a multi-faceted representation is essential for capturing the full complexity of readability.

LLM

Human

(a) Boxplot of the readability scores LLM Human 800

Frequency

Data Source

600

400

200

0

4

2

0

2 Score

4

6

(b) Histogram of the readability scores

Figure 2: Comparison of the readability scores of LLMgenerated code and Human-written code Claude-3.7 (WoC) Claude-3.7 (LeetCode) DeepSeek-v3 (WoC) DeepSeek-v3 (LeetCode)

Model (Data Source)

Model

GPT-4o (WoC) GPT-4o (LeetCode) Grok-3 (WoC) Grok-3 (LeetCode)

5

RQ1: Comparison of the Readability Scores

Following established methodology, we conducted a comparative readability analysis between LLM-generated and human-written code, with results summarized in Table 4 and Figure 2. Table 4 shows that LLM-generated code consistently achieves higher average readability than human-written code across all models, outperforming humans in over 50% of cases. Although WoC-sourced samples exhibit lower absolute scores than LeetCode, which is likely due to the inherent complexity of real-world software scenarios, it does not undermine the validity of our relative comparison, as LLM and human outputs are evaluated within identical task contexts.

Llama-3.1 (WoC) Llama-3.1 (LeetCode) 2

0

2

Readability Score

4

6

Figure 3: Readability score distribution of code generated by different LLMs Figure 2 illustrates the readability score distributions, revealing that while LLMs achieve a marginally higher median, humanwritten code demonstrates superior consistency, as evidenced by a tighter interquartile range and fewer outliers (Figure 2a). This

Ye et al.

phenomenon indicates that LLM-generated code has a less stable readability level, which may be attributed to the inherent nexttoken prediction mechanism [87] underlying LLM code generation, in contrast to programmers’ established programming habits [60]. Figure 2b demonstrates substantial distributional overlap of LLMgenerated and human-written code, with marginally higher modes and peak scores in LLM outputs. To establish a more rigorous comparison, we employed a Wilcoxon signed-rank test [95] on 𝑁 = 5, 869 matched pairs, as the paired design accounts for task-specific variance. The results reveal a statistically significant difference between two groups (𝑊 /𝑊𝑚𝑎𝑥 = 0.270, 𝑝 < 0.001). The moderate effect size (𝑟 = 0.398) indicates a consistent but modest performance lead for LLM-generated code over human-written code when evaluated on identical tasks. Thus, considering the large sample size and practical complexities, the readability of LLM-generated code is at least comparable to, if not slightly higher than, that of human-written code. Additionally, the 5,869 collected prompt-code pairs provide a benchmark enabling a cross-model evaluation of code readability. As shown in Figure 3, the results confirm that simpler LeetCode tasks yield more readable code than complex WoC scenarios across all models. Notably, Claude-3.7 emerges as the top performer, achieving the highest average readability with the lowest variance. Since LLM capacity improves constantly with its development, it is reasonable to focus on Claude-3.7 for subsequent analyses to ensure both representativeness and computational efficiency. Summary for RQ1: Analysis of 5,869 matched code pairs across WoC and LeetCode reveals that LLM-generated code achieves statistically higher readability than human-written code (𝑝 < 0.001) with a moderate effect size (𝑟 = 0.398). It suggests that they are at least comparable in practice. However, a key distinction lies in stability: LLMs reach higher peak readability, whereas human-written code exhibits superior consistency. By establishing this practical parity alongside the stability distinction, our findings validate the non-functional potential of LLMs for systematic software development, while our dataset provides a robust benchmark for future comparative evaluations.

6

RQ2: Distinct Readability Issue Patterns

Table 5 illustrates readability issues between LLM-generated and human-written code across the four analytical dimensions detailed in Section 4. To briefly recap, TF (Semantic/Lexical) evaluates intent and naming; BWF (Structural/Formatting) assesses layout and density; PF (Information-Theoretic) measures logical complexity and entropy; and DF (Visual/Geometric) captures spatial alignment. Our results indicate that poorly-rated human code primarily exhibits issues in BWF and TF, with PF being the least common, while worse-performing LLM code predominantly struggles with TF and PF, leaving BWF as the minor concern. These distinct patterns reflect the underlying differences in code generation mechanisms. Human programmers often prioritize functional correctness and development efficiency over stylistic rigor

Table 5: Distribution of Issues on Readability Dimensions Category

TF

BWF

PF

DF

Human_bad LLM_bad Total

72 60 132

76 26 102

21 53 74

45 30 75

Table 6: Categories of Readability Issue Patterns Category

DC

RC

IS

EC

PS

PN

MV

CD

HC

Human_bad LLM_bad Total

60 10 70

3 24 27

58 15 73

41 48 89

24 16 40

22 21 43

7 8 15

6 2 8

3 0 3

and considerate textual expression, leading to BWF and TF issues [49, 76]. However, their experience-driven reliance on validated concise logic typically results in lower information entropy and predictable structures, thereby minimizing PF issues [60]. In contrast, having been trained on substantial amounts of standardized code [2, 87], LLMs tend to generate style-standard code with less BWF issues, but their lack of experiential best practice often leads to correct but over-engineered solutions, manifesting as PF logic bloat [52, 103]. Furthermore, TF emerged as the most frequent issue dimension in LLM-generated code, which could be attributed to the lack of emphasis on semantic appropriateness in training data and model’s tendency to hallucinate text that is irrelevant to the actual code logic [40, 44]. Table 6 presents all readability issue patterns in the groups with worse readability. In human-written code, Deficient Comments (DC) and Inconsistent Style (IS) are the most common problems, while Redundant Comments (RC) and High Coupling (HC) are rare. This pattern reflects the tendency of programmers to prioritize functional correctness and speed over non-essential tasks like documentation and styling [49]. Regarding comments as secondary to execution, developers often omit necessary explanations or simplify them too much, which leads to high DC but very few RC issues [76]. Similarly, high IS suggests that developers may neglect consistent formatting while focusing on solving the problem at hand. The most prominent issue in LLM-generated code is Excessive Complexity (EC), while Code Duplication (CD) and High Coupling (HC) are infrequent. Unlike humans, LLMs do not inherently aim for the simplest solution. Instead, they often generate over-complicated code that is technically correct but difficult to follow, significantly leading to EC issues [52, 103]. Furthermore, while humans tend to skip comments, LLMs often produce Redundant Comments (RC) by following “add comments” instructions too literally, which instead reduce clarity [44]. The low incidence of CD suggests that LLMs are good at modularizing code and reusing logic without repetition. Finally, both human and LLM code show very low levels of HC. This is likely because our study focuses on function-level snippets, which naturally avoid HC phenomenon. Identifying HC typically requires analyzing file-level code, which is beyond the scope of current readability models. We further discuss it in Section 8.

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

Beyond predefined patterns, our open coding revealed three novel issues specific to LLM-generated code: Unknown API, Redundant Variables, and Overblanking. Unknown API occurs when LLMs introduce external library functions without adequate explanation. Whether these APIs are hallucinations or real, their inclusion often makes the code difficult to follow, likely because models rely on vast training knowledge without knowing whether the referenced libraries are accessible or familiar to the end user [75, 89]. Redundant Variables refer to cases where LLMs define multiple variable names for the same functional purpose, leading to confusion and unnecessary repetition. It likely stems from LLMs’ token-bytoken generation, where context tracking of previously defined variables may be insufficient [52]. Overblanking involves excessive but meaningless blank lines that unnecessarily lengthen the program. Though related to code style, its high prevalence in LLMgenerated code warrants a separate category, suggesting that LLMs often mimic human formatting patterns without understanding the actual structural or semantic purpose of the spacing.

Table 7: P-value of Statistic Analysis of Prompt Dimensions and Code Readability

Dimension

Incremental

Ablation

T-test

Permutation

T-test

Permutation

Style description

0.001

0.001

0.795

0.850

Function signature

0.239

0.240

0.003

0.003

IO Contract

0.353

0.356

0.855

0.887

Few-shot

0.257

0.291

0.643

0.678

Task category

0.282

0.322

0.673

0.684

Persona setting

0.287

0.282

0.281

0.287

Constraint

0.001

0.000

0.807

0.960

*

Bold values indicate statistical significance (𝑝 < 0.05).

Summary for RQ2: LLM-generated and human-written code exhibit distinct readability issue patterns. Poorly-rated human code primarily suffers from structural (BWF) and semantic (TF) issues, specifically DC and IS, reflecting a prioritization of functional correctness over stylistic rigor, while LLMs predominantly struggle with semantic (TF) and information-theoretic (PF) dimensions, characterized by EC and RC. Notably, we identified three novel LLM-specific patterns: Unknown API, Redundant Variables, and Overblanking, which stem from LLM generation mechanisms rather than human coding habits.

7

RQ3: Prompt Efficacy on Code Readability

Figure 4 shows the feature importance F-scores of random forest regression. Function signature demonstrates the highest importance among all evaluated prompt dimensions, followed by constraints and style description, while task category and IO contract play minimal roles. This hierarchy suggests that function signatures establish the stylistic foundation for function-level snippets [22], further influencing readability, while style descriptions and constraints provide

0.4011

Function signature

0.2042

Prompt Dimension

Constraints

0.1781

Style description

0.0708

Persona setting

0.0588

Few-shot instruction

Task category

0.0455

IO contract

0.0415

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Feature Importance

Figure 4: Feature importance of the Random Forest regression result

complementary explicit guidance, offering “do’s” and “don’ts” for textual clarity that aligns with intuitive expectations. Conversely, task category and IO contract appear to drive functional logic and problem-solving rather than influence readability. Notably, the low 𝑅 2 (< 0.3) of our random forest model indicates that prompt engineering has a limited impact on readability scores. As described in Section 3, to substantiate this observation, we conducted statistical significance tests and ablation analysis to isolate the precise effects of each dimension, with results shown in Table 7. Our incremental analysis, which compares baseline prompts with single-dimension additions, shows that style descriptions and constraints significantly correlate with improved code readability. This suggests that when starting from a minimal task description, explicitly declaring readability requirements, whether through positive directives or negative restrictions, maximally improves code readability. On the other hand, our ablation analysis reveals that when using comprehensive prompts, only function signature maintains a significant correlation with readability. This indicates that in well-designed prompts, function signature is crucial for output quality. We hypothesize that signatures help define the “contextual domain” of the generation process, guiding the LLM to adhere to better readability standards when other instructions are relatively complete [22]. These results strongly reinforce our earlier findings through random forest regression. Synthesizing these analyses, we found that function signatures, constraints, and style descriptions are the three most influential prompt dimensions for code readability. Specifically, constraints and style descriptions help simple prompts reach their maximum potential, while function signatures ensure more readable outputs in well-designed prompts. It is important to note that from an absolute perspective, the overall impact of prompt engineering on readability is relatively modest compared to other dominant factors like problem complexity and model capability [3, 52]. However, for users seeking to improve code readability at minimal cost, optimizing these three specific prompt dimensions provides a practical and valuable strategy.

Ye et al.

Summary for RQ3: Among the seven evaluated dimensions, function signatures, style descriptions, and constraints emerged as the most influential factors. Specifically, style descriptions and constraints enhance basic prompts through explicit “do’s” and “don’ts,” while function signatures serve as a vital contextual anchor in more complex prompts. However, prompt engineering alone remains a relatively weak strategy for improving code readability, highlighting the need for more specialized solutions.

8 Discussion 8.1 Implications Our findings offer practical insights for advancing automated software engineering tasks, specifically in tool development, model evaluation, and management of AI-driven software evolution. Multi-dimensional Model Evaluation and Optimization. Current automated evaluation benchmarks for code-generating LLMs focus almost exclusively on functional correctness and task completion rate via unit tests [55, 90]. However, our finding that LLMs achieve numerical readability parity with humans while introducing distinct readability issues highlights significant potential for LLMs to enter systematic software development, but a novel critical correctness-readability gap. Models that generate functionally correct but cryptic code ultimately impose greater cognitive burdens on developers in code maintenance [39]. Thus, we advocate for a multi-dimensional automated evaluation approach. Future benchmarks should integrate non-functional metrics such as readability into their optimization objectives. For model developers, incorporating these readability metrics into the RLHF loop could help automate the generation of code that is not only correct but also architecturally elegant and easier to read. Automated Code Review. Our identification of distinct readability issues in LLM-generated code suggest that traditional static analysis tools [68], which are primarily designed to catch humantypical “bad smells,” may be insufficient for the LLM era. Our results show that, besides LLM-specific readability issues (i.e., Unknown API usage, Redundant Variables, and Overblanking), there is a fundamental divergence: while human-typical readability issues often involve omission (e.g., Deficient Comments), LLM-generated code suffers from commission (e.g., Excessive Complexity). This inspires the development of AI-specific automated reviewers, which should be designed to detect logical bloat, unnecessary variable redundancies, and hallucinated or obscure API usage. Integrating these specialized rules into CI/CD pipelines can help automate the quality gate, ensuring that the productivity gains of AI-assisted coding do not come at the cost of future maintainability [9]. LLM-Generated Code Detection. As LLM-generated code increasingly populates open-source ecosystems, the ability to automate code attribution becomes crucial for security and licensing [40]. Our results suggest that high-level readability patterns provide more interpretable and robust signals besides low-level statistical markers (e.g., token distribution) used in traditional detection methods. ASE researchers can leverage these stylistic and structural fingerprints to build explainable automated detectors. Such tools would not only identify the provenance of a code snippet

but also provide a rationale based on readability characteristics, which is important for transparency in software supply chains. Technical Debt in AI-Assisted Development. The emergence of novel readability issue patterns in LLM-generated code indicates a new form of AI-driven technical debt [85]. Traditional approaches to identifying and measuring code quality issues may inadequately capture LLM-specific problems, potentially leading to underestimation of maintenance burdens in AI-assisted development projects. Automated software maintenance tools should be updated to account for these specific AI-generated patterns. Specifically, automated refactoring agents could be trained to optimize LLM output in a targeted manner, such as collapsing redundant logic, to align LLM-generated code with project-specific readability standards before it enters the codebase. Towards Improving Code Readability. Our analysis of prompt design effectiveness reveals that while overall impact remains limited, specific dimensions like function signature, style description, and constraints are relatively most efficient for readability improvement. As a low-cost strategy, practitioners should focus their prompt optimization efforts on these high-impact dimensions to maximize efficiency. Meanwhile, the limited effectiveness of the identified prompt design dimensions in improving readability shed light on the need for alternative approaches, such as post-processing techniques [70], fine-tuning strategies specifically targeting readability objectives [91], or hybrid human-AI collaborative coding workflows [33] to better improve the readability level of LLM-generated code, finally fostering a healthier software ecosystem.

8.2

Threats to Validity

8.2.1 Internal Validity. . A primary threat to internal validity concerns the challenge of capturing the subjective and multifaceted nature of code readability. Although our model integrates dimensions from validated prior work, it may not be exhaustive, as individual developer preferences vary and cannot be fully measured by a standardized set of metrics. Furthermore, inherent complexity differences between our data sources, where WoC represents production-grade code and LeetCode focuses on constrained algorithmic challenges, inevitably influence code structure and require caution when interpreting cross-source comparisons [77]. Finally, our focus on function-level snippets limits the detection of high coupling, which typically manifests at the file or project level. Therefore, its low prevalence in our findings reflects our chosen granularity rather than an absence of the defect in real-world scenarios.

8.2.2 External Validity. The generalizability of our findings is limited by the focus on Python and a specific subset of LLMs, as results may vary for languages with different syntactic conventions [82] or for models specifically fine-tuned for code generation. Additionally, although our sample is substantial, it may not fully reflect the complexities of production environments or the iterative nature of human-AI collaboration. Real-world code generation often involves multi-round interactions and strict project-specific constraints that independent snippets cannot capture, highlighting the need for future research into collaborative, large-scale software engineering.

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

9

Conclusion

This study evaluates the readability of LLM-generated code against human baselines. While numerical metrics suggest readability parity, our analysis moves beyond readability scores to reveal fundamental qualitative divergences. Poorly-rated human code is primarily characterized by Deficient Comments and Inconsistent Style, whereas LLM-generated code predominantly exhibits Excessive Complexity and Redundant Comments. Additionally, we identified novel, AI-specific issue patterns, such as Unknown API usage, Redundant Variables, and Overblanking, which may correlate with the inherent mechanisms of generative models. Furthermore, while function signatures, style descriptions, and constraints are the most influential prompt dimensions to output readability, overall impact of prompt engineering remains modest. These findings validate the potential for systematic AI integration into software workflows from a non-functional perspective while highlighting its latent technical debt, encouraging future research into approaches to enhance the readability of LLM-generated code to ensure long-term maintainability.

References [1] [n. d.]. LeetCode. https://leetcode.com/problemset/. [2] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). [3] Duaa Alawad, Manisha Panta, Minhaz Zibran, and Md Rakibul Islam. 2019. An empirical study of the relationships between code readability and software complexity. arXiv preprint arXiv:1909.01760 (2019). [4] André Altmann, Laura Toloşi, Oliver Sander, and Thomas Lengauer. 2010. Permutation importance: a corrected feature importance measure. Bioinformatics 26, 10 (2010), 1340–1347. [5] Anthropic. [n. d.]. Claude. https://www.anthropic.com/claude/. [6] Anysphere. [n. d.]. Cursor. https://www.cursor.com/. [7] Ashley. 2024. What is .cursorrule and How to Use It Effectively. https: //medium.com/towards-agi/what-are-cursor-rules-and-how-to-use-themec558468d139. [8] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 (2021). [9] Alberto Bacchelli and Christian Bird. 2013. Expectations, outcomes, and challenges of modern code review. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 712–721. [10] Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning. Vol. 4. Springer. [11] Dustin Boswell and Trevor Foucher. 2011. The art of readable code. " O’Reilly Media, Inc.". [12] Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5–32. [13] Kiran Busch, Alexander Rochlitzer, Diana Sola, and Henrik Leopold. 2023. Just tell me: Prompt engineering in business process management. In International Conference on Business Process Modeling, Development and Support. Springer, 3–11. [14] Raymond PL Buse and Westley R Weimer. 2008. A metric for software readability. In Proceedings of the 2008 international symposium on Software testing and analysis. 121–130. [15] Raymond PL Buse and Westley R Weimer. 2009. Learning a metric for code readability. IEEE Transactions on software engineering 36, 4 (2009), 546–558. [16] Teresa Busjahn, Roman Bednarik, Andrew Begel, Martha Crosby, James H Paterson, Carsten Schulte, Bonita Sharif, and Sascha Tamm. 2015. Eye movements in code reading: Relaxing the linear order. In 2015 ieee 23rd international conference on program comprehension. IEEE, 255–265. [17] Mark Chen. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). [18] Tristan Coignion, Clément Quinton, and Romain Rouvoy. 2024. A performance study of llm-generated code on leetcode. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering. 79–89. [19] Martha E Crosby and Jan Stelovsky. 2002. How do we read algorithms? A case study. Computer 23, 1 (2002), 25–35.

[20] Pablo Roberto Fernandes de Oliveira, Rohit Gheyi, José Aldo Silva da Costa, and Márcio Ribeiro. 2024. Assessing Python Style Guides: An Eye-Tracking Study with Novice Developers. In Simpósio Brasileiro de Engenharia de Software (SBES). SBC, 136–146. [21] George Digkas, Alexander Chatzigeorgiou, Apostolos Ampatzoglou, and Paris Avgeriou. 2020. Can clean new code reduce technical debt density? IEEE Transactions on Software Engineering 48, 5 (2020), 1705–1721. [22] Xi Ding, Rui Peng, Xiangping Chen, Yuan Huang, Jing Bian, and Zibin Zheng. 2024. Do code summarization models process too much information? function signature may be all that is needed. ACM Transactions on Software Engineering and Methodology 33, 6 (2024), 1–35. [23] John Dooley and John Zukowski. 2011. Software development and professional practice. Springer. [24] Jonathan Dorn. 2012. A general software readability model. MCS Thesis available at (web.eecs.umich.edu/~weimerw/students/dorn-mcs-paper.pdf) 5 (2012), 11–14. [25] James L Elshoff and Michael Marcotty. 1982. Improving computer program readability to aid modification. Commun. ACM 25, 8 (1982), 512–521. [26] Tom Fawcett. 2006. An introduction to ROC analysis. Pattern recognition letters 27, 8 (2006), 861–874. [27] Louie Giray. 2023. Prompt engineering with ChatGPT: a guide for academic writers. Annals of biomedical engineering 51, 12 (2023), 2629–2633. [28] Github. [n. d.]. Copilot. https://github.com/features/copilot. [29] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence. arXiv preprint arXiv:2401.14196 (2024). [30] Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. Journal of machine learning research 3, Mar (2003), 1157– 1182. [31] Maurice H Halstead. 1977. Elements of Software Science (Operating and programming systems series). Elsevier Science Inc. [32] Ardis Hanson. 2017. Negative case analysis. The international encyclopedia of communication research methods (2017), 1–2. [33] Mohammad Hassany, Jiaze Ke, Peter Brusilovsky, Arun Balajiee Lekshmi Narayanan, and Kamil Akhuseyinoglu. 2024. Authoring Worked Examples for JAVA Programming with Human AI Collaboration. In Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing. 101–103. [34] Roberta Heale and Dorothy Forbes. 2013. Understanding triangulation in research. Evidence-based nursing 16, 4 (2013), 98–98. [35] Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology 33, 8 (2024), 1–79. [36] Chao Hu, Yitian Chai, Hao Zhou, Fandong Meng, Jie Zhou, and Xiaodong Gu. 2024. How Effectively Do Code Language Models Understand Poor-Readability Code?. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 795–806. [37] Yuan Huang, Nan Jia, Junhuai Shu, Xinyu Hu, Xiangping Chen, and Qiang Zhou. 2020. Does your code need comment? Software: Practice and Experience 50, 3 (2020), 227–245. [38] Reza Iranzad and Xiao Liu. 2025. A review of random forest-based feature selection methods for data science education and applications. International Journal of Data Science and Analytics 20, 2 (2025), 197–211. [39] Ciera Jaspan and Collin Green. 2023. Defining, measuring, and managing technical debt. IEEE Software 40, 03 (2023), 15–19. [40] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM computing surveys 55, 12 (2023), 1–38. [41] Can Jin, Hongwu Peng, Shiyu Zhao, Zhenting Wang, Wujiang Xu, Ligong Han, Jiahui Zhao, Kai Zhong, Sanguthevar Rajasekaran, and Dimitris N Metaxas. 2025. Apeer: Automatic prompt engineering enhances large language model reranking. In Companion Proceedings of the ACM on Web Conference 2025. 2494– 2502. [42] Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huaming Chen. 2024. From llms to llm-based agents for software engineering: A survey of current, challenges and future. arXiv preprint arXiv:2408.02479 (2024). [43] John Johnson, Sergio Lubo, Nishitha Yedla, Jairo Aponte, and Bonita Sharif. 2019. An empirical study assessing source code readability in comprehension. In 2019 IEEE International conference on software maintenance and evolution (ICSME). IEEE, 513–523. [44] Sungmin Kang, Louis Milliken, and Shin Yoo. 2024. Identifying inaccurate descriptions in llm-generated code comments via test execution. arXiv preprint arXiv:2406.14836 (2024). [45] Ron Kohavi et al. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, Vol. 14. Montreal, Canada, 1137–1145. [46] Dawn Lawrie, Henry Feild, and David Binkley. 2006. Syntactic identifier conciseness and consistency. In 2006 Sixth IEEE International Workshop on Source Code Analysis and Manipulation. IEEE, 139–148.

Ye et al.

[47] Dawn Lawrie, Christopher Morrell, Henry Feild, and David Binkley. 2006. What’s in a Name? A Study of Identifiers. In 14th IEEE international conference on program comprehension (ICPC’06). IEEE, 3–12. [48] Dawn Lawrie, Christopher Morrell, Henry Feild, and David Binkley. 2007. Effective identifier names for comprehension and memory. Innovations in Systems and Software Engineering 3, 4 (2007), 303–318. [49] Valentina Lenarduzzi, Terese Besker, Davide Taibi, Antonio Martini, and Francesca Arcelli Fontana. 2021. A systematic literature review on technical debt prioritization: Strategies, processes, factors, and tools. Journal of Systems and Software 171 (2021), 110827. [50] Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161 (2023). [51] Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. GPT detectors are biased against non-native English writers. Patterns 4, 7 (2023). [52] Sherlock A Licorish, Ansh Bajpai, Chetan Arora, Fanyu Wang, and Kla Tantithamthavorn. 2025. Comparing Human and LLM Generated Code: The Jury is Still Out! arXiv preprint arXiv:2501.16857 (2025). [53] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2023), 21558–21572. [54] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM computing surveys 55, 9 (2023), 1–35. [55] Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664 (2021). [56] Michael R Lyu, Baishakhi Ray, Abhik Roychoudhury, Shin Hwei Tan, and Patanamon Thongtanunam. 2024. Automatic programming: Large language models and beyond. ACM Transactions on Software Engineering and Methodology (2024). [57] Yuxing Ma, Chris Bogart, Sadika Amreen, Russell Zaretzki, and Audris Mockus. 2019. World of code: an infrastructure for mining the universe of open source VCS data. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 143–154. [58] Yuxing Ma, Tapajit Dey, Chris Bogart, Sadika Amreen, Marat Valiev, Adam Tutko, David Kennard, Russell Zaretzki, and Audris Mockus. 2021. World of code: enabling a research workflow for mining and analyzing the universe of open source VCS data. Empirical Software Engineering 26 (2021), 1–42. [59] Ggaliwango Marvin, Nakayiza Hellen, Daudi Jjingo, and Joyce NakatumbaNabende. 2023. Prompt engineering in large language models. In International conference on data intelligence and cognitive informatics. Springer, 387–402. [60] Orni Meerbaum-Salant, Michal Armoni, and Mordechai Ben-Ari. 2011. Habits of programming in scratch. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education. 168–172. [61] Prabhaker Mishra, Uttam Singh, Chandra M Pandey, Priyadarshni Mishra, and Gaurav Pandey. 2019. Application of student’s t-test, analysis of variance, and covariance. Annals of cardiac anaesthesia 22, 4 (2019), 407–411. [62] Leon Moonen. 2001. Generating robust parsers using island grammars. In Proceedings eighth working conference on reverse engineering. IEEE, 13–22. [63] Delano Oliveira, Reydne Santos, Benedito De Oliveira, Martin Monperrus, Fernando Castor, and Fernanda Madeiral. 2024. Understanding Code Understandability Improvements in Code Reviews. IEEE Transactions on Software Engineering (2024). [64] OpenAI. [n. d.]. ChatGPT release. https://openai.com/index/chatgpt/. [65] Andy Oram and Greg Wilson. 2007. Beautiful code Leading programmers explain how they think. O’Reilly Media, Inc. [66] Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrea De Lucia. 2014. Do they really smell bad? a study on developers’ perception of bad code smells. In 2014 IEEE International conference on software maintenance and evolution. IEEE, 101–110. [67] Dangfeng Pan, Zhensu Sun, Cenyuan Zhang, David Lo, and Xiaoning Du. 2025. The Hidden Cost of Readability: How Code Formatting Silently Consumes Your LLM Budget. arXiv preprint arXiv:2508.13666 (2025). [68] Sebastiano Panichella, Venera Arnaoudova, Massimiliano Di Penta, and Giuliano Antoniol. 2015. Would static analysis tools help developers with code reviews?. In 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 161–170. [69] Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. Advances in neural information processing systems 34 (2021), 11054–11070. [70] Felix Petersen, Debarghya Mukherjee, Yuekai Sun, and Mikhail Yurochkin. 2021. Post-processing for individual fairness. Advances in Neural Information

Processing Systems 34 (2021), 25944–25955. [71] Valentina Piantadosi, Fabiana Fierro, Simone Scalabrino, Alexander Serebrenik, and Rocco Oliveto. 2020. How does code readability change during software evolution? Empirical Software Engineering 25 (2020), 5374–5412. [72] Daryl Posnett, Abram Hindle, and Premkumar Devanbu. 2011. A simpler model of software readability. In Proceedings of the 8th working conference on mining software repositories. 73–82. [73] Vaclav Rajlich and Prashant Gosavi. 2004. Incremental change in object-oriented programming. IEEE software 21, 4 (2004), 62–69. [74] David Ramel. 2023. GitHub Copilot AI Tech Upgraded, Already Generates 61% of Java Code. https://visualstudiomagazine.com/articles/2023/02/15/copilotupgrade.aspx. [75] Vipula Rawte, Amit Sheth, and Amitava Das. 2023. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922 (2023). [76] Rodrigo REBOUÇAS DE ALMEIDA, Christoph TREUDE, and Uirá KULESZA. 2023. What’s behind tight deadlines? Business causes of technical debt.(2023). In Proceedings of the 16th International Conference on Cooperative and Human Aspects of Software Engineering, Melbourne, Australia. 14–15. [77] Simone Scalabrino, Gabriele Bavota, Christopher Vendome, Mario LinaresVásquez, Denys Poshyvanyk, and Rocco Oliveto. 2017. Automatically assessing code understandability: How far are we?. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 417–427. [78] Simone Scalabrino, Mario Linares-Vásquez, Rocco Oliveto, and Denys Poshyvanyk. 2018. A comprehensive model for code readability. Journal of Software: Evolution and Process 30, 6 (2018), e1958. [79] Simone Scalabrino, Mario Linares-Vasquez, Denys Poshyvanyk, and Rocco Oliveto. 2016. Improving code readability models with textual features. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC). IEEE, 1–10. [80] Mor Shamy and Dror G Feitelson. 2023. Identifying lines and interpreting vertical jumps in eye tracking studies of reading text and code. ACM Transactions on Applied Perception 20, 2 (2023), 1–20. [81] Claude E Shannon. 1948. A mathematical theory of communication. The Bell system technical journal 27, 3 (1948), 379–423. [82] Andreas Stefik and Susanna Siebert. 2013. An empirical investigation into programming language syntax. ACM Transactions on Computing Education (TOCE) 13, 4 (2013), 1–40. [83] Wannita Takerngsaksiri, Chakkrit Tantithamthavorn, Micheal Fu, Jirat Pasuksmit, Kun Chen, and Ming Wu. 2025. Code readability in the age of large language models: An industrial case study from atlassian. In 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 732–742. [84] Gareth Terry, Nikki Hayfield, Victoria Clarke, Virginia Braun, et al. 2017. Thematic analysis. The SAGE handbook of qualitative research in psychology 2, 17-37 (2017), 25. [85] Edith Tom, Aybüke Aurum, and Richard Vidgen. 2013. An exploration of technical debt. Journal of Systems and Software 86, 6 (2013), 1498–1516. [86] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023). [87] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). [88] Antonio Vitale, Emanuela Guglielmi, Rocco Oliveto, and Simone Scalabrino. 2025. Personalized Code Readability Assessment: Are We There Yet? arXiv preprint arXiv:2503.07870 (2025). [89] Chong Wang, Kaifeng Huang, Jian Zhang, Yebo Feng, Lyuye Zhang, Yang Liu, and Xin Peng. 2024. Llms meet library evolution: Evaluating deprecated api usage in llm-based code completion. arXiv preprint arXiv:2406.09834 (2024). [90] Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. 2023. How far can camels go? exploring the state of instruction tuning on open resources. In Proceedings of the 37th International Conference on Neural Information Processing Systems. 74764–74786. [91] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021). [92] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837. [93] J White. 2023. A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv preprint arXiv:2302.11382 (2023). [94] A Wayne Whitney. 1971. A direct method of nonparametric measurement selection. IEEE transactions on computers 100, 9 (1971), 1100–1103. [95] Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics bulletin 1, 6 (1945), 80–83.

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

[96] Michael Williams and Tami Moser. 2019. The art of coding and thematic exploration in qualitative research. International management review 15, 1 (2019), 45–55. [97] Scott N Woodfield, Hubert E Dunsmore, and Vincent Y Shen. 1981. The effect of modularization and comments on program comprehension. In Proceedings of the 5th international conference on Software engineering. 215–223. [98] xAI. [n. d.]. Grok. https://grok.com/. [99] Weiwei Xu, Kai Gao, Hao He, and Minghui Zhou. 2025. Licoeval: Evaluating llms on license compliance in code generation. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 1665–1677. [100] Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing (2024), 100211. [101] Xiang Zhang, Juntai Cao, Chenyu You, and Dujian Ding. 2025. Why prompt design matters and works: A complexity analysis of prompt search space in llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 32525–32555. [102] Li Zhong and Zilong Wang. 2024. Can llm replace stack overflow? a study on robustness and reliability of large language model code generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 21841–21849. [103] Yangtian Zi, Luisa Li, Arjun Guha, Carolyn Anderson, and Molly Q Feldman. 2025. “I Would Have Written My Code Differently’: Beginners Struggle to Understand LLM-Generated Code. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 1479–1488.

Ye et al.

A

Table A.2: Notation Summary

Readability Model Table A.1: Construction of the Readability Model

DIMENSION

FEATURE

TF

CIC (Comment-Identifier Consistency) CIC_syn (Synonym-expanded CIC) ITID (Identifier-Term in Dictionary) NMI (Narrow Meaning Identifiers) CR (Comment Readability) NM (Number of meanings) TC (Text Coherence) NOC (Number of Concepts) NOC_norm (Normalized NOC)

MIN

✓ ✓

✓

AVG ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Line length (characters) Identifier length (characters) N. of identifiers Indentation (preceding whitespace) N. of keywords N. of numbers N. of parentheses N. of brackets N. of periods N. of blank lines N. of comments N. of commas N. of spaces N. of assignments N. of branches (if) N. of loops (for, while) N. of arithmetic operators N. of comparison operators N. of occurrences of any character N. of occurrences of any identifier

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

PF

LOC (Lines of Code) Entropy Halstead Volume

✓ ✓ ✓

DF

Visual keyword density Visual string density Visual comment density Visual complexity Spatial absolute area Spatial relative area Spatial regularity Spatial density Alignment operators Alignment brackets Alignment consistency Text English ratio Text comment ratio Text vocabulary size Text avg identifier length

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

BWF

A.1

MAX

Description Code snippet with 𝑚 lines; 𝑙𝑖 is the 𝑖th line Total, non-empty, code, and valid line counts Token collections with multiplicity for lexical tokens, identifiers, and comments Normalized term collections extracted from comments and identifiers Identifier-term group for local unit 𝑟 ; WordNet dictionary Character vocabularies (snippet / line) and identifier vocabulary Numbers of total, keyword, string, and comment characters Numbers of words, sentences, and syllables in comment text Local textual block; cluster set from DBSCAN Column positions of assignment operators and brackets Raw feature vector and normalized feature value Feature statistics (mean/std) and Logistic Regression parameters Logistic sigmoid function Aggregation operators

STD

𝑚, 𝑚 ne , 𝑚 code , 𝑚 val 𝑇 (𝑠 ), 𝐼 (𝑠 ), 𝐶 (𝑠 )

✓ ✓ ✓ ✓

𝑇𝑐 (𝑠 ),𝑇𝑖 (𝑠 ) 𝐺𝑟 , 𝐷

✓ ✓ ✓ ✓ ✓ ✓

Vchar (𝑠 ), Vchar (𝑙𝑖 ), Vid (𝑠 ) 𝑁 char , 𝑁 kw , 𝑁 str , 𝑁 com 𝑊 , 𝑆, 𝑌 𝑏𝑖 , C (𝑠 ) 𝑃 asg , 𝑃br x ∈ R61 , 𝑥 ′𝑗 𝜇 𝑗 , 𝜎 𝑗 , w, 𝑏

✓ ✓

𝜎 (·) ¤ std( ), ¤ min(·), max(·) avg( ), ✓

𝑚 val denote the numbers of non-empty lines, non-empty code lines, and valid extracted lines used by the corresponding feature module, respectively. For a finite collection of local values Z = {𝑧 1, . . . , 𝑧𝑛 }, we use the standard aggregation operators 𝑛

1 ∑︁ 𝑧𝑖 , 𝑛 𝑖=1 v t 𝑛
2 1 ∑︁ std(Z) = 𝑧𝑖 − avg(Z) , 𝑛 𝑖=1

avg(Z) =

(A.2)

min(Z) = min 𝑧𝑖 ,

Formal definitions of all metrics

1≤𝑖 ≤𝑛

This section outlines the implementation-aligned definitions of the metrics used in our readability model. Since several feature families are instantiated through lightweight heuristics rather than full parsing, the formulas below follow the actual computation rules of our implementation. In addition, detailed symbols used throughout the appendix are summarized in Table A.2. Let a code snippet be denoted by 𝑠 = {𝑙 1, 𝑙 2, . . . , 𝑙𝑚 },

Symbol 𝑠 = {𝑙 1 , . . . , 𝑙𝑚 }

(A.1)

where 𝑙𝑖 is the 𝑖-th line and 𝑚 is the total number of lines. Let 𝑇 (𝑠), 𝐼 (𝑠), and 𝐶 (𝑠) denote the lexical-token collection, identifiertoken collection, and comment-token collection extracted from 𝑠, respectively. Here, |𝐼 (𝑠)| counts multiplicity. Let 𝑚 ne , 𝑚 code , and

max(Z) = max 𝑧𝑖 . 1≤𝑖 ≤𝑛

Unless otherwise specified, undefined quantities caused by missing evidence are assigned the default value 0 in the implementation. A.1.1 TF. TF captures lexical quality, semantic precision, comment– code consistency, and conceptual coherence. CIC and CIC_syn. Let 𝑇𝑐 (𝑠) be the set of normalized terms extracted from all comments in 𝑠, and let 𝑇𝑖 (𝑠) be the set of normalized identifier terms extracted from the entire snippet. Then CIC(𝑠) =

|𝑇𝑐 (𝑠) ∩ 𝑇𝑖 (𝑠)| . |𝑇𝑐 (𝑠) ∪ 𝑇𝑖 (𝑠)|

(A.3)

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

For synonym-expanded matching, let 𝑆 syn (𝐴) denote the union of WordNet synonym terms associated with term set 𝐴. The implementation expands both sides: 𝑇e𝑐 (𝑠) = 𝑇𝑐 (𝑠) ∪ 𝑆 syn (𝑇𝑐 (𝑠)), 𝑇e𝑖 (𝑠) = 𝑇𝑖 (𝑠) ∪ 𝑆 syn (𝑇𝑖 (𝑠)), CICsyn (𝑠) =

(A.4)

ITID, NMI, and NM. Let 𝐺𝑟 ⊆ 𝐼 (𝑠) denote the identifier-term group of local unit 𝑟 . Then (𝑟 )

𝑡 ∈𝐺𝑟

NM (𝑟 ) =

(A.5)

∑︁ 1 senses(𝑡), max(1, |𝐺𝑟 |) 𝑡 ∈𝐺

𝑌 (𝑠) 𝑊 (𝑠) − 84.6 . CR(𝑠) = 206.835 − 1.015 (A.6) 𝑆 (𝑠) 𝑊 (𝑠) TC. Let {𝑏 1, . . . , 𝑏𝑞 } be the local textual blocks extracted from the snippet, where each block is a contiguous group of lines produced by the block-extraction heuristic in the implementation. For any pair (𝑏𝑖 , 𝑏 𝑗 ) with 1 ≤ 𝑖 < 𝑗 ≤ 𝑞,

indent(𝑙),

𝑙 ∈𝐿code (𝑠 )

(A.10)

max indent(𝑙),

𝑙 ∈𝐿code (𝑠 )

and average per-line counts of lexical categories, ∑︁ 1 Count 𝜒 (𝑠) = # 𝜒 (𝑙), max(1, 𝑚 code )

(A.11)

𝑙 ∈𝐿code (𝑠 )

where # 𝜒 (𝑙) is the implementation-specific count of category 𝜒 on line 𝑙. For identifiers, let 𝐼 (𝑠) be the extracted identifier collection, where |𝐼 (𝑠)| counts multiplicity. Then ∑︁ 1 |𝑡 |, IdLenavg (𝑠) = max(1, |𝐼 (𝑠)|) 𝑡 ∈𝐼 (𝑠 )

CharPerLinemax (𝑠) = IdPerLinemax (𝑠) =

A.1.2 BWF. BWF consists of lightweight formatting and low-level structural statistics computed from lines, identifiers, and simple lexical counts. These features are implementation-aligned scalar statistics rather than outputs of a single unified counting formalism. Let 𝐿code (𝑠) denote the set of non-empty code lines in 𝑠, with |𝐿code (𝑠)| = 𝑚 code . Representative features include average and maximum line length, ∑︁ 1 LineLenavg (𝑠) = |𝑙 |, max(1, 𝑚 code ) 𝑙 ∈𝐿code (𝑠 ) (A.9) LineLenmax (𝑠) = max |𝑙 |,

(A.12)

max

|𝑙 |,

max

|𝐼 (𝑙)|.

𝑙 ∈𝐿code (𝑠 ) 𝑙 ∈𝐿code (𝑠 )

Some BWF features are ratios, such as 𝑚 blank (𝑠) , max(1, 𝑚) 𝑚 comment (𝑠) CommentLineRatio(𝑠) = , max(1, 𝑚) BlankLineRatio(𝑠) =

(A.13)

where 𝑚 blank (𝑠) and 𝑚 comment (𝑠) denote the numbers of blank lines and detected comment lines or comment segments. A.1.3 PF. PF captures size-related and information-theoretic properties. LOC. LOC is the number of non-empty lines:

(A.7)

Snippet-level TC features are obtained by applying min(·), avg(·), and max(·) over all pairwise values {TC (𝑖,𝑗 ) }. NOC and NOC_norm. Each valid extracted line is represented as a token-set vector and clustered by DBSCAN. Let C(𝑠) be the set of non-noise clusters. Then NOC(𝑠) = |C(𝑠)|, (A.8) |C(𝑠)| NOCnorm (𝑠) = , 𝑚 val where 𝑚 val is the number of valid extracted lines used in clustering.

𝑙 ∈𝐿code (𝑠 )

∑︁

IdLenmax (𝑠) = max |𝑡 |,

𝑟

|𝑇 (𝑏𝑖 ) ∩ 𝑇 (𝑏 𝑗 )| . |𝑇 (𝑏𝑖 ) ∪ 𝑇 (𝑏 𝑗 )|

1 max(1, 𝑚 code )

𝑡 ∈𝐼 (𝑠 )

where 𝐷 is the WordNet dictionary, depth(𝑡) is the maximum hypernym depth of token 𝑡, and senses(𝑡) is the number of WordNet senses of 𝑡. Snippet-level features are obtained by aggregating the local values across all valid local units. In particular, ITID and NMI use min(·), avg(·), and max(·), while NM uses avg(·) and max(·). CR. The implementation concatenates all comment text in 𝑠 into a single sequence. Let 𝑊 (𝑠), 𝑆 (𝑠), and 𝑌 (𝑠) denote the resulting numbers of words, sentences, and syllables. Then

TC (𝑖,𝑗 ) =

Indentavg (𝑠) = Indentmax (𝑠) =

|𝑇e𝑐 (𝑠) ∩ 𝑇e𝑖 (𝑠)| . |𝑇e𝑐 (𝑠) ∪ 𝑇e𝑖 (𝑠)|

|{𝑡 ∈ 𝐺𝑟 : 𝑡 ∈ 𝐷 }| ITID = , max(1, |𝐺𝑟 |) ∑︁ NMI (𝑟 ) = depth(𝑡),

average and maximum indentation,

LOC(𝑠) =

𝑚 ∑︁

1[𝑙𝑖 ≠ ∅].

(A.14)

𝑖=1

Entropy. For line 𝑙𝑖 , let 𝑝𝑖 (𝑐) be the empirical probability of character 𝑐 among all characters on that line. The implementation uses character-level entropy: ∑︁ 𝐻 (𝑙𝑖 ) = − 𝑝𝑖 (𝑐) log2 𝑝𝑖 (𝑐), (A.15) 𝑐 ∈ Vchar (𝑙𝑖 )

where Vchar (𝑙𝑖 ) is the set of distinct characters on line 𝑙𝑖 . At the snippet level,
Entropyavg (𝑠) = avg {𝐻 (𝑙𝑖 ) : 𝑙𝑖 ≠ ∅} , (A.16)
Entropystd (𝑠) = std {𝐻 (𝑙𝑖 ) : 𝑙𝑖 ≠ ∅} . We further define Vchar (𝑠) =

𝑚 Ø

Vchar (𝑙𝑖 ).

(A.17)

𝑖=1

Halstead Volume. Let 𝜂 1, 𝜂 2 be the numbers of distinct operators and operands, and let 𝑁 1, 𝑁 2 be their total occurrences. With 𝜂 = 𝜂 1 + 𝜂 2 and 𝑁 = 𝑁 1 + 𝑁 2 , HV(𝑠) = 𝑁 log2 𝜂.

(A.18)

Ye et al.

A.1.4 DF. DF models the snippet as a visual object using lightweight display-oriented approximations. These features should be interpreted as heuristic proxies, not exact geometric measures. Let 𝑚 ∑︁ 𝑁 char (𝑠) = |𝑙𝑖 |, (A.19) 𝑖=1

and let 𝑁 kw (𝑠), 𝑁 str (𝑠), and 𝑁 com (𝑠) denote the numbers of keyword, string-literal, and comment characters identified by the implementation. Then 𝑁 kw (𝑠) VKD(𝑠) = , max(1, 𝑁 char (𝑠)) 𝑁 str (𝑠) VSD(𝑠) = , (A.20) max(1, 𝑁 char (𝑠)) 𝑁 com (𝑠) VCD(𝑠) = . max(1, 𝑁 char (𝑠)) Visual complexity is approximated by the normalized size of the character set: |Vchar (𝑠)| (A.21) VC(𝑠) = . 95 Let 𝑊max (𝑠) = max |𝑙𝑖 |. (A.22) 1≤𝑖 ≤𝑚 Then the spatial features are SAA(𝑠) = 𝑁 char (𝑠), 𝑁 char (𝑠) SRA(𝑠) = , max(1, 𝑚 · 𝑊max (𝑠)) 𝑚 ne SD(𝑠) = , max(1, 𝑚) std({|𝑙𝑖 |}𝑚 𝑖=1 ) SR(𝑠) = 1 − . max(1, avg({|𝑙𝑖 |}𝑚 𝑖=1 ))

A.2

Extraction rules and preprocessing details

We apply a unified snippet-level workflow to all samples. To preserve structural and visual signals, preprocessing is limited to lightweight normalization, including line-ending standardization, removal of obvious encoding artifacts, and trimming of superfluous leading and trailing blank lines, while preserving internal whitespace, indentation, blank lines, and comment placement. After preprocessing, each snippet is processed with lightweight language-aware extraction rules based on lexical scanning and regular-expression matching. We identify identifiers, comments, keywords, operators, delimiters, literals, and line boundaries without relying on full parsing, which keeps the pipeline applicable to incomplete or noisy snippets. WordNet is used for synonym expansion, sense counting, and semantic-depth estimation. Identifier tokens are normalized by lowercasing and decomposing naming conventions such as camelCase and snake_case. For a token not found in WordNet, its synonym set is empty, with sense- and depth-based quantities set to zero. For NOC estimation, each valid line is represented as a token-set vector. For two lines 𝑙𝑖 and 𝑙 𝑗 , the token-overlap distance is defined as |𝑇 (𝑙𝑖 ) ∩ 𝑇 (𝑙 𝑗 )| . 𝛿 (𝑙𝑖 , 𝑙 𝑗 ) = 1 − (A.28) |𝑇 (𝑙𝑖 ) ∪ 𝑇 (𝑙 𝑗 )| DBSCAN is then applied to obtain the number of non-noise clusters. After extraction, all quantities are represented as snippet-level scalar features and concatenated in a fixed order:

(A.23)

Let 𝑃 asg (𝑠) and 𝑃br (𝑠) denote the collections of column positions of assignment operators and bracket symbols, respectively. We use the convention std(𝑃) = 0 if |𝑃 | ≤ 1. Then 1 AlignOp(𝑠) = , 1 + std(𝑃 asg (𝑠)) 1 (A.24) AlignBr(𝑠) = , 1 + std(𝑃 br (𝑠)) AlignOp(𝑠) + AlignBr(𝑠) AlignCons(𝑠) = . 2 Let 𝐼 user (𝑠) denote the extracted user-defined identifier collection, and let 𝐼 eng (𝑠) ⊆ 𝐼 user (𝑠) denote the subset whose decomposed terms match English words. Then |𝐼 eng (𝑠)| (A.25) TextEnglish(𝑠) = . max(1, |𝐼 user (𝑠)|) Let 𝑊com (𝑠) be the number of comment words and 𝑊code (𝑠) the number of identifier-derived code words. Then 𝑊com (𝑠) TextComment(𝑠) = . (A.26) max(1,𝑊com (𝑠) + 𝑊code (𝑠)) The remaining text-based display measures are TextVocab(𝑠) = |Vid (𝑠)|, ∑︁ 1 (A.27) TextIdLen(𝑠) = |𝑡 |, max(1, |𝐼 user (𝑠)|) 𝑡 ∈𝐼 user (𝑠 )

where Vid (𝑠) is the set of unique extracted user-defined identifiers.

x = [TF, BWF, PF, DF] ∈ R61 .

(A.29)

If a metric is undefined because the required evidence is absent, we assign a default value of zero to keep the feature dimension fixed. Once the raw feature vector x is extracted, we apply feature-wise z-score normalization: 𝑥𝑗 − 𝜇𝑗 𝑥 ′𝑗 = , (A.30) 𝜎𝑗 where 𝜇 𝑗 and 𝜎 𝑗 are the empirical mean and standard deviation estimated by the scaler used in the training pipeline. The feature weights are obtained from the coefficient vector w of a Logistic Regression model fitted on the training-set features after StandardScaler normalization and SFS-based subset selection, and are used as weights to compute the readability score ∑︁ 𝑥 ′𝑗 𝑤 𝑗 . (A.31) 𝑗

In addition, we formulate readability assessment as a supervised learning task using Logistic Regression. Given the normalized feature vector x′ , the model predicts 𝑝 (𝑦 = 1 | x′ ) = 𝜎 (w⊤ x′ + 𝑏) =

1
, 1 + exp − (w⊤ x′ + 𝑏)

(A.32) where w is the coefficient vector, 𝑏 is the intercept, and 𝜎 (·) is the logistic sigmoid. We use L2 regularization. To reduce redundancy, we further apply Sequential Forward Selection (SFS). Let F denote the full candidate feature set and S𝑡 the selected subset after step 𝑡. The procedure starts from S0 = ∅.

(A.33)

The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

At step 𝑡 + 1, we add the feature maximizing the model-selection criterion: 𝑓 ★ = arg max M (S𝑡 ∪ {𝑓 }), 𝑓 ∈ F\S𝑡 (A.34) S𝑡 +1 = S𝑡 ∪ {𝑓 ★ }.

Moreover, at inference time, the optimized model outputs 𝑟 (𝑠) = 𝑝 (𝑦 = 1 | x′ (𝑠)) = 𝜎 (w⊤ x′ (𝑠) + 𝑏),

(A.35)

which is used as the final readability prediction. In addition to the full-model prediction rate, we also compute family-specific prediction rate by fitting separate Logistic Regression models on TF, BWF, PF, and DF.

Related documents

• Two's a Crowd: Human and AI-Based Copresence for Developers with ADHD#1006931
• License Compliance in Open Source Cybersecurity Projects#1006932
• Embedding Drift in Code Vulnerability Models Under Intended Behaviour-Preserving Transformations#1006933
• SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?#1006934
• Helpful but Fallible: Developer Experiences of AI Tools Under a Coordinated Industrial Roll-out#1006935