arXiv:2604.13725v1 [cs.SE] 15 Apr 2026
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation Jia Feng
Zhanyue Qin
Cuiyun Gao∗
Harbin Institute of Technology Shenzhen, China [email protected]
Harbin Institute of Technology Shenzhen, China [email protected]
Harbin Institute of Technology Shenzhen, China [email protected]
Ruiqi Wang
Chaozheng Wang
Yingwei Ma
Harbin Institute of Technology Shenzhen, China [email protected]
The Chinese University of Hong Kong Hong Kong, China [email protected]
Independent Researcher Beijing, China [email protected]
Xiaoyuan Xie Wuhan University Wuhan, China [email protected]
Abstract
1
Repository-level code intelligence tasks, such as cross-file completion and project-aware code generation, require large language models (LLMs) to process long, multi-file contexts spanning complex dependencies. Such lengthy inputs introduce three practical challenges: crucial context can be obscured by surrounding noise across multiple files, or be truncated due to the limited context windows, and inference latency and memory consumption grow rapidly with context length. Context compression mitigates these risks by condensing long inputs into compact surrogates. While it has received considerable attention in the natural language processing literature, its applicability to code tasks remains largely unexplored; beyond efficiency gains, compression may further benefit LLM performance on code tasks by reducing contextual noise inherent in large, multi-file repositories. Therefore, we present the first systematic empirical study of context compression for repository-level code intelligence, organizing eight representative methods into three paradigms by their output representation space: discrete token sequences, continuous latent vectors, and visual tokens. We evaluate them on code completion and generation, two of the most widely studied tasks in repositorylevel code intelligence, jointly measuring task performance and deployment efficiency. Our results demonstrate that context compression is effective for code: at 4× compression, methods based on continuous latent vectors surpass full-context performance by up to 28.3% with respect to the BLEU score, indicating that the latent vector compression filters repository noise rather than merely truncating context. On the efficiency side, all three paradigms reduce inference cost relative to full-context decoding. In particular, both visual and text-based compression achieve up to 50% reduction in end-to-end latency at high compression ratios, approaching the cost of inference without repository context. These findings establish context compression as a viable approach for repository-level code intelligence and provide guidance for paradigm selection under different deployment constraints.
Large language models (LLMs) have revolutionized many software engineering tasks, including code generation, program repair, code summarization, and automated testing [6, 18, 27, 30, 38]. State-ofthe-art code LLMs such as DeepSeek-Coder [13] and Qwen2.5Coder [15] are increasingly applied to repository-level tasks including cross-file completion, API-aware generation, and codebase question answering, which require reasoning over long, heterogeneous contexts spanning API definitions, dependency files, and historical implementations. Processing such inputs introduces three practical bottlenecks: inference cost and latency grow rapidly with context length due to the quadratic complexity of self-attention [14, 31, 33]; task-critical evidence is frequently obscured by surrounding noise, a phenomenon known as “lost in the middle” [23]; and finite context limits force truncation when inputs span large codebases or cross-module dependencies. These challenges are particularly acute for code, where semantic dependencies are dense and long-range, syntactic constraints are strict, and the information-to-noise ratio is considerably lower than in natural language. Context compression attempts to address these bottlenecks by transforming raw context into a compact surrogate representation. Existing methods can be categorized into three paradigms by the representation space of their outputs. Text-to-Text (T2T) methods produce a shorter, discrete token sequence via token-level filtering or rewriting; representative approaches include LLMLingua [16], LLMLingua-2 [26], and LongLLMLingua [17]. Text-to-Vector (T2V) methods encode context into a compact set of continuous latent tokens; ICAE [10] distills long contexts into fixed-size memory slots, while gist-token approaches [24] and 500xCompressor [21] extend this paradigm to more aggressive compression ratios. Textto-Image (T2I) methods render context as images consumed by a vision-language model, exploiting the higher information density of the visual modality; optical encoding has been shown to achieve up to 10× token reduction while maintaining high reconstruction fidelity [2, 36, 37], with extensions to diverse long-context settings [8].
∗ Corresponding author
Introduction
Conference’17, July 2017, Washington, DC, USA
Despite this growing body of work, context compression has been studied almost exclusively on NLP tasks in method-specific settings. Whether and how these techniques transfer to code LLMs remains an open question: repository-level code imposes constraints absent in natural language, including long-range cross-file dependencies, strict syntactic structure, and a lower information-to-noise ratio that amplifies the risk of aggressive filtering. We address this gap by presenting, to the best of our knowledge, the first systematic empirical study of context compression for repository-level code LLMs, instantiating eight representative methods across all three paradigms and studying them on code generation and completion, two of the most widely studied repository-level tasks, jointly measuring task performance and deployment efficiency across multiple model scales. We study the following research questions (RQs) and summarize the findings: RQ1: How do different compression paradigms affect task performance? Context compression not only preserves downstream task performance but can actively improve it over fullcontext inference. T2V surpasses the full-context baseline by up to 28.3% in terms of BLEU on Python completion. We attribute this to the latent vectors acting as a learned filter that suppresses repository boilerplate and reinforces task-relevant signals. T2I tracks fullcontext performance closely on completion tasks and sustains competitive performance at moderate compression ratios. T2T provides training-free compression that is effective at mild ratios, though performance gaps widen on generation tasks where cross-file relational structure is critical. RQ2: How does compression ratio affect task performance? T2V sustains its performance advantage across the full evaluated range of 4× to 128×, with all variants exceeding full-context performance on generation at every ratio, demonstrating that effective compression does not require conservative token budgets. T2I peaks at 4× on completion and degrades gradually with further downscaling, yet its generation performance remains consistently below full context regardless of ratio, as uniform rendering loses cross-file relational structure at any resolution. T2T degrades most sharply: generation performance converges to the no-context baseline at approximately 7× to 12×, establishing a practical upper bound for perplexity-based pruning on code tasks. RQ3: How do compression paradigms and ratios affect throughput and resource usage? All three paradigms deliver measurable efficiency gains over full-context inference, with decoding cost reduction as the primary driver across paradigms. T2I achieves a 33% reduction in end-to-end latency at its performanceoptimal 4× ratio, with resource costs converging to the context-free baseline at higher compression ratios. T2V maintains a stable and predictable overhead profile across all evaluated ratios, remaining well below full-context cost throughout. T2T reduces total inference latency by over 35% relative to full-context inference at moderate compression ratios, with decoding cost and GPU memory converging toward the resource cost of inference without repository context at higher ratios. Collectively, these results demonstrate that context compression is not only viable but, under appropriate paradigm selection, performance-enhancing for repository-level code intelligence, yielding both performance improvements and substantial efficiency
Trovato et al.
gains that make it a practically compelling approach for code LLM deployment. In summary, this paper makes the following contributions: • We introduce context compression to repository-level code intelligence through the first systematic empirical study spanning three paradigms (T2T, T2V, T2I) and eight representative methods, evaluated on code generation and completion across multiple model scales. • We reveal that latent-vector compression can surpass fullcontext performance on code tasks, establishing that compression need not be a lossy process and demonstrating paradigm-specific performance-ratio scaling behavior previously unstudied in the code domain. • We provide a deployment-oriented efficiency analysis and distill actionable guidance for selecting compression strategies based on task type, compression budget, and resource constraints. The remainder of this paper reviews related work (Section 2), presents the unified evaluation framework (Section 3), describes the experimental setup (Section 4), reports and analyzes results (Section 5), discusses implications and threats to validity (Section 6), and concludes with future directions (Section 7).
2 Related Work 2.1 Code Large Language Models Code intelligence has evolved from encoder and encoder-decoder pre-trained models such as CodeBERT [9], GraphCodeBERT [12], CodeT5 [35], and UniXcoder [11] to large decoder-only code LLMs including Codex [4], StarCoder [5], CodeLlama [29], DeepSeekCoder [13], and Qwen2.5-Coder [15], with progressively stronger capabilities in generation, infilling, and multi-language reasoning. General-purpose frontier models such as GPT-4 [25], Gemini [32], and DeepSeek [3] further achieve competitive coding performance through large-scale pretraining and instruction alignment. The focus of code LLM application has shifted from functionlevel benchmarks to repository-level tasks including cross-file completion, project-aware bug fixing, and codebase question answering, requiring models to reason over heterogeneous, long-form contexts spanning multiple files and dependencies. Performance degrades when task-relevant signals are sparse and distributed within lengthy inputs, motivating research on context management strategies beyond model scaling. Our work investigates context compression as a deployment-oriented solution to this bottleneck, studying its applicability to repository-level code tasks.
2.2
Context Compression
Context compression reduces prompt length while preserving taskcritical information, thus lowering inference latency, memory usage, and API cost. We categorize existing approaches based on their distinct representation spaces: Text-to-Text (T2T). T2T methods produce a shorter discrete token sequence through filtering, rewriting, or reordering without modifying the downstream model. Selective Context [20] removes lowinformation phrases using lexical self-information. LLMLingua [16] introduces a budget controller and small-model-guided iterative
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
token pruning with distribution alignment. LongLLMLingua [17] adds a question-aware coarse-to-fine strategy with document reordering to mitigate “lost in the middle” effects. LLMLingua-2 [26] reformulates compression as token classification via a lightweight bidirectional encoder trained through data distillation, substantially reducing compression latency while preserving cross-domain generalization. T2T approaches are model-agnostic and produce human-readable outputs, though fidelity degrades under aggressive compression as pruning may sever long-range dependencies. Text-to-Vector (T2V). T2V methods encode context into a compact set of continuous latent tokens, enabling the downstream model to condition on a fixed-size surrogate. Gist tokens [24] learn to cache demonstration semantics for reuse at test time. RMT [1] introduces segment-level recurrence with explicit memory tokens, supporting arbitrarily long inputs at bounded per-segment cost. ICAE [10] pretrains a LoRA-based encoder to compress contexts into fixed memory slots processed by a frozen LLM decoder, achieving effective 4× compression. 500xCompressor [21] pushes this to extreme ratios, distilling hundreds of tokens into a single special token. R3 Mem [34] further proposes a reversible compression architecture with hierarchical retention and high-fidelity reconstruction. T2V representations are memory-efficient and well-suited to repeated context reuse, though opaque latent encodings may attenuate exact token-level details critical for code correctness. Text-to-Image (T2I). T2I methods render textual content as images consumed by a vision-language model, exploiting the higher information density of the visual modality. DeepSeek-OCR [36] demonstrates up to 10× token reduction while maintaining 97% reconstruction fidelity at moderate compression ratios. C3 [22] proposes cascading models of different sizes and finds that efficiency gains may stem from latent token density rather than the image modality per se. AgentOCR [8] extends this paradigm to agentic settings with segment optical caching, preserving over 95% of agent performance at more than 50% token reduction. T2I methods offer strong throughput benefits under strict token budgets, though fine-grained lexical signals such as identifiers, indentation, and delimiters may be lost under aggressive downscaling. These paradigms have been studied predominantly on NLP benchmarks in method-specific settings. Their effectiveness for repositorylevel code tasks, where long-range structural dependencies and strict correctness requirements impose notably stricter compression demands, remains unexplored. Our work addresses this gap.
3
Methodology
We organize the diverse compression methods studied in this paper under a common abstraction, illustrated in Figure 1. A compression operator transforms raw long code context into a compact surrogate representation under a resource budget, which is then consumed by a downstream model together with the task instruction. This abstraction lets us characterize methods by their output representation space and construction mechanism, independently of specific model architectures.
3.1
Conference’17, July 2017, Washington, DC, USA
Unified Problem Formulation
Let the input be a token sequence X = [𝑡 1, . . . , 𝑡𝑛 ] ∈ V 𝑛 and a task instruction 𝑞 ∈ Q. A compression operator under budget 𝐵 maps the context to a compact surrogate: C𝐵 : V 𝑛 → Z,
s.t.
Cost(Z) ≤ 𝐵
(1)
where Z is the surrogate representation space and 𝐵 constrains token count, memory, or latency. The downstream prediction is 𝑓 (𝑞, Z), with the query applied at inference time. The compression objective minimizes the output divergence from full-context inference: min E ( X,𝑞)∼D [𝑑 (𝑓 (𝑞, C𝐵 (X)), 𝑓 (𝑞, X))] + 𝜆 · R (C𝐵 (X)), C𝐵
(2)
where 𝑑 (·, ·) measures output divergence and R (·) is the rate penalty. Query-conditioned methods (e.g., retrieval-aware T2T) are a special case C𝐵 (X; 𝑞).
3.2
Text-to-Image (T2I)
The T2I paradigm, shown in the left panel of Figure 1, converts the code context from a token sequence into a visual representation consumed by a vision-language model (VLM). This exploits the observation that the visual modality can carry significantly more information per token than discrete text, enabling aggressive compression without modifying the downstream LLM. When the pipeline acceptsthe input sequence, it is first rendered into a high-resolution raster image via a layout-aware function: I = Render(X; 𝜃 layout ),
(3)
where 𝜃 layout controls text density, font size, and spatial arrangement. The image is then scaled isotropically to a target resolution that determines the visual token budget, with the target area derived from 𝑊 × 𝐻 = 𝑁 vis × 𝑝 2 where 𝑝 is the patch size and 𝑁 vis is the desired token count. The rescaled image is partitioned into a regular grid of 𝑝 × 𝑝 patches, and a vision encoder finally maps these patches into the latent representation: Z = Evis (I),
(4)
yielding 𝑁 vis visual tokens. Crucially, this decouples the token count from the original sequence length 𝑛: compression ratio is controlled entirely by the target resolution, not by token filtering. The main risk is that fine-grained lexical signals such as exact identifiers and indentation may be degraded under aggressive downscaling.
3.3
Text-to-Vector (T2V)
Unlike T2I, which avoids modifying the downstream model by switching to the visual modality, the T2V paradigm projects discrete code tokens into a continuous latent space via a learned encoder, producing a fixed-size set of memory tokens that the downstream model conditions on in place of the original sequence. Crucially, only the lightweight compression module is trained; the downstream LLM decoder remains entirely frozen, leaving the base model’s generative capability unchanged. The input is first divided into segments X = {𝑆 1, . . . , 𝑆𝑚 } for scalability. Each segment is then mapped to a compact memory block 𝐺𝑖 through two orthogonal design choices: memory organization and memory extraction mechanism.
Overview
Conference’17, July 2017, Washington, DC, USA
...
Trovato et al.
t1 t2 t3 t4 t5 t6 t7
...
Long Code Context
Text-to-Image
Compression Operator
t1 t2 t3 t4 t5 t6 t7
...
Task Instruction
Compression Operator
tn
Downstream Model
Compressed Representation
Text-to-Text
Text-to-Vector tn
Step 1: Rendering
Step 1: Segmentation
Step 1: Segmentation t1 t2 t3 t4 t5 t6 t7
...
tn
s1 s2
...
sk
Step 2: Compression Memory Organization
Step 2: Compression Step3: Partition
Segment-Local
Context-Propagated
S1
G1
G0
S2
G2
G1
...
...
Sk
Gk
S1
G1
S2
G2
...
Memory
Transformer
Step 4: Visual Encoding Compressed Visual Tokens
Hidden State
Step 3: Ouput
Memory
...
tn
S1 S2 S3 S4 S5 S6
...
Sm
S1 S2 S3 S4 S5 S6 Importance Scoring
...
Sk
Gk-1
t1 t2 t3 t4 t5 t6 t7
Step 2: Coarse-Grained Compression
Gk
Segmant Context
Learnable Queries
Transformer
CrossAttention
Hidden State
Memory
G1 G2 G3 G4 G5 G6
...
Sm
...
Selection Mask
S1 S3 S6 S8 S10 S15
Memory Extraction Self-Attentive Aggregation Query-Driven Extraction Segmant Context
Output
t1 t2 t3 t4 t5 t6 t7
Sk
...
tp
...
Step 3: Fine-Grained Compression t1 t2 t3 t4 t5 t6 t7 Importance Scoring
Selection Mask
t2 t5 t6 t8 t10 t12 t17
Gk
tp
...
ts
...
Figure 1: Overview of the unified context compression framework (top) and the three compression paradigms (bottom) . Memory organization controls whether inter-segment context is accumulated. Under Segment-Local Memory Compression (SLMC), each segment is encoded independently: 𝐺𝑖 = Compress(E (𝑆𝑖 )),
(5)
enabling parallel processing at the cost of cross-segment modeling. Under Context-Propagated Memory Compression (CPMC), the previous memory block is carried forward: 𝐺𝑖 = Compress(Concat(𝐺𝑖 −1, E (𝑆𝑖 ))),
(6)
3.4
Text-to-Text (T2T)
Unlike T2I and T2V, which change the representation space of the context, the T2T paradigm performs compression entirely in the discrete token space. It retains the model-agnostic property of the original prompt while reducing its length through a score-andselect mechanism: each token or segment is assigned an importance score and a binary selection mask retains only the most informative units under the budget: 𝜔𝑖 = Score(𝑢𝑖 | X),
max
∑︁
𝑀𝑖 · 𝜔𝑖
s.t.
∑︁
𝑀𝑖 ≤ 𝐵. (8)
𝑀
allowing progressive accumulation of context while keeping perstep memory bounded. The extraction mechanism determines how information is distilled into 𝐺𝑖 . Self-Attentive Memory Aggregation (SAMA) lets segment representations and memory tokens interact via standard transformer attention. Query-Driven Memory Extraction (QDME) introduces learnable queries 𝑄 lat ∈ R𝑘 ×𝑑 that actively retrieve information from the segment: (𝑄 lat𝑊𝑄 )(𝐻𝑆𝑖 𝑊𝐾 ) ⊤ 𝐺𝑖 = Softmax (𝐻𝑆𝑖 𝑊𝑉 ), (7) √ 𝑑 where 𝑘 directly controls the compression ratio and reduces attention complexity from O (𝑛 2 ) to O (𝑘𝑛). Combining the two memory organization strategies with the two extraction mechanisms yields four variants: SAMA-SLMC, SAMA-CPMC, QDME-SLMC, and QDME-CPMC, which we evaluate systematically in our experiments. The primary cost of T2V relative to T2I and T2T is that the compression module must be trained before deployment, requiring a pretraining stage followed by downstream adaptation.
𝑖
𝑖
Here, X and 𝐵 are consistent with their meanings in the Equation 1. 𝑢𝑖 represents a token or segment within the context that needs to be compressed, and 𝜔𝑖 represents the importance score of it. 𝑀𝑖 is either 1 or 0, indicating whether or not 𝑢𝑖 is masked. For the repository-level code tasks considered in this paper, we have applied this mechanism across two distinct stages. In the coarse stage, the context is segmented into structural units (e.g., code lines or functions), importance scores are computed at the segment level (e.g., via perplexity or retrieval relevance), and lowscoring segments are discarded. In the fine stage, token-level scores are computed within retained segments and a second selection mask produces the final compressed sequence. This coarse-to-fine decomposition makes the combinatorial selection problem tractable while preserving flexibility in the choice of scoring function. The compressed output remains human-readable and requires no modification to the downstream model, but fidelity can degrade at high compression ratios as aggressive pruning may sever long-range cross-file dependencies.
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
4
Experimental Setup
We implement all compression methods within the unified pipeline described in section 3. Each paradigm uses its corresponding downstream model (Qwen2.5-VL for T2I, Qwen2.5-Coder for T2V and T2T), with the task instruction 𝑞 kept identical across all settings so that observed differences arise solely from the compression operator C𝐵 . For each paradigm, we sweep a range of compression ratios to support the cross-paradigm comparison in RQ1–RQ3.
4.1
Subject Models
We select models from the Qwen2.5 family [28] at two scales, covering both code-specialized and multimodal variants. • Qwen2.5-Coder (QC) is a code-specialized language model family with strong capabilities in code completion, infilling, and long-context reasoning. We use QC at 3B and 7B scales to instantiate the T2V and T2T paradigms. • Qwen2.5-VL (QV) is a vision-language model family supporting joint visual and textual understanding. We use QV at 3B and 7B scales to instantiate the T2I paradigm.
4.2
Tasks and Benchmark
We evaluate on two generation-oriented code intelligence tasks using ComplexCodeEval [7], a benchmark designed to reflect realistic repository-level coding scenarios with structured inputs including function signatures, docstrings, and API usages. Code generation requires synthesizing a complete function body from a natural language specification (signature and docstring), testing whether compressed representations preserve sufficient semantic content for correct program synthesis. Code completion requires predicting the missing portion of a partially observed code snippet given its surrounding context, evaluating how well compressed representations retain local and cross-file contextual cues under a token budget. Following the original benchmark setup, we randomly sample 100 instances from each of the Java and Python subsets as a held-out evaluation set of 200 instances. For T2V, we additionally use the ComplexCodeEval training split (10k+ samples) for downstream adaptation of the compression module, with the 200 evaluation instances withheld from all training stages.
4.3
Compared Methods
We compare methods from all three paradigms against two noncompression baselines: • Full Context: the uncompressed repository context is fed directly to the downstream model, serving as the performance upper bound. • No Context: only the task instruction is provided without any context, representing the lower bound. For each paradigm, we evaluate the following representative methods: T2I. We implement a rendering-based pipeline that converts the code context into a compact visual representation consumed by Qwen2.5-VL (see subsubsection 4.4.1).
Conference’17, July 2017, Washington, DC, USA
T2V. We evaluate all four combinations of memory organization and extraction mechanism defined in section 3: SAMA-SLMC (T2VSS), SAMA-CPMC (T2V-SC), QDME-SLMC (T2V-QS), and QDMECPMC (T2V-QC). T2T. We evaluate three methods from the LLMLingua family [16, 17, 26]: LLMLingua removes low-information tokens based on small-model perplexity; LongLLMLingua extends this with queryaware scoring and reordering strategies to mitigate “lost-in-themiddle” effects; LLMLingua-2 reformulates compression as token classification using a lightweight bidirectional encoder, reducing latency while maintaining cross-domain generalization.
4.4
Implementation Details
4.4.1 Text-to-Image (T2I). We implement the T2I pipeline described in section 3 on top of Qwen2.5-VL, where one visual token corresponds to a 28 × 28 patch. Source code is rasterized in high-contrast black/white RGB; images are isotropically scaled with 𝑊 and 𝐻 aligned to multiples of 28 and downsampled with LANCZOS resampling [19] to preserve text edges at high compression ratios. Compression ratio is controlled by the target visual token budget 𝑁 = ⌊𝑛/𝑟 ⌋, where 𝑛 is the original token count. We evaluate 𝑟 ∈ {2, 4, 8, 16, 32, 64, 128}. 4.4.2 Text-to-Vector (T2V). Following the standard two-stage protocol in prior T2V work [10, 21], we train only the memory extraction operators while keeping the downstream LLM decoder frozen throughout. The two-stage design is necessary because pretraining on a text-recovery objective alone does not yield memory tokens that optimally steer the frozen decoder toward task-specific outputs; downstream adaptation closes this gap without modifying the base model. Pretraining uses StarCoderData [5] (100K Python + 100K Java). The encoder compresses the input into memory tokens, which the frozen LLM decodes; only the extraction module receives gradient updates. We train for 3 epochs with BF16 mixed-precision, learning rate 1 × 10−4 , cosine decay with 100 warmup steps, effective batch size 256, and sequence length 16,384 with gradient checkpointing. Downstream fine-tuning initializes from the pretrained checkpoint and adapts on the ComplexCodeEval training split for 3 epochs under the same learning rate schedule, using 8-step gradient accumulation to handle long-context inputs. The base Qwen2.5Coder model is never updated at any stage. Compression ratio is controlled by 𝑘, the number of memory tokens per segment (query count in QDME, memory slot count in SAMA); we evaluate 𝑟 ∈ {4, 8, 16, 32, 64, 128} by adjusting 𝑘 per segment size. 4.4.3 Text-to-Text (T2T). We apply LLMLingua, LongLLMLingua, and LLMLingua-2 with their default configurations. All three methods operate on the discrete token sequence and require no taskspecific training on the evaluation data. Unlike T2I and T2V, T2T methods accept a target compression ratio as a soft threshold and prune tokens accordingly; the actual achieved ratio may deviate from the target, as it depends on the input-specific token distribution and each method’s scoring function. We set target ratios of {2, 4, 8, 16, 32} for all three methods. 4.4.4 Inference and Hardware. All methods use greedy decoding (temperature = 0, top-𝑝 = 1). Standard inference runs with vLLM.
Conference’17, July 2017, Washington, DC, USA
Trovato et al.
Table 1: Comparison of compression paradigms at ∼4× budget on ComplexCodeEval. QC = Qwen2.5-Coder, QV = Qwen2.5-VL; T2V variants SS/SC/QS/QC denote SAMA-SLMC/CPMC and QDME-SLMC/CPMC; all metrics higher is better. Bold indicates the optimal result; underline indicates the second-best result (all subsequent tables adhere to this setting). Code Completion Method
Ratio BLEU
ES
EM
BLEU
ES
EM
BLEU
ES
BLEU
ES
QV-3B
plain context T2I
– – 4.0×
10.32 18.03 14.57
34.03 40.56 39.93
8.00 17.00 12.00
14.58 17.77 19.27
36.66 43.52 41.27
12.00 15.00 17.00
4.07 8.36 4.51
16.72 24.71 19.83
4.98 12.01 3.40
16.00 29.34 15.77
QV-7B
plain context T2I
– – 4.0×
20.29 24.91 24.11
43.69 48.98 48.52
19.00 25.00 26.00
16.52 20.47 19.29
38.90 45.02 43.91
15.00 19.00 18.00
4.18 9.19 4.52
16.44 25.46 17.29
5.72 14.49 7.91
18.94 31.29 21.73
QC-3B
plain context T2V-SS T2V-SC T2V-QS T2V-QC T2T-LL T2T-LL2 T2T-LLL
– – 4.0× 4.0× 4.0× 4.0× 3.7× 3.7× 4.6×
27.59 31.83 34.48 33.04 36.76 35.06 25.69 25.99 25.78
52.77 60.19 59.93 58.32 62.59 60.80 51.93 54.67 51.64
28.00 34.00 38.00 35.00 41.00 37.00 27.00 26.00 26.00
25.60 31.70 29.37 27.91 29.32 31.11 27.89 29.44 29.00
49.29 59.82 53.51 52.06 54.25 54.69 52.33 54.12 53.56
23.00 34.00 27.00 25.00 27.00 29.00 27.00 27.00 26.00
5.32 10.27 10.99 11.94 12.09 12.43 7.17 6.21 6.99
18.90 28.75 33.16 34.16 33.18 33.41 23.08 21.79 22.07
7.19 16.58 12.49 15.47 15.42 14.84 8.86 9.75 8.05
21.18 35.45 35.54 37.79 37.64 37.52 23.99 26.93 22.81
QC-7B
plain context T2V-SS T2V-SC T2V-QS T2V-QC T2T-LL T2T-LL2 T2T-LLL
– – 4.0× 4.0× 4.0× 4.0× 3.7× 3.7× 4.6×
29.74 32.21 41.34 38.16 40.03 37.24 29.69 30.32 30.07
54.97 56.64 67.51 62.78 65.77 63.75 55.11 54.22 56.92
27.00 33.00 42.00 41.00 44.00 40.00 29.00 29.00 30.00
28.95 32.60 28.87 31.03 33.44 35.12 29.01 28.14 31.16
50.45 55.90 53.78 54.75 60.29 60.08 52.14 51.80 55.86
28.00 34.00 27.00 29.00 33.00 35.00 28.00 27.00 30.00
6.14 10.49 12.83 12.41 13.58 12.19 6.89 7.54 7.30
20.27 28.25 34.19 33.32 34.27 32.90 23.08 24.01 22.88
7.60 17.92 15.68 15.79 17.23 13.26 9.71 10.55 10.22
21.35 36.09 38.46 38.33 39.86 35.25 25.09 26.21 25.89
Python
Java
For RQ3 efficiency profiling, we switch to native Hugging Face Transformers inference to minimize framework-induced runtime variance; throughput is reported as the inverse of end-to-end latency under identical hardware and batch settings. All experiments are conducted on a server with 8×A100 80GB GPUs; each individual training or inference run uses a single GPU.
4.5
Code Generation
Model
Evaluation Metrics
We follow the official ComplexCodeEval evaluation protocol [7] and report task-specific automatic metrics. Code generation. We report BLEU and Edit Similarity to measure n-gram overlap and string-level structural similarity between generated code and references. Code completion. We additionally report Exact Match (EM), which measures whether the predicted completion exactly matches the reference.
5 Result 5.1 RQ1: How do different compression paradigms affect task performance? Table 1 shows that at ∼4× compression, paradigm choice is the primary factor determining whether compression improves or degrades downstream performance.
Python
Java
T2V: the latent bottleneck filters repository noise. On QC7B Python, all four T2V variants exceed the full-context baseline on both completion (T2V-SS: 41.34/67.51 BLEU/ES vs. context 32.21/56.64, +28.3% BLEU) and generation (T2V-QS: 13.58/34.27 vs. context 10.49/28.25, +29.5% BLEU). At QC-3B, QDME variants exceed context on Python completion (T2V-QS: 36.76 vs. 31.83) but not on Java, suggesting sufficient model capacity is required to realize the supra-context effect. Repository context is informationally sparse: raw multi-file sequences are dominated by boilerplate, redundant declarations, and import chains that dilute task-specific signal. The latent bottleneck is trained to reconstruct task-relevant content under a fixed memory budget, effectively acting as a learned filter rather than a truncator. Variant selection is language-sensitive: segment-local encoding dominates on QC-7B Python completion (SS > QS > SC > QC), while context-propagated organization is preferred on Java (QC > QS > SC > SS), consistent with Python’s function-level modularity versus Java’s cross-class dependencies.
Finding 1. T2V surpasses full-context performance on code tasks, achieving up to +28.3% BLEU on Python completion at 7B scale. The magnitude of improvement is language- and scaledependent: gains are consistent across all variants on Python at 7B but weaker or absent on Java at 3B scale.
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
(a) Text2Image
Conference’17, July 2017, Washington, DC, USA
(b) Text2Vector
(c) Text2Text
64
60
40 62
38 37 36 35
58
Edit Similarity (%)
Edit Similarity (%)
Python
Edit Similarity (%)
39
60 58 56
34 8×
16×
32×
Compression ratio
64×
128×
4×
Edit Similarity (%)
Java
Edit Similarity (%)
42 40 38 36
8×
16×
32×
64×
Compression ratio
60
60
58
58
56 54 52 50
34 2×
4×
8×
16×
32×
Compression ratio
Text2Image
plain
64× context
128×
50
128×
Edit Similarity (%)
4×
54 52
54 2×
56
2×
4×
2×
4×
8×
17×
35×
8×
17×
35×
Compression ratio
56 54 52 50
4×
8×
16×
32×
64×
Compression ratio
SAMA-SLMC SAMA-CPMC
QDME-SLMC QDME-CPMC
128× plain context
LLMLingua LLMLingua2
Compression ratio LongLLMLingua plain
context
Figure 2: Edit Similarity (%) on code completion across compression ratios for QC-3B (T2V, T2T) and QV-3B (T2I). Dashed lines indicate no-context (plain) and full-context (context) baselines; higher is better. T2I: uniform rendering causes non-selective information loss. T2I closely tracks full context on completion (QV-7B Python: 24.11/48.52 vs. 24.91/48.98; QV-3B Java: 19.27/41.27, marginally above context 17.77/43.52), but degrades sharply on generation (QV-7B Python: 4.52/17.29 vs. context 9.19/25.46, −50.8% BLEU; QV-3B Java: 3.40/15.77, below the no-context baseline 4.98/16.00). This divergence stems from how rendering operates: every token is mapped to an equal image area with no mechanism to emphasize task-relevant regions, so uniform downscaling degrades function signatures, API chains, and cross-file imports at the same rate as surrounding boilerplate. Completion is tolerant because local syntactic cues survive moderate resolution reduction; generation requires intact cross-file relational structure that non-selective fidelity loss cannot preserve.
Finding 2. T2I performs competitively on completion but degrades sharply on generation, falling up to −50.8% BLEU below full context on Python at 7B scale. This task asymmetry is consistent across model sizes and languages.
T2T: perplexity-based scoring misaligns with code structure. On QC-3B Python completion, T2T-LL (25.69 BLEU) and T2T-LLL (25.78) fall below the no-context baseline (27.59), while the same methods remain above plain on Java (T2T-LL: 27.89; T2TLLL: 29.00). This language asymmetry reflects a structural property of Python: indentation markers are syntactically load-bearing but
locally predictable, making them early pruning targets. Their removal corrupts scope boundaries and control flow, causing the pruned context to actively mislead the model. Java’s explicit brace delimiters are more robust to token removal, which explains why T2T completion remains competitive on Java. On generation, the collapse is consistent across both languages, as import declarations and function signatures are regular patterns assigned low perplexity and thus removed before higher-ratio compression engages; once severed, cross-file dependencies cannot be recovered (T2T-LL QC-7B Python generation: 6.89/23.08 vs. plain 6.14/20.27). Finding 3. Perplexity-based pruning degrades code task performance in a language-asymmetric manner: T2T falls below the no-context baseline on Python completion at 3B scale while remaining above it on Java. On generation, performance reaches no-context level at moderate compression ratios across both languages.
5.2
RQ2: How does compression ratio affect performance?
Figures 2 and 3 report ES across compression ratios from 2× to 128×. Each paradigm’s ratio sensitivity is a direct extension of the compression mechanism identified in RQ1. T2V performance is insensitive to compression ratio. All four T2V variants produce near-flat ES curves from 4× to 128× on both tasks. On Python completion, QDME-SLMC ranges between
Conference’17, July 2017, Washington, DC, USA
Trovato et al.
(a) Text2Image
25
(b) Text2Vector
24
21 20 19
30
Edit Similarity (%)
Edit Similarity (%)
28 26 24 22 20
17 2×
4×
30
8×
16×
32×
Compression ratio
64×
128×
4×
28
Edit Similarity (%)
26 24 22 20 18 16 2×
4×
8×
16×
32×
Compression ratio
Text2Image
plain
64× context
128×
26 24 22 20
8×
16×
32×
64×
Compression ratio
128×
37.5
36
35.0
34
32.5
Edit Similarity (%)
Python
Edit Similarity (%)
22
18
Java
28
32
23
Edit Similarity (%)
(c) Text2Text
34
30.0 27.5
64×
Compression ratio
SAMA-SLMC SAMA-CPMC
QDME-SLMC QDME-CPMC
128× plain context
2×
4×
35×
8×
17×
35×
26
22 32×
17×
28
24
16×
8×
Compression ratio
30
22.5 8×
4×
32
25.0
4×
2×
20
LLMLingua LLMLingua2
Compression ratio LongLLMLingua plain
context
Figure 3: Edit Similarity (%) on code generation across compression ratios for QC-3B (T2V, T2T) and QV-3B (T2I). Dashed lines indicate no-context (plain) and full-context (context) baselines; higher is better. 62.49% and 63.94% (context: 60.19%); on generation, SAMA-SLMC holds between 32.40% and 33.27% (context: 28.75%). All four variants exceed full context on generation at every ratio across both languages, a stronger result than on completion, where Java variants remain below context throughout. The fixed memory bottleneck extracts task-relevant structure regardless of input length; increasing ratio reallocates the same latent representation rather than discarding additional information. T2I completion and generation respond differently to ratio. On completion, T2I peaks at 4× (Python: 39.93%; Java: 41.27%), degrades beyond 8×, and partially recovers at high ratios (Python: 34.15% at 32× to 36.04% at 128×; Java: from 34.03% at 16× to 35.35% at 128×, below plain 36.66% at most ratios). On generation, ES is nearly flat across all ratios (Python: 19.20%–20.35%; Java: 15.77%– 18.40%), well below context at every point. Local syntactic cues sufficient for completion survive moderate resolution reduction, so ratio governs how much of this signal is preserved; cross-file relational structure required for generation is lost at any resolution, making the generation gap ratio-independent. T2T generation utility collapses at a method-specific critical ratio. On completion, all three methods degrade gradually and remain above plain across most of the evaluated range (LLMLingua2 Python: 54.67% at 3.7× to 50.98% at 34.8×, vs. plain 52.77%). On generation, degradation is substantially faster: LLMLingua Python ES reaches 20.31% at 12×, equalling the no-context baseline (20.27%); LLMLingua2 falls below plain at 8.1× (19.76%); LongLLMLingua crosses the threshold at 7× on both Python (19.84%) and Java (22.44% vs. plain 21.35%). Python reaches these critical ratios consistently
earlier than Java, in line with its higher structural sensitivity to perplexity-based pruning (Finding 3). Finding 4. T2V performance is stable across 4×–128×, with the supra-context generation advantage maintained at all evaluated ratios. T2I completion peaks at 4× and degrades with further compression, while generation remains nearly flat and consistently below full context regardless of ratio. T2T generation reaches no-context level at 7×–12×, with Python reaching this threshold at lower ratios than Java.
5.3
RQ3: How do compression methods/ratios affect throughput and resource usage?
Tables 2–4 report latency and GPU memory for QV-3B (T2I) and QC3B (T2V, T2T). All methods remain well below full-context overhead across all ratios, with decoding dominating the total costs. T2I achieves the most pronounced efficiency improvement with ratio. Compression and decoding latency decrease monotonically: compression falls from 0.42 s at 2× to 0.29 s at 128×; decoding from 6.78 s to 4.19 s, converging to plain (4.17 s). GPU memory follows the same trend, from 8.54 GB to 7.10 GB, nearly matching plain (7.09 GB). At 4×, the performance peak for completion (Finding 4), total latency is already 33% below full context (6.02 s vs. 8.95 s); at 128×, the overhead above plain reduces to 0.31 s and 0.01 GB. T2V maintains stable compression cost while decoding latency decreases gradually. Compression overhead is small and
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
Conference’17, July 2017, Washington, DC, USA
Table 2: T2I latency breakdown (s/sample) and peak GPU memory under increasing compression ratios; lower is better for all columns.
Table 4: T2T latency breakdown (s/sample) and peak GPU memory across LLMLingua-family methods and compression ratios; lower is better for all columns.
Method
Ratio
Comp.
Decod.
Total
GPU (GB)
Method
Ratio
Comp.
Decod.
Total
GPU (GB)
Plain Context
– –
0.00 0.00
4.17 8.95
4.17 8.95
7.09 10.78
Plain Context
– –
0.00 0.00
4.21 9.13
4.21 9.13
5.85 10.92
2.0× 4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
0.42 0.36 0.34 0.31 0.30 0.30 0.29
6.78 5.67 5.50 5.18 4.83 4.47 4.19
7.20 6.02 5.83 5.50 5.13 4.76 4.48
8.54 7.79 7.39 7.22 7.14 7.11 7.10
LLMLingua
2.0× 3.7× 5.8× 8.1× 12.0×
0.64 0.66 0.63 0.63 0.63
6.15 5.17 4.67 4.75 4.45
6.80 5.83 5.30 5.38 5.08
6.54 6.16 6.01 5.94 5.90
LLMLingua2
1.8× 3.7× 8.1× 17.1× 34.8×
0.22 0.22 0.22 0.22 0.22
7.31 7.05 5.33 4.77 4.36
7.52 7.27 5.54 4.98 4.58
6.69 6.26 6.03 5.93 5.88
LongLLMLingua
2.6× 4.6× 6.1× 6.5× 7.0×
1.36 1.38 1.30 1.28 1.24
5.43 4.78 4.63 4.29 4.30
6.80 6.15 5.93 5.57 5.54
6.75 6.26 6.05 5.91 5.87
Text-to-Image
Table 3: T2V latency breakdown (s/sample) and peak GPU memory across all four variants and compression ratios; lower is better for all columns. Variant
Ratio
Comp.
Decod.
Total
GPU (GB)
Plain Context
– –
0.00 0.00
4.21 9.13
4.21 9.13
5.85 10.92
SAMA-SLMC
4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
0.22 0.23 0.22 0.21 0.22 0.25
6.29 6.03 6.08 5.54 5.71 5.46
6.51 6.26 6.30 5.75 5.93 5.71
6.57 6.51 6.48 6.46 6.45 6.45
SAMA-CPMC
4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
0.24 0.25 0.25 0.25 0.25 0.24
6.38 6.28 6.17 6.09 6.28 5.82
6.63 6.53 6.41 6.34 6.53 6.07
6.68 6.53 6.47 6.44 6.43 6.42
QDME-SLMC
4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
0.22 0.22 0.23 0.22 0.22 0.22
6.21 6.00 6.05 5.98 6.05 5.82
6.42 6.22 6.28 6.20 6.26 6.04
6.52 6.51 6.51 6.51 6.51 6.51
QDME-CPMC
4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
0.25 0.25 0.27 0.25 0.26 0.26
6.49 6.21 6.25 6.21 6.40 6.19
6.74 6.46 6.52 6.46 6.66 6.45
6.60 6.54 6.51 6.49 6.48 6.48
ratio-independent across all variants (0.21–0.27 s). Decoding latency decreases with ratio (SAMA-SLMC: 6.29 s at 4× to 5.46 s at 128×; QDME-SLMC: 6.21 s to 5.82 s), with a modest accompanying decrease in GPU memory (SAMA-SLMC: 6.57 GB to 6.45 GB). All variants remain well below full context at every ratio, and the gap to plain narrows to approximately 1.5 s and 0.6 GB at high ratios. CPMC variants incur slightly higher compression overhead than SLMC counterparts (0.24–0.27 s vs. 0.21–0.23 s), consistent with the additional cross-segment computation described in section 3. T2T compression cost is method-dependent and ratio-stable. Compression overhead varies substantially across methods but is
stable within each: LLMLingua2 costs 0.22 s (comparable to T2V), LLMLingua 0.63–0.66 s, and LongLLMLingua 1.24–1.38 s due to reordering. Decoding latency decreases with ratio in all three methods, and GPU memory follows the same trend. All methods converge toward total latencies of 4.58–5.54 s at their most aggressive settings, approaching the plain baseline. As established in Finding 4, however, generation performance has already converged to the no-context level at those ratios. Finding 5. All three paradigms reduce inference cost relative to full-context decoding across all evaluated ratios. T2I achieves the most pronounced improvement, with both compression and decoding costs decreasing monotonically to nearwithout-context levels at 128×. T2V compression overhead is small and ratio-independent; decoding latency decreases gradually, with all variants remaining well below full-context cost throughout. T2T compression overhead is method-dependent but ratio-stable; all methods approach without-context latency at aggressive compression settings.
6 Discussion 6.1 Compression Fidelity The three paradigms produce qualitatively different surrogate representations, which determines how their compression fidelity can be characterized. T2T outputs human-readable compressed text, enabling direct qualitative inspection. Figure 4 shows LLMLingua applied to a Java file at two ratios. At 1.7×, the output is nearly identical to the original—only minor whitespace and import consolidation occur. At 3.4×, several import statements and intermediate variable assignments are pruned while the core logic survives. This reveals a structural weakness of perplexity-based scoring: imports and type annotations are linguistically predictable and thus receive low importance scores, yet they encode the cross-file dependencies that
Conference’17, July 2017, Washington, DC, USA
Trovato et al.
<filename>src/test/java/tdl/s3/testframework/rules/LocalTestBucket.java package tdl.s3.testframework.rules; import com.amazonaws.auth.AWSCredentials; import com.amazonaws.auth.AWSStaticCredentialsProvider; import com.amazonaws.auth.BasicAWSCredentials; import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration; import com.amazonaws.services.s3.AmazonS3ClientBuilder; public class LocalTestBucket extends TestBucket { public LocalTestBucket() { EndpointConfiguration endpoint = new EndpointConfiguration("http://127. 0.0.1:9000", "us-east-1"); AWSCredentials credential = new BasicAWSCredentials( "minio_access_key", "minio_secret_key" ); amazonS3 = AmazonS3ClientBuilder .standard() .withPathStyleAccessEnabled(true) .withCredentials(new AWSStaticCredentialsProvider(credential)) .withEndpointConfiguration(endpoint) .build(); bucketName = "localbucket"; if (!amazonS3.doesBucketExist(bucketName)) { amazonS3.createBucket(bucketName); } uploadPrefix = "prefix/"; } }
<filename>src/test/java/tdl/s3/testframework/rules/LocalTestBucket.java import com.amazonaws.authredentialsimport.amazonaws.authProvider.amazonaws.authr edentialsimport.amazonaws.builderBuilderConfiguration.amazonaws33ClientBuilder; publicBucket extendsBucket { publicBucket() Configuration endpointConfiguration("://1:900", "us1 redentials credentialredentials( "io_access_key _secret_key amazon3Builder .withStyleEnabl ed(truewithwith) .build(); bucketName = "localbucket"; if (!amazonS3.doesBucketExist(bucketName)) { amazonS3.createBucket(bucketName); } uploadPrefix = "prefix/"; } }
Original Content
Compressed Content (Ratio: 3.4x)
Compressed Content (Ratio: 1.7x) <src/tdl/s3/testframework/LocalTest.amazonaws.amazonaws3Builder ials "io with) .build(); bucketName = "localbucket"; if (!amazonS3.doesBucketExist(bucketName)) { amazonS3.createBucket(bucketName); } uploadPrefix = "prefix/"; } }
redentialsredent
Figure 4: T2T (LLMLingua) compression applied to a representative Java source file. Left: original content. Middle and right: compressed outputs at 1.7× and 3.4× ratios, respectively. Table 5: Text recovery results (BLEU and ES) for T2I and T2V across compression ratios, evaluated on 100 code files sampled from StarCoderData. Ratio 2.0× 4.0× 8.0× 16.0× 32.0× 64.0× 128.0×
Text-to-Image BLEU ES 63.53 82.50 22.80 46.38 4.59 24.85 0.57 18.48 0.35 15.49 0.23 10.42 0.18 7.45
Text-to-Vector BLEU ES – – 84.12 91.57 66.99 80.49 53.66 69.83 35.55 56.87 28.34 50.60 16.51 41.09
generation tasks rely on—consistent with the performance collapse observed at higher ratios in RQ2. T2I and T2V produce non-human-readable representations (patch embeddings and latent memory vectors), making qualitative analysis infeasible. We instead evaluate their information retention via a text recovery task: reconstruct the original source code from the compressed representation and measure BLEU and ES against the ground truth. We sampled 100 code files from StarCoderData [5]; results are shown in Table 5. T2V preserves substantially more information at every ratio—at 4×, BLEU = 84.12 / ES = 91.57 versus T2I’s 22.80 / 46.38; at 128×, T2V still yields 16.51 / 41.09 while T2I collapses to 0.18 / 7.45. This contrast reflects the core difference between the two paradigms: T2V’s learned bottleneck actively distills task-relevant token structure, whereas T2I’s rasterization distributes all content uniformly across pixels and irreversibly
loses fine-grained lexical signals under aggressive downscaling— providing a direct explanation for the downstream performance gaps observed in RQ1 and RQ2.
6.2
Implications of Findings
We summarize the implications of our study from both practitioners’ and researchers’ perspectives. For practitioners: Paradigm selection should be driven by task type, deployment constraints, and compression budget, as these factors interact in ways that no single method addresses optimally. (1) When output performance is the primary requirement, particularly for generation tasks, latent-vector compression (text-to-vector, T2V) is the recommended choice: it can surpass full-context performance and remains stable across a wide compression range (4× to 128×) with predictable, ratioindependent efficiency overhead. The key cost is that a taskspecific compression module must be trained per base model, though the base LLM itself remains frozen throughout. (2) When training-free deployment is required, two options are available depending on model type. Visual compression (textto-image, T2I) applies to vision-language models and is effective for completion tasks at moderate ratios; it should be avoided for generation tasks, as uniform rendering loses cross-file relational structure regardless of compression ratio. Token-filtering compression (text-to-text, T2T) is modelagnostic and requires no modification to the base LLM, making it the most portable option; however, it should be limited
On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation
to mild ratios and larger models, as structurally critical tokens degrade rapidly under aggressive compression. (3) When throughput or latency is the primary constraint, T2I offers the most favorable progressive latency scaling and suits throughput-sensitive deployments; T2V provides stable, ratio-independent overhead well-suited to predictable serving environments. For researchers: Our findings reveal several open challenges specific to programming language structure that do not arise in natural language settings, pointing to concrete directions for future work. (1) T2V’s supra-context performance indicates that repository context contains substantial redundancy that standard fullcontext decoding does not resolve. Researchers should investigate task-adaptive or query-conditioned memory allocation strategies to further improve the selectivity of the latent bottleneck. (2) Perplexity-based scoring is poorly aligned with code structure: tokens most critical for cross-file reasoning are linguistically predictable and thus assigned low importance. Compression methods for code should incorporate programspecific signals such as AST node types, control-flow reachability, and cross-file dependency frequency. (3) Visual compression currently allocates equal pixel area to all tokens, with no mechanism to prioritise structurally important regions. Layout-aware rendering strategies that assign higher resolution to function signatures and import blocks are a promising direction for improving fidelity under aggressive downscaling. (4) The language-specific performance asymmetries observed across all three paradigms indicate that programming language syntax should be treated as a first-class signal in compression design rather than assumed to transfer uniformly across languages.
6.3
Threats to Validity
Hardware and Environment Variability. Efficiency metrics such as GPU memory and inference latency are sensitive to hardware configuration and software environment, and absolute values may not transfer directly to other setups. We mitigate this by running all methods and compression ratios on the same server under identical software environments, ensuring that the relative trends and crossparadigm comparisons reported in RQ3 reflect differences in the compression operators rather than environmental noise. Potential Data Leakage. Since the pretraining corpora of the evaluated models are not publicly disclosed, we cannot fully rule out overlap with our evaluation instances. However, all methods are evaluated under identical prompts and experimental settings, so any leakage would affect all paradigms equally and would not distort the relative comparisons that are the focus of this study.
7
Conclusion
We present the first systematic empirical study of context compression for repository-level code intelligence, categorizing representative methods into three paradigms: Text-to-Vector (T2V), Text-to-Image (T2I), and Text-to-Text (T2T). We evaluate them on
Conference’17, July 2017, Washington, DC, USA
code completion and generation tasks across multiple model scales, jointly measuring task performance, ratio sensitivity, and deployment efficiency. Our study yields three principal findings. First, T2V surpasses full-context performance on both tasks, with the latent vectors acting as a selective filter on informationally sparse repository context rather than a mere compressor. Second, ratio sensitivity is paradigm-specific: T2V is stable across a wide compression range, T2I degrades with visual resolution due to non-selective rendering, and T2T’s effective information loss outpaces its nominal token reduction. Third, T2I offers the most favorable latency scaling with compression ratio, while T2V incurs a fixed, predictable efficiency overhead; T2T provides latencies close to the context-free baseline only at ratios where performance has already collapsed. Together, these findings establish context compression as a viable and practically beneficial strategy for code LLMs, and surface open challenges specific to programming language structure that distinguish this setting from the natural language domain.
References [1] Aydar Bulatov, Yuri Kuratov, and Mikhail S. Burtsev. 2022. Recurrent Memory Transformer. arXiv:2207.06881 [cs.CL] https://arxiv.org/abs/2207.06881 [2] Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025. PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model. arXiv:2510.14528 [cs.CV] https://arxiv. org/abs/2510.14528 [3] DeepSeek-AI. 2025. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL] https://arxiv.org/abs/2412.19437 [4] Mark Chen et al. 2021. Evaluating Large Language Models Trained on Code. arXiv:2107.03374 [cs.LG] https://arxiv.org/abs/2107.03374 [5] Raymond Li et al. 2023. StarCoder: may the source be with you! arXiv:2305.06161 [cs.CL] https://arxiv.org/abs/2305.06161 [6] Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. Zhang. 2023. Large Language Models for Software Engineering: Survey and Open Problems. arXiv:2310.03533 [cs.SE] https: //arxiv.org/abs/2310.03533 [7] Jia Feng, Jiachen Liu, Cuiyun Gao, Chun Yong Chong, Chaozheng Wang, Shan Gao, and Xin Xia. 2024. ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (ASE ’24). ACM, 1895–1906. doi:10.1145/3691620.3695552 [8] Lang Feng, Fuchao Yang, Feng Chen, Xin Cheng, Haiyang Xu, Zhenglin Wan, Ming Yan, and Bo An. 2026. AgentOCR: Reimagining Agent History via Optical Self-Compression. arXiv:2601.04786 [cs.LG] https://arxiv.org/abs/2601.04786 [9] Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. arXiv:2002.08155 [cs.CL] https://arxiv.org/abs/2002.08155 [10] Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. 2024. In-context Autoencoder for Context Compression in a Large Language Model. arXiv:2307.06945 [cs.CL] https://arxiv.org/abs/2307.06945 [11] Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. UniXcoder: Unified Cross-Modal Pre-training for Code Representation. arXiv:2203.03850 [cs.CL] https://arxiv.org/abs/2203.03850 [12] Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. arXiv:2009.08366 [cs.SE] https://arxiv.org/abs/2009.08366 [13] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming – The Rise of Code Intelligence. arXiv:2401.14196 [cs.SE] https: //arxiv.org/abs/2401.14196 Attention mechanisms in neural networks. [14] Hasi Hays. 2026. arXiv:2601.03329 [cs.LG] https://arxiv.org/abs/2601.03329 [15] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024.
Conference’17, July 2017, Washington, DC, USA
Qwen2.5-Coder Technical Report. arXiv:2409.12186 [cs.CL] https://arxiv.org/ abs/2409.12186 [16] Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. arXiv:2310.05736 [cs.CL] https://arxiv.org/abs/2310.05736 [17] Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. arXiv:2310.06839 [cs.CL] https://arxiv.org/abs/2310.06839 [18] Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2026. A Survey on Large Language Models for Code Generation. ACM Transactions on Software Engineering and Methodology 35, 2 (Jan. 2026), 1–72. doi:10.1145/3747588 [19] Cornelius Lanczos. 1964. Evaluation of noisy data. Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis 1, 1 (1964), 76–85. [20] Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing Context to Enhance Inference Efficiency of Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 6342–6353. doi:10.18653/v1/2023.emnlp-main.391 [21] Zongqian Li, Yixuan Su, and Nigel Collier. 2024. 500xCompressor: Generalized Prompt Compression for Large Language Models. arXiv:2408.03094 [cs.CL] https://arxiv.org/abs/2408.03094 [22] Fanfan Liu and Haibo Qiu. 2025. Context Cascade Compression: Exploring the Upper Limits of Text Compression. arXiv:2511.15244 [cs.CL] https://arxiv.org/ abs/2511.15244 [23] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics 12 (2024), 157–173. doi:10.1162/tacl_a_00638 [24] Jesse Mu, Xiang Lisa Li, and Noah Goodman. 2024. Learning to Compress Prompts with Gist Tokens. arXiv:2304.08467 [cs.CL] https://arxiv.org/abs/2304.08467 [25] OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774 [26] Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression. arXiv:2403.12968 [cs.CL] https: //arxiv.org/abs/2403.12968 [27] Fei Qi, Yingnan Hou, Ning Lin, Shanshan Bao, and Nuo Xu. 2024. A Survey of Testing Techniques Based on Large Language Models. In Proceedings of the 2024 International Conference on Computer and Multimedia Technology (Sanming, China) (ICCMT ’24). Association for Computing Machinery, New York, NY, USA, 280–284. doi:10.1145/3675249.3675298 [28] Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2025. Qwen2.5 Technical Report. arXiv:2412.15115 [cs.CL] https://arxiv.org/abs/2412.15115 [29] Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2024. Code Llama: Open Foundation Models for Code. arXiv:2308.12950 [cs.CL] https://arxiv.org/abs/2308.12950 [30] Weisong Sun, Yun Miao, Yuekang Li, Hongyu Zhang, Chunrong Fang, Yi Liu, Gelei Deng, Yang Liu, and Zhenyu Chen. 2025. Source Code Summarization in the Era of Large Language Models. arXiv:2407.07959 [cs.SE] https://arxiv.org/ abs/2407.07959 [31] Yutao Sun, Zhenyu Li, Yike Zhang, Tengyu Pan, Bowen Dong, Yuyi Guo, and Jianyong Wang. 2026. Efficient Attention Mechanisms for Large Language Models: A Survey. arXiv:2507.19595 [cs.CL] https://arxiv.org/abs/2507.19595 [32] Gemini Team. 2025. Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 [cs.CL] https://arxiv.org/abs/2312.11805 [33] Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-Attention with Linear Complexity. arXiv:2006.04768 [cs.LG] https://arxiv.org/abs/2006.04768 [34] Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, and Bang Liu. 2025. R3 Mem: Bridging Memory Retention and Retrieval via Reversible Compression. arXiv:2502.15957 [cs.CL] https://arxiv.org/abs/2502.15957 [35] Yue Wang, Weishi Wang, Shafiq Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. arXiv:2109.00859 [cs.CL] https://arxiv.org/abs/2109. 00859
Trovato et al.
[36] Haoran Wei, Yaofeng Sun, and Yukun Li. 2025. DeepSeek-OCR: Contexts Optical Compression. arXiv:2510.18234 [cs.CV] https://arxiv.org/abs/2510.18234 [37] Haoran Wei, Yaofeng Sun, and Yukun Li. 2026. DeepSeek-OCR 2: Visual Causal Flow. arXiv:2601.20552 [cs.CV] https://arxiv.org/abs/2601.20552 [38] Boyang Yang, Zijian Cai, Fengling Liu, Bach Le, Lingming Zhang, Tegawendé F. Bissyandé, Yang Liu, and Haoye Tian. 2025. A Survey of LLM-based Automated Program Repair: Taxonomies, Design Paradigms, and Applications. arXiv:2506.23749 [cs.SE] https://arxiv.org/abs/2506.23749