When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition Chencheng Zhu1 , Xiaoyang Li2 , Taotao Cai2 2

1 UNSW Sydney University of Southern Queensland

arXiv:2608.09490v1 [cs.LG] 10 Aug 2026

Abstract Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function. We separate parameter geometry from functional geometry and measure pairwise functional non-additivity over a twodimensional task-vector surface, using a first-token predictivedistribution interaction ratio conditioned on an input distribution and evaluated with norm-matched controls, three training seeds, and response-only fine-tuning. On Qwen2.5-1.5B, code+safety is more non-additive than the matched code+math control on code and instruction prompts, but not on math prompts. In a prospectively specified six-task expansion, all eight high-versus-low comparisons of unseen task pairs have the predicted sign. The primary ordering further persists under full-parameter fine-tuning at 0.5B, Qwen2.5 LoRA scale tests up to 7B, and a Llama-3.1-8B cross-architecture audit. External validation exposes a sharper boundary: raw public code, instruction, and safety prompts preserve the continuous contrast, whereas an instruction-style wrapper collapses it on the identical public-code prompts, and EvalPlus pass@1 interactions do not robustly reproduce it. Weight-space composition therefore supports coarse, input- and format-conditioned functional statements across adaptation methods, scales, and one additional model family, not a universal merging-performance predictor.

Introduction Task arithmetic offers an appealing interface for model reuse: subtract a shared base model from models fine-tuned for individual tasks, then add the resulting task vectors to combine capabilities (Ilharco et al. 2023). It extends work on averaging compatible solutions (Izmailov et al. 2018; Wortsman et al. 2022; Matena and Raffel 2022; Ramé et al. 2023). Its simplicity has motivated methods that resolve sign conflicts, select coefficients, localize weights relevant to each task, or otherwise improve merged performance (Yadav et al. 2023; Yang et al. 2024; Wang et al. 2024). Yet a displacement is defined in parameter space, whereas success is judged by behavior; the two geometries need not agree. Parameter, representation, and functional geometry are distinct: they concern weights, activations induced by inputs, and conditional maps from inputs to outputs. Paths of low loss, symmetry alignment, and local linearization clarify weight connectivity or fine-tuning regimes (Garipov et al.

2018; Draxler et al. 2018; Entezari et al. 2022; Ainsworth, Hayase, and Srinivasa 2023; Malladi et al. 2023); task arithmetic in tangent space connects these views for model editing (Ortiz-Jimenez, Favero, and Frossard 2023). None implies that a task vector has a global meaning independent of the input or composes benignly with another. Rather than optimizing coefficients or resolving parameter conflicts, we scan a two-dimensional composition surface as a diagnostic. We compare the actual output distribution with an additive prediction assembled from paths along the two axes. This isolates their interaction after subtracting the marginal curvature of each axis (Figure 1). The resulting interaction is indexed by a prompt distribution and tested with matched seeds, norm intervention, prospective transfer tests, and targeted stress audits. Functional interference is jointly determined by the task pair and input distribution. After norm matching on Qwen2.5-1.5B, code+safety exceeds code+math by 5.96 points on code prompts and 7.58 on instruction prompts, but only 0.10 on math prompts. All eight prospectively specified comparisons between high and low bins involving held out tasks have the predicted sign, although finer ordering is unreliable. The contrast persists under targeted audits of parameterization, scale, and model family. Yet public transfer depends on format: raw public code, instruction, and safety prompts preserve it, an instruction wrapper suppresses it on identical code prompts, and pass@1 does not robustly reproduce it. These findings support three contributions: • We define a functional interaction surface conditioned on the input that separates non-additivity between directions from effects along each axis. • Prospective tests on unseen pairs and targeted stress tests show transfer beyond the original pairs and parameterization, across audited scales, and to one additional model family. • We establish boundaries relevant to deployment: the functional contrast persists even though the audited clustering diagnostics are not significant, public transfer depends on format, and continuous non-additivity cannot replace evaluation on intended prompts and behavioral metrics.

Related Work Task vectors and local linearity. Task vectors express fine-tuning as displacements that support addition and negation (Ilharco et al. 2023). Weight averaging and modeconnectivity results characterize compatible basins and paths (Izmailov et al. 2018; Garipov et al. 2018; Draxler et al. 2018; Wortsman et al. 2022; Matena and Raffel 2022), while permutation alignment extends compatibility across symmetric parameterizations (Entezari et al. 2022; Ainsworth, Hayase, and Srinivasa 2023). Tangent-space arithmetic and kernel analyses connect composition to local function-space structure (Ortiz-Jimenez, Favero, and Frossard 2023; Malladi et al. 2023); intrinsic-dimension and LoRA theory indicate that effective adaptation may occupy a small subspace (Aghajanyan, Zettlemoyer, and Gupta 2021; Jang, Lee, and Ryu 2024). Fine-tuned models are also known to occupy compact regions of weight space (Gueta et al. 2023); our clustering audit therefore asks whether such regional structure predicts how two displacements interact, not whether it exists. We test whether such directions define stable behavioral axes across inputs. Interference in model merging. TIES-Merging addresses redundant updates and sign disagreement (Yadav et al. 2023); DARE sparsifies task deltas (Yu et al. 2024); AdaMerging learns coefficients (Yang et al. 2024); and RegMean or MaTS use data statistics or task parameter subspaces (Jin et al. 2023; Tam, Bansal, and Raffel 2024). Other methods localize sparse weights (Wang et al. 2024; He et al. 2024), modify feature alignment (Stoica et al. 2024), or guide task vector combination (Cheng et al. 2025). Merging language models remains task dependent (Morrison et al. 2024), and heterogeneous checkpoints violate assumptions of clean experts (Hitit, Girrbach, and Akata 2026). Two recent studies are closest to ours and differ in target. Zhou et al. (2026) predict pairwise merge performance from interpretable checkpoint properties, and Sivaramakrishnan et al. (2026) show that simple parameter-space angles are not a stable predictor of adapter interference. We instead measure input-conditioned non-additivity in the space of output distributions, and we make no claim to predict benchmark merge performance; our external audits characterize where that claim would fail. LoRA geometry and safety directions. Low-Rank Adaptation freezes base weights and trains low-rank factors (Hu et al. 2022); its neural tangent optimization has been characterized theoretically (Jang, Lee, and Ryu 2024). AdapterFusion and AdapterSoup combine modular adaptations in activation or weight space (Pfeiffer et al. 2021; Chronopoulou et al. 2023), and LoraHub assembles LoRA modules with learned coefficients for unseen tasks (Huang et al. 2024). For LoRA, KnOTS uses a shared SVD space (Stoica et al. 2025), while clustering by rank composes smaller semantic units (Zhao et al. 2025). Our audit with full parameter fine-tuning tests whether the ordering is an artifact of low-rank adaptation. Safe LoRA projects updates toward an aligned subspace (Hsu et al. 2024), but we make no claim that a global safety subspace exists.

Task relations. Task embeddings and grouping methods quantify which tasks may transfer or compete (Achille et al. 2019; Standley et al. 2020; Fifty et al. 2021). These precedents motivate our prospectively specified family ordering, but our target differs: functional non-additivity after training rather than allocation during joint training. Functional evaluation. Distances between output distributions provide continuous sensitivity even when task accuracy is discrete. For external behavioral evaluation, we use EvalPlus, which augments HumanEval and MBPP with additional tests to expose functionally incorrect code (Chen et al. 2021; Austin et al. 2021; Liu et al. 2023). The discrepancy between our continuous interaction metric and pass@1 is treated as a substantive validity boundary rather than a benchmark failure.

Measuring Functional Interaction Task vector surface. Let θ0 be a shared base model and ∆a = θa − θ0 , ∆b = θb − θ0 be LoRA displacements for two tasks after merging the adapters into the base weights. All quantities for a task are evaluated for a matched training seed s; we suppress this index until defining the estimand for each seed below. We evaluate θ(α, β) = θ0 + α∆a + β∆b ,

α, β ∈ [0, 1.2]

(1)

on a 7×7 grid. The point (1, 1) is the ordinary task arithmetic merge, while the axes retain the full nonlinear path for each task. For prompt x, denote the logits for the next token by ℓα,β (x) and let pα,β (· | x) = softmax(ℓα,β (x)). A nointeraction prediction is assembled in logit space: ℓadd (x; α, β) = ℓα,0 (x) + ℓ0,β (x) − ℓ0,0 (x).

(2)

Unlike a chord from the base to the merged endpoint, this prediction reuses both observed axis paths. This removes the marginal nonlinearity of each direction before measuring their joint effect. Softmax is applied only after logit composition, ensuring a normalized predictive distribution. Equivalently, for vocabulary item v, padd (v | x) ∝

pα,0 (v | x)p0,β (v | x) . p0,0 (v | x)

(3)

This construction is a normalized additive displacement in log probability coordinates and is invariant to arbitrary scalar shifts of any constituent logit vector. It is a chosen operationalization of additivity between directions, not a definition of functional interaction that is independent of the model. Interaction ratio conditioned on the input. tion for a prompt is

The interac-

Ix (α, β) = JSD(pα,β (· | x), padd (· | x)) ,

(4)

where padd = softmax(ℓadd ), and the base displacement for a prompt is Mx (α, β) = JSD(pα,β (· | x), p0,0 (· | x)). Let G◦ = {0.2, 0.4, . . . , 1.2}2 denote the 36 interior grid points. To make the interaction comparable across surfaces

Figure 1: Measurement pipeline for input-conditioned functional interaction. Two task vectors from a shared base define a 7 × 7 composition surface that is evaluated on prompts x ∼ X. At each interior point, the observed joint distribution is compared with an additive counterfactual constructed from the observed axis logits. Pointwise Jensen–Shannon divergences measure joint interaction and displacement from the base. The numerator and denominator are averaged separately over prompts and the 36 interior points before forming a seed-level ratio of expectations. Reported estimates then average the seed-level ratios with equal weight. with different marginal movement, we normalize it by the actual displacement from the base: (s)

Ia,b (X) = Ex∼X,(α,β)∼G◦ Ix(s) (α, β), (s)

Ma,b (X) = Ex∼X,(α,β)∼G◦ Mx(s) (α, β),

(5)

(s)

(s) Ra,b (X) =

Ia,b (X) (s)

,

ε = 10

−12

,

Ma,b (X) + ε

where both expectations are uniform over prompts in X and grid points in G◦ , and all distributions in a given ratio come from matched training seed s. The reported family estimate is P (s) R̄a,b (X) = |S|−1 s∈S Ra,b (X) with S = {42, 123, 456}. The input distribution X is an explicit argument. We never aggregate it away before testing whether a pair contrast changes across prompt strata. Radial intervention. Update magnitude is a potential confound: a small vector may appear more additive merely because it remains closer to the base. Our primary mechanism analysis rescales every rank-16 task vector to the median norm of all vectors across tasks and seeds in the corresponding audit block. Thus, the target is computed separately for the calibration with four tasks, the expansion to six tasks, and each directed stress audit with three tasks. This core median norm matching preserves directions while equalizing radii. These endpoints are interventions, not trained checkpoints; results at the natural radii remain secondary descriptions of training displacements. Statistical estimands. Within each training seed, we separately average the interaction numerator and base displacement denominator over prompts and interior grid points, take their ratio, and then average the ratios across seeds with equal weight. We do not average ratios for individual prompts in the primary estimand and do not filter prompts by denominator size. Uncertainty is estimated with a hierarchical bootstrap that resamples seeds as the outer unit and prompts as the

inner unit and recomputes the ratio of means. Matched contrasts share prompt indices across families. Because three seeds yield a minimum exact sign test value of 0.125 under a directional alternative even with perfect concordance, we report bootstrap intervals over prompts and directional consistency across seeds separately. Nominal correlations across related task families are descriptive. Normalization audit. Because R can be unstable when the base displacement is small, a frozen reporting audit records denominator quantiles for every combination of family, stratum, and seed. It flags but never removes boundary cases. No primary cell crosses the specified boundary. The audit reproduces the ratio of expectations exactly and preserves its sign pattern when ratios are instead computed for each prompt before aggregation; the magnitudes depend on the estimator. Diagnostics in parameter space. We audit global cosine similarity, the TIES sign conflict rate over coordinates that are nonnegligible in both vectors, and layerwise overlap between singular subspaces. These features test whether simple parameter summaries explain the observed functional surface; they are not used to define the primary interaction.

Experimental Setup Models and fine-tuning. We use Qwen2.5 base models at 0.5B, 1.5B, and 7B parameters (Qwen Team 2025). For each task and seed {42, 123, 456}, we train a rank-16 LoRA adapter on 3,000 examples for two epochs. LoRA is applied to the attention query, key, value, and output projections and the MLP gate, up, and down projections. We use α = 32, dropout 0.05, learning rate 2 × 10−4 , cosine scheduling, 5% warmup, effective batch size 16, and bfloat16. In a separate 0.5B audit, we fine-tune all parameters (Full FT) on math, code, and safety. This audit uses the same data, seeds, response-only objective, epochs, and effective batch size, with a prospectively specified learning rate of 2 × 10−5 . The audit on a second architecture repeats the directed math, code, and safety LoRA test on the Llama-3.1-8B base model with

Study

Tasks

Seeds

Surfaces

1.5B calibration 1.5B OOS 0.5B replication 0.5B Full FT audit Qwen2.5-7B scale test Llama-3.1-8B architecture test Public raw/wrapped prompts

4 6 4 3 3 3 2 fam.

3 3 3 3 3 3 3

9 36 9 6 6 6 6

Table 1: Main experimental blocks. Each surface is evaluated separately on every available prompt stratum.

open weights (Dubey et al. 2024). We lock the checkpoint revision before training, and the analysis computes LoRA inner products directly from adapter factors and composes two adapters during the forward pass. The frozen protocol supervises response tokens and the true EOS token while masking prompt and padding positions. Only checkpoints trained under this response-only protocol enter the main study. An earlier pilot protocol shared the padding and EOS identifiers, which inadvertently masked real EOS targets; it was replaced before any surface reported here was evaluated, and no pilot checkpoint enters any reported estimate. Tasks and prompts. The four calibration tasks are math (GSM8K; Cobbe et al. 2021), code (CodeAlpaca-20k; Chaudhary 2023), general instruction following (Alpaca; Taori et al. 2023), and safety response modeling (PKUSafeRLHF; Ji et al. 2024). The prospectively specified expansion adds XSum summarization (Narayan, Cohen, and Lapata 2018) and OPUS Books translation from English to French (Tiedemann 2012). Long XSum documents are truncated inside the document field while preserving the response delimiter. Each controlled stratum contains 60 prompts held out from training and is evaluated independently. Calibration and confirmatory roles. The initial 1.5B block with four tasks is exploratory calibration. It evaluates three anchor families: code+math, code+safety, and instruction+safety. The central code+safety versus code+math comparison is useful because both families share the same code checkpoint within each seed; its 1.5B result is descriptive rather than prospective. We froze this comparison before the 0.5B replication and subsequent audits of parameterization, scale, and model family. The expansion to six tasks evaluates nine families involving a task held out from calibration plus the three anchors, for 12 families and 36 surfaces across families and seeds. Table 1 therefore counts selected surfaces for each family and seed, not every possible task pair. Prospective specifications. Before training summarization and translation adapters, we froze coarse interaction bins. Pairing either unseen task with safety or instruction was predicted high; pairing with code was predicted middle; pairing with math was predicted low. Summarization+translation was registered as a bridge above the low bin. Success required at least 75% of comparisons between unseen high and low pairs to have the predicted sign. Before training 0.5B

adapters, we separately registered that code+safety would exceed code+math on code and instruction prompts, with at least two of three seed differences positive per stratum. Before the Full FT, Qwen2.5-7B, and Llama-3.1-8B audits, we froze the same two primary strata and required all three seed differences to be positive with the interval from the hierarchical bootstrap excluding zero for a strong persistence result. The 7B and Llama tests evaluated two LoRA adapters directly after confirming equivalence between dense and adapter implementations on the 1.5B pipeline. The OOS bins encode a directional hypothesis from calibration: the new summarization and translation tasks should interact more with instruction or safety than with math, with code intermediate. The dated local specification is summarized in the appendix together with its result trace. Because no immutable external registry is available, we describe the test as prospectively specified rather than externally registered. External evaluation. We evaluate transfer to public prompts at three levels. First, the raw public strata use EvalPlus HumanEval+ and MBPP+ prompt fields for code (Liu et al. 2023), the GSM8K test split for math (Cobbe et al. 2021), AlpacaEval instructions (Dubois et al. 2024), and a stratified XSTest sample with 30 safe and 30 unsafe prompts for safety (Röttger et al. 2024). Each stratum contains 60 prompts sampled with a frozen seed and recorded with dataset revisions and prompt hashes. Second, a format bridge diagnostic reruns the identical 60 public code prompts inside the Alpaca instruction wrapper used by response-only training. Third, for execution behavior, we generate greedy code on all 164 HumanEval+ and 378 MBPP+ prompts, save raw generations and audits of EOS and length caps, sanitize with the official EvalPlus pipeline, and evaluate pass@1. Continuous and discrete evaluations answer different questions and are not substituted for one another.

Results We organize the evidence around four questions: where nonadditivity appears across inputs; whether its coarse ordering transfers to unseen pairs; whether it survives changes in parameterization, scale, and model family; and whether it transfers across prompt formats and execution metrics.

Functional Interference Depends on the Input On Qwen2.5-1.5B, norm matching does not remove the core family contrast. Table 2 compares code+safety with code+math, which share the same code checkpoint within each seed. This stratum dependence shows that neither pair identity nor input distribution alone determines compositionality. The effects at the natural radii and after norm matching are similar on code and instruction prompts, ruling out vector norm as a sufficient explanation. A cached reporting audit reproduces the primary estimator exactly and finds the same sign pattern when ratios are averaged across prompts. The magnitudes depend on the estimator; full numerator, denominator, and results for each seed appear in the appendix. The contrast is not produced by a smaller denominator: on code

0.5B

1.5B

7B

Raw public

Wrapped

(a) Structure depends on prompt stratum

Pass@1

(b) Boundary depends on prompt format Raw public code

Math

Raw public instruction Raw public safety

Code

Raw public math Instruction

Wrapped public code HumanEval+ pass@1

Safety

MBPP+ pass@1 0

2

4

6

8

10

12

14

−2.5

Matched contrast (percentage points)

0.0

2.5

5.0

7.5

10.0

12.5

15.0

External contrast (percentage points)

Figure 2: Functional interference depends on input stratum, prompt format, and evaluation metric. (a) Contrasts between code+safety and code+math after norm matching across controlled prompt strata for Qwen2.5-1.5B and 0.5B. Points are means and bars are 95% intervals from a hierarchical bootstrap over prompts within seeds (n = 3 training seeds). (b) Raw public prompts preserve the continuous contrast on code, instruction, and safety inputs, whereas an Alpaca wrapper collapses the contrast on the same public code prompts. Pass@1 does not robustly reproduce the contrast.

Stratum

C+M C+S

Math Code Instruction Safety

2.88 2.98 +0.10 [−0.83, 0.91] 2.53 8.49 +5.96 [4.14, 7.93] 1.74 9.31 +7.58 [5.59, 9.69] 4.24 6.20 +1.97 [−0.03, 4.14]

Contrast [95% CI]

Table 2: Qwen2.5-1.5B interaction ratios R after norm matching (Eq. 5) and their contrast, in percentage points. C+M denotes code+math and C+S denotes code+safety. Brackets are 95% intervals from the hierarchical bootstrap. prompts the numerator rises from 0.00352 to 0.01663, a factor of about 4.7, while the denominator rises from 0.14048 to 0.19728, a factor of about 1.4. On math prompts the absolute interaction is not zero, but numerator and denominator increase in approximate proportion, so the normalized contrast is near zero.

Prospective Directional Test on Unseen Pairs The expansion to six tasks contains 18 checkpoints and evaluates 12 prospectively specified task pair families across three seeds, yielding 36 surfaces. Nine families contain at least one task held out from calibration, and three are calibration anchors. We call the nine families involving held out tasks OOS pairs. Their parameter directions show weak but detectable task structure: the difference between mean within task and between task cosine is 0.0184 (p = 0.0002), silhouette is 0.0122 (p = 0.0002), and 1-NN task accuracy is 0.444 (p = 0.0062) under permutation of task labels. These small absolute values motivate testing functional interactions directly. After unblinding, all 8/8 prospectively specified comparisons between high and low bins have the predicted sign

Registered bin

Task pair family

High Bridge Middle High High High Low Middle Low

safety+translation summarization+translation code+summarization instruction+summarization instruction+translation safety+summarization math+summarization code+translation math+translation

Mean R 16.62 12.22 11.10 10.98 10.35 9.40 8.52 7.08 6.06

Table 3: Prospectively specified ordering of unseen families on the 1.5B surface after norm matching. Scores average three seeds and six prompt strata and are percentages.

when averaged across six strata; including the prospectively specified code+math low anchor yields 12/12. The score is descriptive because pair comparisons share checkpoints. Fine ranking is less reliable: high exceeds middle in only 5/8 comparisons. Among OOS families, ordinal Spearman correlation is ρ = 0.571 (n = 9, nominal p = 0.108); including prospectively specified calibration anchors gives ρ = 0.697 (n = 12, nominal p = 0.0118). The anchor-inclusive result partly depends on the anchors used to define the bins and is therefore reported as a secondary analysis. Under an exact design-preserving randomization test over task labels, the observed 8/8 corresponds to p = 0.083, so we treat the outcome as prospective directional evidence rather than a definitive inferential test. The comparison bins and success rule were recorded before the summarization and translation adapters were trained; the appendix summarizes that specification and its trace from predictions to results.

(a) Preregistered unseen-family ordering

(b) Ordering is input-conditioned 75% criterion 16.62

Safety + Translation

37.5

Math

12.22

Summarization + Translation Code + Summarization

11.10

Instruction + Summarization

10.98

50.0

Code

Instruction

100.0

Safety

100.0

Summarization

100.0

10.35

Instruction + Translation

9.40

Safety + Summarization

8.52

Math + Summarization 7.08

Code + Translation

High Bridge

6.06

Math + Translation 0.0

2.5

5.0

7.5

10.0

12.5

15.0

Middle Low 17.5

75.0

Translation

0

Mean norm-matched interaction ratio (%)

20

40

60

80

100

High > low comparisons (%)

Figure 3: A prospectively specified directional score transfers to unseen pairs, but depends on the input. (a) Mean interaction ratio after norm matching for nine task pair families containing at least one task held out from calibration. Colors denote bins frozen before the summarization and translation adapters were trained. Values average three seeds and six prompt strata. The high bin exceeds the low bin in 8/8 unseen comparisons, while the finer high versus middle ordering holds in 5/8. (b) Fraction of comparisons between high and low bins with the predicted sign within each prompt stratum. The dashed line is the prospectively specified 75% success threshold. Condition

Raw C

Norm C [95% CI]

Outcome

raw math raw code raw instruction raw safety wrapped code pass@1 (HE+) pass@1 (MBPP+)

+0.01 +8.09 +10.17 +3.16 +1.00 n/a n/a

+0.36 [−0.59, 1.31] +6.91 [3.71, 10.87] +11.09 [8.34, 13.94] +3.96 [1.24, 6.93] +0.34 [−0.16, 0.86] +4.14 [−2.44, 12.60] −0.15 [−2.65, 2.38]

strong eq. residual residual residual strong eq. CI includes 0 CI includes 0

Table 4: Summary of public prompt boundaries in percentage points. Raw C is the matched contrast at natural checkpoint radii; Norm C applies core median norm matching. Continuous intervals use a hierarchical bootstrap over prompts within seeds; pass@1 intervals use a hierarchical bootstrap over tasks within seeds. The OOS result still depends on the input. Accuracy for comparisons between high and low bins is 37.5% on math prompts, 50% on code prompts, 100% on instruction, safety, and summarization prompts, and 75% on translation prompts. These differences preclude interpreting the successful aggregate ordering as a global relation in parameter space.

Functional Structure Persists Without Detectable Parameter Clustering The prospectively specified 0.5B test passes both primary strata. As summarized in Table 5, code and instruction contrasts are positive with 3/3 seed differences in each case, at magnitudes close to the 1.5B estimates. Parameter geometry behaves differently. At 0.5B, the cosine gap between task directions is 0.0045 (p = 0.1146), silhouette is −0.0008 (p = 0.0886), and 1-NN task accuracy

is 0.25 (p = 0.3755). The functional contrast persists although these diagnostics do not detect global task clustering at 0.5B. The supported conclusion is persistence without detectable clustering under this audit, not absence of parameter structure.

Stress Tests Preserve the Controlled Functional Hierarchy The same primary contrast survives three stress tests that target different alternative explanations. In the 0.5B audit with full parameter fine-tuning, both primary strata are positive with 3/3 seed differences and hierarchical bootstrap intervals excluding zero (Table 5). The effect is weaker than LoRA on instruction and safety prompts, so parameterization modulates magnitude, but the ordering is not an artifact of rank-16 LoRA. The contrast does not attenuate in this 7B audit, which is in-

Audit

Code

Instruction

0.5B LoRA +6.21 [3.11, 9.04] +5.26 [3.74, 6.70] 0.5B Full FT +4.37 [2.58, 6.27] +2.98 [2.43, 3.54] Qwen2.5-7B LoRA +9.52 [7.14, 12.77] +10.56 [9.23, 11.88] Llama-3.1-8B LoRA +4.87 [3.86, 5.88] +5.30 [4.03, 6.59]

Table 5: Stress test contrasts for Rcode+safety − Rcode+math in percentage points. Brackets are 95% intervals from the hierarchical bootstrap. Each row uses its own frozen protocol and radius after norm matching.

consistent with the simplest version of a capacity-bottleneck explanation. At 7B both controlled contrasts are larger than at 1.5B and positive in all three seeds. The Llama-3.1-8B audit further tests transfer to a second model family. Its geometry, computed directly from adapter factors, shows a difference of 0.0385 between mean within task and between task cosine under permutation of task labels (p = 0.0032). Functionally, Llama-3.1-8B satisfies the frozen strong rule, again with 3/3 seed differences positive and intervals excluding zero (Table 5).

Prompt Format Marks the Public Boundary In each studied public stratum of 60 raw prompts, public math behaves as the intended low anchor: the contrast after norm matching is +0.36 points and satisfies the practical equivalence gate. In contrast, raw public code, instruction, and safety prompts retain positive contrasts of +6.91, +11.09, and +3.96 points after norm matching, respectively (Table 4). These results rule out the broader claim that public prompts generally attenuate the signal. Here C = Rcode+safety − Rcode+math . The frozen practical equivalence region is [−2, +2] points; “strong eq.” requires both the hierarchical 90% interval and every 90% bootstrap interval over prompts within each seed to lie wholly inside that region. The format bridge identifies the source of the discrepancy. When the identical 60 public code prompts are wrapped in the Alpaca instruction template, the contrast after norm matching collapses from +6.91 to +0.34 points and satisfies this strong gate. The boundary therefore depends on prompt format rather than on public provenance alone. A frozen check over sequences gives a second view of the same boundary. Using reference continuations through teacher forcing where available, public code references retain a positive contrast of +3.72 points after norm matching, with interval [2.77, 4.85]. Public math references show a smaller residual of +1.87 points with interval [0.88, 2.93], indicating that the practical equivalence claim for public math is specific to the primary estimand at the first token. Execution behavior is weaker still. With all three seeds, the HumanEval+ difference in absolute factorial pass@1 interaction is +4.14 points with interval [−2.44, 12.60] and bootstrap probability 0.841 of being positive. The corresponding MBPP+ estimate is −0.15 points with interval [−2.65, 2.38]. EvalPlus does not establish transfer to benchmark merge performance; its wide intervals remain compatible with moderate effects.

Static parameter and data-similarity summaries did not reliably predict held out families.

Discussion and Limitations The experiments support a conditional view of task vector semantics. A direction can participate in a reproducible functional relation without forming a clean global parameter cluster, and that relation changes with the input and its serialization. Format is part of the conditioning distribution rather than a more “true” serialization, and the prospective test establishes only a coarse ordering under specified prompt distributions. This boundary gives the method a practical role. Parameter summaries remain inexpensive screening features, but they average away the inputs on which a merge will be used. Functional surfaces can instead validate composition on representative prompt families, while behavioral evaluation remains a separate deployment gate. Geometry is diagnostic rather than dispositive. The stress tests exclude narrow explanations without establishing invariance. Full FT is audited only at 0.5B, and Llama-3.1-8B covers three tasks under one response-only protocol. Three seeds check directional consistency but provide limited distribution-free power. Controlled prompts are diagnostic instruments, and first-token JSD does not reliably predict decoded behavior. Norm-matched endpoints are interventions, and shared checkpoints make correlations across families dependent. Without same-task and randomdirection anchors, ratio values should be read comparatively rather than on an absolute scale. We do not report task accuracy for the individual adapters, so each task label describes training data rather than verified capability. Layer-resolved decompositions of the interaction are the natural next test of a mechanism. Broader impact. Merging task vectors can reduce the cost of maintaining separate models for each task, but unreliable composition can alter capabilities or safety behavior. Our measurements are diagnostics, not safety guarantees. In particular, including an adapter trained for safety does not establish performance on harmful compliance or excessive refusal; deployment requires behavioral and safety evaluation for the intended application. Artifacts. The appendix records the frozen protocols and model revision information. Machine-readable audits and prompt hashes are planned for the public artifact. Llama weights are not redistributed.

Conclusion Task-vector interference is measurable and conditionally generalizable, but not global. Coarse structure survives unseenpair tests, full fine-tuning, Qwen scaling, and a Llama audit, yet varies with input and format and does not reliably predict pass@1 interaction. Task-vector composition is best viewed as an input-conditioned functional geometry with explicit validity boundaries, not a universal semantic coordinate system in weight space.

References Achille, A.; Lam, M.; Tewari, R.; Ravichandran, A.; Maji, S.; Fowlkes, C. C.; Soatto, S.; and Perona, P. 2019. Task2Vec: Task Embedding for Meta-Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6430–6439. Aghajanyan, A.; Zettlemoyer, L.; and Gupta, S. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Ainsworth, S. K.; Hayase, J.; and Srinivasa, S. 2023. Git ReBasin: Merging Models Modulo Permutation Symmetries. In International Conference on Learning Representations. Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.; Terry, M.; Le, Q. V.; and Sutton, C. 2021. Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732. Chaudhary, S. 2023. Code Alpaca: An Instruction-Following LLaMA Model for Code Generation. GitHub repository. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; de Oliveira Pinto, H. P.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. 2021. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374. Cheng, R.; Xiong, F.; Wei, Y.; Zhu, W.; and Yuan, C. 2025. Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors. In Proceedings of the 42nd International Conference on Machine Learning. Chronopoulou, A.; Peters, M.; Fraser, A.; and Dodge, J. 2023. AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models. In Findings of the Association for Computational Linguistics: EACL 2023, 2054–2063. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168. Draxler, F.; Veschgini, K.; Salmhofer, M.; and Hamprecht, F. A. 2018. Essentially No Barriers in Neural Network Energy Landscape. In Proceedings of the 35th International Conference on Machine Learning, volume 80, 1309–1318. Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. 2024. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783. Dubois, Y.; Galambosi, B.; Liang, P.; and Hashimoto, T. B. 2024. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. In First Conference on Language Modeling. Entezari, R.; Sedghi, H.; Saukh, O.; and Neyshabur, B. 2022. The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks. In International Conference on Learning Representations. Fifty, C.; Amid, E.; Zhao, Z.; Yu, T.; Anil, R.; and Finn, C. 2021. Efficiently Identifying Task Groupings for Multi-Task Learning. In Advances in Neural Information Processing Systems, volume 34.

Garipov, T.; Izmailov, P.; Podoprikhin, D.; Vetrov, D. P.; and Wilson, A. G. 2018. Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs. In Advances in Neural Information Processing Systems, volume 31. Gueta, A.; Venezian, E.; Raffel, C.; Slonim, N.; Katz, Y.; and Choshen, L. 2023. Knowledge is a Region in Weight Space for Fine-tuned Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023. He, Y.; Hu, Y.; Lin, Y.; Zhang, T.; and Zhao, H. 2024. Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic. Transactions on Machine Learning Research. Hitit, O. K.; Girrbach, L.; and Akata, Z. 2026. A Systematic Study of In-the-Wild Model Merging for Large Language Models. Transactions on Machine Learning Research. Hsu, C.-Y.; Tsai, Y.-L.; Lin, C.-H.; Chen, P.-Y.; Yu, C.-M.; and Huang, C.-Y. 2024. Safe LoRA: The Silver Lining of Reducing Safety Risks When Fine-Tuning Large Language Models. arXiv preprint arXiv:2405.16833. Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. Huang, C.; Liu, Q.; Lin, B. Y.; Pang, T.; Du, C.; and Lin, M. 2024. LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition. In Conference on Language Modeling. Ilharco, G.; Ribeiro, M. T.; Wortsman, M.; Gururangan, S.; Schmidt, L.; Hajishirzi, H.; and Farhadi, A. 2023. Editing Models with Task Arithmetic. In International Conference on Learning Representations. Izmailov, P.; Podoprikhin, D.; Garipov, T.; Vetrov, D. P.; and Wilson, A. G. 2018. Averaging Weights Leads to Wider Optima and Better Generalization. In Conference on Uncertainty in Artificial Intelligence, 876–885. Jang, U.; Lee, J. D.; and Ryu, E. K. 2024. LoRA Training in the NTK Regime Has No Spurious Local Minima. In Proceedings of the 41st International Conference on Machine Learning, volume 235, 21306–21328. Ji, J.; Hong, D.; Zhang, B.; Chen, B.; Dai, J.; Zheng, B.; Qiu, T.; Li, B.; and Yang, Y. 2024. PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference. arXiv preprint arXiv:2406.15513. Jin, X.; Ren, X.; Preotiuc-Pietro, D.; and Cheng, P. 2023. Dataless Knowledge Fusion by Merging Weights of Language Models. In International Conference on Learning Representations. Liu, J.; Xia, C. S.; Wang, Y.; and Zhang, L. 2023. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. arXiv preprint arXiv:2305.01210. Malladi, S.; Wettig, A.; Yu, D.; Chen, D.; and Arora, S. 2023. A Kernel-Based View of Language Model Fine-Tuning. In Proceedings of the 40th International Conference on Machine Learning, volume 202, 23610–23641.

Matena, M. S.; and Raffel, C. A. 2022. Merging Models with Fisher-Weighted Averaging. In Advances in Neural Information Processing Systems, volume 35. Morrison, J.; Smith, N. A.; Hajishirzi, H.; Koh, P. W.; Dodge, J.; and Dasigi, P. 2024. Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging. In Findings of the Association for Computational Linguistics: EMNLP 2024, 15604–15621. Narayan, S.; Cohen, S. B.; and Lapata, M. 2018. Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Ortiz-Jimenez, G.; Favero, A.; and Frossard, P. 2023. Task Arithmetic in the Tangent Space: Improved Editing of PreTrained Models. In Advances in Neural Information Processing Systems, volume 36. Pfeiffer, J.; Kamath, A.; Rücklé, A.; Cho, K.; and Gurevych, I. 2021. AdapterFusion: Non-Destructive Task Composition for Transfer Learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, 487–503. Qwen Team. 2025. Qwen2.5 Technical Report. arXiv preprint arXiv:2412.15115. Ramé, A.; Ahuja, K.; Zhang, J.; Cord, M.; Bottou, L.; and Lopez-Paz, D. 2023. Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization. In Proceedings of the 40th International Conference on Machine Learning, volume 202, 28656–28679. Röttger, P.; Kirk, H.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D. 2024. XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 5377–5400. Sivaramakrishnan, G.; Seetha, S. K. K.; Balaji, K. G.; and Ranganathan, S. B. V. 2026. Understanding Adapter Interference in Language Models: Limits of Parameter-Space Geometry. arXiv preprint arXiv:2606.11262. Standley, T.; Zamir, A.; Chen, D.; Guibas, L.; Malik, J.; and Savarese, S. 2020. Which Tasks Should Be Learned Together in Multi-task Learning? In Proceedings of the 37th International Conference on Machine Learning, volume 119, 9120–9132. Stoica, G.; Bolya, D.; Bjorner, J. B.; Ramesh, P.; Hearn, T.; and Hoffman, J. 2024. ZipIt! Merging Models from Different Tasks without Training. In International Conference on Learning Representations. Stoica, G.; Ramesh, P.; Ecsedi, B.; Choshen, L.; and Hoffman, J. 2025. Model Merging with SVD to Tie the KnOTS. In International Conference on Learning Representations. Tam, D.; Bansal, M.; and Raffel, C. 2024. Merging by Matching Models in Task Parameter Subspaces. Transactions on Machine Learning Research. Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. Stanford

Alpaca: An Instruction-Following LLaMA Model. Stanford Center for Research on Foundation Models. Tiedemann, J. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, 2214–2218. Wang, K.; Dimitriadis, N.; Ortiz-Jimenez, G.; Fleuret, F.; and Frossard, P. 2024. Localizing Task Information for Improved Model Merging and Compression. In Proceedings of the 41st International Conference on Machine Learning. Wortsman, M.; Ilharco, G.; Gadre, S. Y.; Roelofs, R.; Gontijo-Lopes, R.; Morcos, A. S.; Namkoong, H.; Farhadi, A.; Carmon, Y.; Kornblith, S.; and Schmidt, L. 2022. Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy without Increasing Inference Time. In Proceedings of the 39th International Conference on Machine Learning, volume 162, 23965–23998. Yadav, P.; Tam, D.; Choshen, L.; Raffel, C.; and Bansal, M. 2023. TIES-Merging: Resolving Interference When Merging Models. In Advances in Neural Information Processing Systems. Yang, E.; Wang, Z.; Shen, L.; Liu, S.; Guo, G.; Wang, X.; and Tao, D. 2024. AdaMerging: Adaptive Model Merging for Multi-Task Learning. In International Conference on Learning Representations. Yu, L.; Yu, B.; Yu, H.; Huang, F.; and Li, Y. 2024. Language Models Are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. In Proceedings of the 41st International Conference on Machine Learning, volume 235, 57755–57775. Zhao, Z.; Shen, T.; Zhu, D.; Li, Z.; Su, J.; Wang, X.; and Wu, F. 2025. Merging LoRAs Like Playing LEGO: Pushing the Modularity of LoRA to Extremes through Rank-Wise Clustering. In International Conference on Learning Representations. Zhou, L.; Zhao, B.; Yu, R.; and Rodolà, E. 2026. Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success. arXiv preprint arXiv:2601.22285.

Expanded Methods and Reproducibility Additive Counterfactual in Logit Space For a prompt x, let ℓα,β (x) be the logits at the first token for θ0 + α∆a + β∆b . The no-interaction counterfactual is ℓadd (x; α, β) = ℓα,0 (x) + ℓ0,β (x) − ℓ0,0 (x).

(6)

Softmax is applied only after this construction in logit space, ensuring that the counterfactual is a normalized distribution. The interaction for a prompt is Ix (α, β) = JSD(pα,β , padd ). The denominator of the reported ratio is Mx (α, β) = JSD(pα,β , p0,0 ). We average Ix and Mx separately over prompts and interior grid points, then report their ratio; we do not average ratios for individual prompts with unstable small denominators. The grid is exactly {0.0, 0.2, 0.4, 0.6, 0.8, 1.0, 1.2} on each axis. We cache the base and logits along each axis and evaluate all points on the two dimensional surface, but only points with α > 0 and β > 0 enter the primary surface mean. Interior grid points and prompts receive equal weight within each seed. JSD at the first token uses the complete vocabulary and natural logarithms, so it is reported in nats; no filtering based on the denominator is applied. The core median intervention rescales each displacement using its global Frobenius norm, computed by summing squared entries over all included parameter tensors, to the median norm of the core set of rank-16 checkpoints before evaluating the surface. Deltas from fine-tuning of all parameters include embeddings, normalization parameters, and the tied language model head exactly once after deduplicating shared parameters.

Training Protocol for Response Tokens An earlier pilot protocol shared the padding and EOS identifier, which inadvertently masked real EOS targets. The corrected final protocol, response-only-eos-v220260712, tokenizes the prompt and response separately, masks every prompt and padding position in the label sequence, and supervises the response together with a true EOS token. Each encoded example is checked for four invariants: prompt labels are masked, response labels are nonempty, padding labels are masked, and the final real input and label tokens equal EOS. If a response is truncated, its last retained token is replaced with EOS. Long XSum documents are truncated inside the source field before the Alpaca template is assembled, preserving the response marker.

Statistical Estimation Uncertainty intervals use a hierarchical bootstrap with the training seed as the outer unit and prompts sampled within seed. Matched family contrasts reuse the same prompt indices because both families are evaluated on the same prompt set. Reported 95% intervals are the 2.5th and 97.5th percentiles of the bootstrap replicates. We additionally report the three differences across seeds. With only three seeds, an exact sign test cannot yield p < 0.05 under a directional alternative, even when all signs agree. The bootstrap intervals quantify

Item

Frozen value

Base models Examples per task Epochs Optimizer schedule Learning rate Device / effective batch LoRA rank / α / dropout Target modules Seeds Composition grid Controlled prompts

Qwen2.5-0.5B, Qwen2.5-1.5B 3,000 2 cosine; 5% warmup 2 × 10−4 4 / 16 16 / 32 / 0.05 q,k,v,o,gate,up,down projections 42, 123, 456 7 × 7, scales 0 to 1.2 60 per stratum

Table 6: Frozen configuration for training and surface evaluation. uncertainty across the prompt distribution and do not turn prompts into independent training replications. Permutation of task labels holds the learned displacement geometry fixed and permutes task labels while preserving the number of checkpoints assigned to each task. Spearman correlations across families are labeled nominal because task pairs share checkpoints. Exploratory tests across multiple families are not used as confirmatory evidence.

Estimator and Denominator Audit The primary estimator is the ratio of expectations (RoE) for each seed as defined in the main paper. As a cached sensitivity analysis, we also compute the expectation of ratios over prompts (EoR), Ex [Ix /(Mx + 10−12 )], without filtering prompts or changing the bootstrap draws. The audit reproduces all 68 frozen bootstrap summary values exactly at tolerance 10−12 . No cell at the family, stratum, and seed level meets the prospectively specified rule for denominator boundaries. No audited prompt-level denominator fell below 10−4 , and no denominator-based prompt filtering was applied. Table 7 shows that the absolute numerator and denominator vary substantially by family and stratum, so the main result is not inferred from a contrast alone. The cached surface evaluation predates prompt hash fields recorded during generation and has no protocol file. The estimator audit records a hash computed later from the locally archived prompt text but does not claim token provenance from the time of generation. This limits provenance strength, not the exact reproduction check against the frozen bootstrap outputs. Results in formats that machines can read, denominator diagnostics, and the reconstructed protocol accompany the artifact.

Audit in Parameter Space Prospectively Specified Unseen Families The complete ordering of nine families appears in the main paper’s OOS table; the matrix over pairs, strata, and seeds accompanies the artifact in a format that machines can read. The primary comparison counted all registered pairs from the high and low bins, producing 8/8 preregistered signs. Adding the prospectively specified code+math low anchor gives 12/12. The bridge exceeds both low families (2/2),

I¯

Stratum

Family

Math Math Code Code Instruction Instruction Safety Safety

code+math code+safety code+math code+safety code+math code+safety code+math code+safety

0.00228 0.00554 0.00352 0.01663 0.00107 0.01628 0.00197 0.01921

M̄ RoE (%) EoR (%) 0.08114 0.18635 0.14048 0.19728 0.06252 0.17504 0.04696 0.31038

2.88 2.98 2.53 8.49 1.74 9.31 4.24 6.20

2.75 3.30 2.55 8.44 2.02 10.07 4.62 6.31

Table 7: Cached estimator decomposition for the Qwen2.5-1.5B surface after norm matching. I¯ and M̄ are means of the numerator and denominator across three seeds; RoE and EoR are means of the corresponding estimates for each seed. JSD quantities are in nats. Stratum

Estimator Seed 42 Seed 123 Seed 456

Math RoE Math EoR Code RoE Code EoR Instruction RoE Instruction EoR Safety RoE Safety EoR

−0.81 −0.05 +6.09 +5.72 +7.64 +8.31 +1.70 +1.67

+0.75 +0.94 +7.53 +7.97 +9.30 +9.51 +4.17 +3.69

Mean [95% CI]

+0.35 +0.10 [−0.83, 0.91] +0.77 +0.55 [−0.19, 1.13] +4.27 +5.96 [4.14, 7.93] +4.01 +5.90 [3.94, 8.11] +5.79 +7.58 [5.59, 9.69] +6.31 +8.04 [6.06, 10.04] +0.03 +1.97 [−0.03, 4.14] −0.30 +1.69 [−0.26, 3.64]

Table 8: Matched code+safety minus code+math contrasts under RoE (primary) and EoR (sensitivity), in percentage points. Intervals use the same hierarchical bootstrap over prompts within seeds and shared prompt indices. The code and instruction sign pattern is stable across estimators; magnitudes depend on the estimator.

Metric

1.5B, six tasks

0.5B, four tasks

Checkpoints 18 12 Cosine gap 0.0184 (p = .0002) 0.0045 (p = .1146) Silhouette 0.0122 (p = .0002) −0.0008 (p = .0886) 1-NN accuracy 0.444 (p = .0062) 0.250 (p = .3755) Participation ratio 16.68 10.82 Stable rank 13.80 9.22

Table 9: Diagnostics of parameter directions. p values are results from permutation of task labels. Participation ratio and stable rank are descriptive because their attainable values depend on checkpoint count.

whereas high exceeds middle in 5/8 comparisons. Accuracy for high and low comparisons within each prompt stratum is 37.5% (math), 50% (code), 100% (instruction), 100% (safety), 100% (summarization), and 75% (translation). Exact design-preserving randomization test. The only randomization that preserves this design relabels the four calibration-era tasks, whose prospectively specified bins form the multiset {High, High, Middle, Low}. Because two labels are identical there are 4!/2! = 12 distinct assignments, each yielding the same 4 × 2 = 8 high-versus-low family comparisons. Enumerating all 12, the observed assignment is the unique maximum at 8/8; the next highest assignments reach 7/8. The exact one-sided p-value is therefore 1/12 = 0.083, which is also the smallest value this design can attain. The test reports the finite-sample resolu-

Stratum Math Code Instruction Safety

1.5B contrast

0.5B contrast

+0.10 [−0.83, 0.91] +5.96 [4.14, 7.93] +7.58 [5.59, 9.69] +1.97 [−0.03, 4.14]

+0.50 [−0.08, 1.31] +6.21 [3.11, 9.04] +5.26 [3.74, 6.70] +5.52 [3.21, 8.26]

Table 10: Interaction ratio contrasts between code+safety and code+math after norm matching, in percentage points with 95% intervals from the hierarchical bootstrap. tion of the prospective specification rather than a definitive inferential result. The enumeration script and its machinereadable outputs, listing every assignment with its hit count, are included with the artifact.

Matched Contrast Across Scales At 0.5B, all three seed differences are positive for code (+7.45, +3.27, +7.91 points) and instruction (+6.17, +3.70, +5.90 points). This test was frozen before inspecting the 0.5B response-only surfaces.

Public Prompts, Format, and Execution Boundary The public prompt analysis evaluates raw prompt sets for code, math, instruction, and safety. The public math contrast after norm matching is +0.36 points and satisfies the practical equivalence gate. Raw public code, instruction, and safety prompts retain positive contrasts of +6.91, +11.09, and +3.96 points after norm matching, respectively. A for-

mat bridge diagnostic then wraps the identical 60 public code prompts in the Alpaca instruction template; the contrast after norm matching collapses to +0.34 points. The public boundary depends on prompt format rather than reflecting a simple failure on public distributions. The secondary measurement over sequences was frozen before any result on public prompts was observed and uses reference continuations through teacher forcing where such references are available. On public code references, the contrast over sequences after norm matching is +3.72 points with interval [2.77, 4.85], reproducing the direction at the first token with lower magnitude. On public math references, the contrast over sequences after norm matching is +1.87 points with interval [0.88, 2.93]. The equivalence conclusion for public math is specific to the primary estimand at the first token; at sequence resolution, a small but detectable residual remains. For execution evaluation, let S0 , Sa , Sb , Sab be pass@1 for the base, endpoints for each task, and merged model. The signed factorial interaction is IS = Sab − Sa − Sb + S0 . We compare |IS | between code+safety and code+math. Across three HumanEval+ seeds, the absolute matched contrast is +4.14 points with interval [−2.44, 12.60]; MBPP+ is approximately zero. These outcomes do not support a claim about predicting benchmark performance.

Compute and Artifact Plan The 1.5B replication and expansion were run in a Linux container on an NVIDIA GeForce RTX 5090 with 32 GB memory; environment validation logs record the exact PyTorch/CUDA versions and bfloat16 support. The initial 0.5B pilot used an NVIDIA A100 with 40 GB memory. Exact environment lock files, configurations, prompt strata, prospective prediction documents, figure source tables, analysis scripts, and checkpoint identifiers will accompany the public artifact. Because Llama-3.1 weights are governed by the Llama 3.1 Community License, the artifact will not redistribute the base checkpoint; it will provide the locked model revision, adapter metadata, and analysis outputs. The released artifact will distinguish the earlier pilot that masked EOS from the final response-only protocol.

Claim Boundary Checklist The evidence supports functional interaction that depends on the input, coarse ordering of unseen families, persistence under fine-tuning of all parameters at 0.5B, persistence across Qwen2.5 scales, and a directed audit on Llama-3.1-8B as a second architecture. It does not support a global semantic coordinate system, universal architecture generality, broad generality under fine-tuning of all parameters, prediction of interference for individual examples, or prediction of downstream performance. Adapters trained for safety are used as task families; the study does not measure harmful compliance or excessive refusal and makes no claim of guaranteed safety.

Related documents

• Task-Aware Identifiability: Observation Quotients, Statistical Geometry, and Representation Accessibility#670476
• Task-Aware Identifiability: Observation Quotients, Statistical Geometry, and Representation Accessibility#668411
• Excess capacity and representational sharing: A lesson from computational accounts of cognitive control#691277
• Social Circuits behind Multi-agent Echo Chambers#1128714
• A hierarchical integration model of effort perception: peripheral, central, cognitive, temporal, and applied mechanisms revealed through blood flow restriction.#436875