Preprint. Work in progress.
D EAD W EIGHTS , L IVE S IGNALS : F EEDFORWARD G RAPHS OF F ROZEN L ANGUAGE M ODELS
arXiv:2604.08335v1 [cs.LG] 9 Apr 2026
Marcus Armstrong Navid Ayoobi Arjun Mukherjee Department of Computer Science University of Houston Houston, TX 77204 {miarmstr, nyoobi}@cougarnet.uh.edu, [email protected]
A BSTRACT We present a feedforward graph architecture in which heterogeneous frozen large language models serve as computational nodes, communicating through a shared continuous latent space via learned linear projections. Building on recent work demonstrating geometric compatibility between independently trained LLM latent spaces Armstrong et al. (2026), we extend this finding from static two-model steering to end-to-end trainable multi-node graphs, where projection matrices are optimized jointly via backpropagation through residual stream injection hooks. Three small frozen models (Llama-3.2-1B, Qwen2.5-1.5B, Gemma-2-2B) encode the input into a shared latent space whose aggregate signal is injected into two larger frozen models (Phi-3-mini, Mistral-7B), whose representations feed a lightweight cross-attention output node. With only 17.6M trainable parameters against approximately 12B frozen, the architecture achieves 87.3% on ARCChallenge, 82.8% on OpenBookQA, and 67.2% on MMLU, outperforming the best single constituent model by 11.4, 6.2, and 1.2 percentage points respectively, and outperforming parameter-matched learned classifiers on frozen single models by 9.1, 5.2, and 6.7 points. Gradient flow through multiple frozen model boundaries is empirically verified to be tractable, and the output node develops selective routing behavior across layer-2 nodes without explicit supervision.
1
I NTRODUCTION
The prevailing paradigm for improving language model performance is scaling: larger models, more data, more compute. Yet smaller models trained on specialized corpora routinely outperform larger general models on narrow tasks Abdin et al. (2024); Team (2024), and models trained with different objectives or architectures exhibit systematically different strengths at equivalent parameter counts. The information needed to solve a given problem may already exist, distributed unevenly across a population of existing models — the challenge is aggregating it. Current ensemble and multi-agent approaches pursue this at the output level: routing queries to specialized models Jiang et al. (2024), ensembling token distributions Jiayi et al. (2023), or having models critique each other’s responses Wu et al. (2023); Moura (2023). These methods treat each model as a black box. The geometric structures in LLM latent spaces that encode semantic relationships, reasoning patterns, and factual knowledge remain inaccessible to other models in the ensemble. Armstrong et al. (2026) showed this need not be the case: independently trained LLMs converge to geometrically compatible latent spaces, and a linear projection suffices to translate activations between heterogeneous architectures. We exploit this compatibility in a new direction — not for inference-time correction between two models, but as a differentiable communication medium for a feedforward graph of frozen LLMs. Learned projection matrices translate each node’s hidden state into a shared latent space; the aggregate signal is injected into downstream nodes via residual stream hooks; and the entire system is trained end-to-end via backpropagation through the frozen model boundaries. Only the projection matrices and a lightweight cross-attention output node are updated — approximately 17.6M parameters against 12B frozen. 1
Preprint. Work in progress.
We make four contributions: 1. Tractable gradient flow. Gradients flow cleanly through multiple frozen LLM boundaries, with layer-1 projection matrices receiving approximately 13% of the gradient signal at the output node — attenuated but sufficient for learning, with skip connections providing no additional benefit. 2. Strong benchmark performance. The graph achieves 87.3% on ARC-Challenge, 82.8% on OpenBookQA, and 67.2% on MMLU, outperforming the best single constituent model by 11.4, 6.2, and 1.2 percentage points respectively. 3. Parameter-matched ablations. On all three benchmarks, the graph outperforms learned classifiers trained on the best single model’s frozen representations with a matched or larger parameter budget — by 9.1, 5.2, and 6.7 points. The gains arise from the communication mechanism, not the classifier. 4. Emergent selective routing. Without supervision, the output node develops asymmetric attention over the two layer-2 nodes, preferentially weighting Phi-3-mini’s representations — a pattern consistent with the geometric regularity induced by its synthetic training corpus.
2
R ELATED W ORK
Activation steering and residual stream intervention. Prior work has shown that transformer behavior can be controlled by directly manipulating internal activations without modifying weights. Turner et al. (2024) demonstrated that fixed bias vectors reliably shift model behavior across behavioral dimensions, and Zou et al. (2023) showed that interpretable concept directions generalize across tasks and model families. These methods establish the residual stream as a writable medium — a foundational assumption of our injection mechanism. Armstrong et al. (2026) extended this to cross-architecture settings, showing that a linear projection suffices to translate activation vectors between independently trained LLMs and that injecting translated representations corrects model behavior without weight updates. Three of their findings directly inform our design: the affine compatibility of cross-architecture latent spaces (justifying linear projection matrices), the superiority of dense over sparse projections (R2 ≈ 0.50 for Ridge vs. ≈ 0.01 for Lasso), and domain orthogonality of offline-fitted projections (motivating end-to-end training against a task loss). Our work generalizes theirs from static two-model steering to end-to-end trainable multi-node graphs, transforming geometric compatibility from an analytical tool into a differentiable communication medium. Model composition and ensemble methods. Mixture-of-experts architectures Jiang et al. (2024) route tokens to specialized subnetworks within a single model, while multi-agent frameworks Wu et al. (2023); Moura (2023) coordinate LLMs through natural language. Both operate at the token level; internal representations remain inaccessible across model boundaries. Model stitching Bansal et al. (2021) demonstrated that independently trained networks can be composed via a learned linear adapter, establishing affine compatibility as an empirically grounded prior. We extend this from pairwise adapters to a full graph topology with jointly optimized projection matrices. The Platonic Representation Hypothesis Huh et al. (2024) provides theoretical grounding: neural networks converge to a shared underlying representation of reality, with architecture-specific differences reducible to coordinate transformations. Frozen backbone adaptation. Parameter-efficient methods such as LoRA Hu et al. (2021) and prefix tuning Li and Liang (2021) adapt a single frozen model with minimal parameters, but do not address how multiple frozen models might communicate. Bai et al. (2025) showed that frozen LLM blocks act as gradient coherence rectifiers, producing stable gradient flow through frozen boundaries — directly addressing the theoretical concern about our architecture’s trainability. Yin and Wang (2025) studied multi-node LLM computation graphs and recommended skip connections and auxiliary losses to mitigate gradient attenuation; our empirical analysis finds neither necessary at two-layer depth (Section 5). Our work differs from all of the above by composing multiple frozen models into a graph where inter-node communication occurs in a shared continuous latent space learned from the task signal. 2
Preprint. Work in progress.
3
M ETHOD
We define a feedforward graph G = (V, E) where each node v ∈ V is a frozen pretrained LLM and each directed edge (u, v) ∈ E carries a learned projection matrix Wuv ∈ Rds ×du . Only the projection matrices and a lightweight output node are updated during training; all LLM weights θv remain frozen throughout. 3.1
A RCHITECTURE
Layer 1: Diverse Encoding. Three heterogeneous frozen LLMs — Llama-3.2-1B (d=2048), Qwen2.5-1.5B (d=1536), Gemma-2-2B (d=2304) — each receive the input with a distinct analytical framing prefix (“analyze the factual content”, “analyze the reasoning structure”, “analyze the language and framing”). Each node produces a hidden state at relative depth lT = 0.90, extracted at the final token position and L2-normalized: hi =
hidden(x ⊕ pi , θi , lT ) ∥hidden(x ⊕ pi , θi , lT )∥2
i ∈ {1, 2, 3}
(1)
We extract at depth 0.90 because deeper layers yield more fully resolved semantic representations; L2 normalization ensures the projection learns directional alignment rather than magnitude differences. Shared Latent Space.
Each hidden state is projected and averaged into a shared vector: z1 = 31
3 X
Wi hi
Wi ∈ Rds ×di ,
ds = 1024
(2)
i=1
ds = 1024 is deliberately neutral — not equal to any constituent model’s hidden dimension — so every projection must perform genuine geometric work rather than approximate an identity mapping. Layer 2: Injection and Refinement. z1 is injected into the residual stream of each layer-2 node — Phi-3-mini (d=3072) and Mistral-7B (d=4096) — at relative depth lS = 0.75. Since ds = 1024 ̸= dj , we linearly interpolate z1 to the target dimension before injection; this interpolation is differentiable, preserving gradient flow to the layer-1 matrices. The blend follows the magnituderescaling formulation of Armstrong et al. (2026): h′ = (1 − α) hnode + α(z̃1 · ∥hnode ∥2 ) ,
α = 0.25
(3)
where z̃1 is the interpolated injection vector. Magnitude rescaling ensures the injected signal is compatible with the residual stream regardless of dimensional differences. We use α = 0.25; prior work shows reasoning representations become fragile for α ≥ 0.8. Each layer-2 node completes its forward pass and its hidden state is projected: (j)
z2 = Wj+3 h′j
j ∈ {4, 5}
(4)
Output Node.
A cross-attention output node aggregates both layer-2 representations: (4) (5) (4) (5) o = LayerNorm MHA q, [z2 , z2 ], [z2 , z2 ] , ŷ = softmax(Wcls o)
(5)
where q ∈ Rds is a learned query, MHA uses 4 heads, and Wcls ∈ R4×ds maps to the four answer choices. The attention weights implicitly route the prediction between the two layer-2 nodes based on their representations’ compatibility with the learned query. Figure 1 illustrates the complete architecture. 3.2
T RAINING
We minimize cross-entropy loss L = − log ŷc⋆ where c⋆ is the correct answer index. Gradients flow backward through the output node, through W4 and W5 , through the differentiable injection operations, and into W1 , W2 , W3 . Frozen LLM weights receive no gradients. 3
Preprint. Work in progress.
Layer 1
Layer 2
"facts"
Llama-3.2-1B frozen · d=2048
W1
Phi-3-mini Shared
frozen · d=3072
W4
latent space "reasoning"
Input
Qwen2.5-1.5B
z1 W2
frozen · d=1536
Cross-attn (5) [z(4) 2 , z2 ]
1024
ds = 1024
avg. of Wihi 4 heads
Output A/B/C/D
inject =0.25 at lS=0.75
"language"
Gemma-2-2B
W3
learned query
Mistral-7B
W5
frozen · d=4096
frozen · d=2304
Trained: W1 W5 + output node (17.6M params) Frozen: all LLM weights ( 12B params) Frozen LLM Shared latent / cross-attn Output node (trained)
Figure 1: Frozen LLM Graph Architecture. Three heterogeneous frozen layer-1 models encode the input with distinct framings; their hidden states are projected via W1 , W2 , W3 into a shared latent space and averaged to form z1 , which is injected into two frozen layer-2 models at lS =0.75. Layer-2 representations are projected via W4 , W5 into a second shared space, over which a cross-attention output node produces the final prediction. Only the five projection matrices and output node (17.6M params) are trained. We use AdamW Loshchilov and Hutter (2017) with ηW = 10−3 for projection matrices and ηout = 10−4 for the output node, weight decay 10−4 , gradient clip 1.0, effective batch size 8, and cosine annealing. All models are loaded in bfloat16 via HuggingFace Wolf et al. (2020); the full graph fits on a single 80GB A100. Injection hooks are registered immediately before each forward pass and removed afterward to prevent cross-example contamination. Full hyperparameters are in Table 2; trainable parameter counts are in Table 4 (Appendix).
4
E XPERIMENTS
4.1
DATASETS AND BASELINES
We evaluate on three multiple-choice benchmarks covering distinct reasoning demands. MMLU Hendrycks et al. (2021) provides 14,042 training and 1,531 validation questions across 57 subjects (STEM, humanities, social sciences). ARC-Challenge Clark et al. (2018) provides 1,119 training and 1,165 test questions drawn from standardized science tests. OpenBookQA Mihaylov et al. (2018) provides 4,957 training and 500 test questions requiring integration of a core science fact with commonsense inference. Together the three benchmarks cover broad factual recall, standardized scientific reasoning, and fact-integrated inference. We compare against two baselines under the same zero-shot greedy decoding protocol. Singlemodel greedy: each constituent model evaluated individually with no learned components, establishing the per-benchmark ceiling. Parameter-matched learned head: an MLP classifier trained on the frozen hidden states of the strongest single-model baseline at budget matched to the graph 4
Preprint. Work in progress.
network’s 17.6M parameters (16.8M for Mistral-7B on MMLU; 22.8M for Qwen2.5-1.5B on ARC and OpenBookQA — slightly over budget, making this a conservative comparison). Hidden states are extracted at lT = 0.90 and L2-normalized, identical to the graph network’s protocol. This baseline isolates whether gains arise from the communication mechanism or from the learned classifier alone. 4.2
R ESULTS
Table 1: Results on MMLU, ARC-Challenge (ARC), and OpenBookQA (OBQA). Best singlemodel result per benchmark is underlined. The learned head uses Mistral-7B for MMLU and Qwen2.5-1.5B for ARC and OBQA. Method MMLU ARC OBQA Single-model greedy Llama-3.2-1B-Instruct Qwen2.5-1.5B-Instruct Gemma-2-2B-IT Phi-3-mini-4K-Instruct Mistral-7B-Instruct-v0.3
44.0% 59.0% 60.0% 43.0% 66.0%
50.8% 75.9% 67.1% 75.3% 73.9%
56.6% 76.6% 60.8% 62.2% 70.4%
Parameter-matched learned head Head on Mistral-7B (16.8M) Head on Qwen2.5-1.5B (22.8M)
60.5% —
— 78.2%
— 77.6%
Ours Frozen LLM graph (17.6M)
67.2%
87.3%
82.8%
Margins vs. best single model vs. learned head
+1.2pp +6.7pp
+11.4pp +9.1pp
+6.2pp +5.2pp
The graph network outperforms the best single model and the parameter-matched head on every benchmark. The margins against the learned head (6.7–9.1pp) are particularly informative: the head receives a larger parameter budget on ARC and OpenBookQA yet still loses by substantial margins, confirming the gains arise from multi-node communication rather than the classifier. A notable secondary finding is that the learned head on Mistral-7B underperforms Mistral’s greedy decoding on MMLU (60.5% vs. 66.0%), while heads on Qwen modestly exceed greedy on ARC and OpenBookQA. We hypothesize that Mistral’s instruction tuning optimizes its output distribution for multiple-choice generation in a way that is not recoverable from the penultimate representation alone. The graph network circumvents this by using Mistral as a refinement node rather than a representation to classify directly. 4.3
T RAINING DYNAMICS
Figure 2 shows validation accuracy over training. ARC-Challenge converges rapidly — 80.7% after just 50 gradient steps — reflecting the immediate utility of aggregated multi-node representations. OpenBookQA peaks at 85.4% validation mid-epoch 2 and plateaus; training stops at epoch 3. MMLU converges more slowly over 14,042 training examples, peaking at 67.2% at epoch 2 under cosine annealing. Full hyperparameters are in Table 2.
5
A NALYSIS
5.1
G RADIENT F LOW T HROUGH F ROZEN M ODEL B OUNDARIES
We validate gradient flow with a minimal two-node graph: Llama-3.2-1B as source, Qwen2.5-1.5B as destination, connected by a single projection matrix W ∈ R1536×2048 and a four-class output head. We measure gradient norms, the W -to-head gradient ratio, and a permutation control (tar5
Preprint. Work in progress.
Table 2: Training hyperparameters for all graph network experiments. Hyperparameter Value Optimizer LR (W1 –W5 ) / output node Weight decay Effective batch size LR schedule Gradient clip Epochs lS / lT / α ds / output heads Precision / hardware
ARC-Challenge 90%
AdamW 10−3 / 10−4 10−4 8 Cosine annealing 1.0 4 (MMLU, ARC); 3 (OBQA) 0.75 / 0.90 / 0.25 1,024 / 4 bfloat16 / A100 80GB
MMLU (57 subjects)
87.3% (test)
75%
Accuracy (%)
85%
Accuracy (%)
67.2% (epoch 2)
70%
80% 75%
65% 60% 55%
70% 65%
E1
0
100
E2
E3 E4 Val (200-sample) Test (full, end of epoch) Best single model (75.9%)
200
300
Training step
400
50%
500
E1
0
E2
2000
Val (200-sample, mid-epoch) E3 E4 Val (full, end of epoch) Best single model (66.0%)
4000
Training step
6000
Figure 2: Validation accuracy over training on ARC-Challenge (left) and MMLU (right). Violet line: mid-epoch accuracy on 200-example subset. Coral markers: end-of-epoch full evaluation. Dashed line: best single-model baseline. ARC-Challenge reaches 80.7% after 50 steps; MMLU peaks at 67.2% at epoch 2.
get activations randomly shuffled to destroy semantic correspondence while preserving marginal statistics). Results are summarized in Table 3. Gradients flow cleanly: W receives approximately 13% of the output head’s gradient signal — attenuated but well above catastrophic collapse, and comparable to attenuation across equivalent depth in standard feedforward networks. The permutation R2 of −0.243 versus Ridge R2 of 0.299 confirms the projection captures genuine semantic geometry rather than distributional artifacts. Skip connections produce a 1.00× improvement, confirming they are unnecessary at this depth and simplifying the final architecture. 5.2
P ROJECTION M ATRIX G RADIENT N ORMS D URING T RAINING
Figure 3 shows gradient norms for W1 –W5 across 4 epochs of MMLU training. Three findings emerge. No dead nodes. All five projection matrices receive nonzero gradients throughout training, confirming all communication channels remain active. Heterogeneous layer-1 nodes are essential here — identical models would produce symmetric gradients and degenerate to the same solution. 6
Preprint. Work in progress.
Table 3: Gradient flow validation (Llama-1B → Qwen-1.5B). Negative permutation R2 confirms genuine cross-architecture alignment. Metric Value Ridge projection R2 Permutation control R2 W gradient max W gradient mean Grad norm ratio (W / head) Skip connection improvement
0.299 −0.243 2.48 × 10−1 1.81 × 10−3 0.130 1.00× (neutral)
Layer-2 projections receive stronger gradients. W4 and W5 consistently exhibit larger norms than W1 –W3 , as expected: layer-2 matrices feed directly into the output node while layer-1 matrices must traverse two frozen model boundaries first. The ratio is consistent with the 0.130 measured in the validation experiment. Layer-1 projections do not specialize. W1 , W2 , and W3 maintain nearly identical gradient norms throughout training despite receiving different analytical framing prefixes. We attribute this to the averaging operation: if two matrices produce similar projections, the third receives no gradient pressure to diverge. Representational diversity therefore derives from the heterogeneity of the frozen models themselves rather than from learned specialization of the projection matrices — a limitation we discuss in Section 6. 5.3
E MERGENT ROUTING IN THE O UTPUT N ODE
Without any supervision on routing, the output node develops a consistent and substantial asymmetry between W4 (Phi-3-mini) and W5 (Mistral-7B). On MMLU, W4 gradient norms exceed W5 by 10– 30× in early training, settling to 5–10× by epoch 4. On ARC-Challenge the asymmetry is larger early and partially closes by the final epoch. This asymmetry is a direct consequence of the cross-attention weights: a higher weight on W4 means W4 receives a stronger backward signal, so the gradient ratio reflects implicit routing. We attribute Phi-3-mini’s dominant role to its synthetic training corpus, which prior work has shown produces geometrically regular representations more amenable to external redirection Armstrong et al. (2026). The network discovers this property autonomously — without any prior indicating a preference — because Phi-3-mini’s representations generate more consistent gradient signals. W5 remains active throughout (an order of magnitude above zero), indicating Mistral-7B contributes a secondary but nonzero signal. The benchmark-specificity of the asymmetry (larger on ARC than MMLU) is consistent with domain-dependent steerability, though controlled ablations are needed to disentangle this from dataset size and training duration effects.
6
C ONCLUSION
We introduced frozen LLM graphs: heterogeneous pretrained LLMs as computational nodes communicating through a shared continuous latent space via end-to-end learned projections. With only 17.6M trainable parameters against 12B frozen, the architecture achieves 87.3% on ARC-Challenge (+11.4pp over the best single model, +9.1pp over a parameter-matched head), 82.8% on OpenBookQA (+6.2pp, +5.2pp), and 67.2% on MMLU (+1.2pp, +6.7pp). Gradient flow through frozen model boundaries is tractable at 13% of output-node signal strength, and the output node develops asymmetric routing toward Phi-3-mini without supervision. Limitations. Non-specializing L1 projections: W1 –W3 converge to similar gradient norms despite distinct framing prefixes; averaging reduces gradient pressure for differentiation. Representational diversity derives from model heterogeneity rather than learned projection specialization. Single training runs: all results use one run per benchmark; the MMLU +1.2pp margin should be interpreted cautiously. Scheduler instability: cosine annealing interacts poorly with checkpoint resumption, causing oscillating MMLU validation accuracy; warmup-then-constant schedules should 7
Preprint. Work in progress.
Projection matrix gradient norms MMLU full run W1 (Llama-1B) W2 (Qwen-1.5B) W4 dominant W3 (Gemma-2B) (Phi-3-mini)
Gradient norm
0.4
W4 (Phi-3-mini) W5 (Mistral-7B)
0.3 0.2 L1 projections (W1, W2, W3)
0.1 0.0
W5 weaker (Mistral-7B) E1
0
E2
200
400
E3
600
Training step
E4
800
1000
Figure 3: Projection matrix gradient norms over training (MMLU, 57 subjects). W1 –W3 (violet) converge to nearly identical norms, showing layer-1 matrices do not specialize. W4 (Phi-3-mini, coral) consistently dominates W5 (Mistral-7B, teal), reflecting emergent selective routing. All five matrices remain active. be preferred. Multiple-choice only: generalization to open-ended generation and longer contexts is untested. Future directions. The most impactful near-term extension is replacing fixed average pooling with learned attention-weighted pooling over layer-1 nodes, directly addressing the nonspecialization limitation. Beyond that: scaling to deeper graphs via multi-GPU deployment; warmstarting projection matrices from offline Ridge solutions to accelerate convergence; and probing the shared latent space z1 for interpretable geometric structure. The frozen constraint ensures that alignment properties established during pretraining are preserved throughout composition — as the ecosystem of open-weight LLMs grows, treating them as composable graph nodes rather than monolithic systems to retrain becomes increasingly practical.
R EFERENCES M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024. M. Armstrong, N. Ayoobi, and A. Mukherjee. Thinking in different spaces: Domain-specific latent geometry survives cross-architecture translation. arXiv preprint arXiv:2603.20406, 2026. L. Bai, Z. Xiong, H. Lin, G. Xu, X. Xie, R. Guo, Z. Kang, H. Zheng, and H. Kim. Frozen language models are gradient coherence rectifiers in vision transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025. Y. Bansal, P. Nakkiran, and B. Barak. Revisiting model stitching to compare neural representations. arXiv preprint arXiv:2106.07682, 2021. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021. 8
Preprint. Work in progress.
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. M. Huh, B. Cheung, T. Wang, and P. Isola. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987, 2024. A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. de las Casas, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. Q. Jiayi et al. Ensemble of large language models for curated labeling and ranking of student explanations. In Proceedings of the 16th International Conference on Educational Data Mining, 2023. X. L. Li and P. Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), 2021. I. Loshchilov and F. Hutter. arXiv:1711.05101, 2017.
Decoupled weight decay regularization.
arXiv preprint
T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018. J. Moura. CrewAI: Framework for orchestrating role-playing autonomous AI agents. https: //github.com/joaomdmoura/crewAI, 2023. Q. Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2024. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020. Q. Wu, G. Bansal, J. Zhang, Y. Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang. AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arXiv preprint arXiv:2308.08155, 2023. L. Yin and Z. Wang. LLM-AutoDiff: Auto-differentiate any LLM workflow. arXiv preprint arXiv:2501.16673, 2025. A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023.
A
T RAINABLE PARAMETER B REAKDOWN
Table 4 provides the full parameter count for each trainable component of the frozen LLM graph. All five LLM backbone weights are frozen throughout training; only the projection matrices and output node receive gradient updates. The output node’s 4,205,572 parameters break down as follows: the 4-head multi-head attention over ds = 1024 contributes 4 × (10242 /4) × 3 + 10242 ≈ 4,194,304 parameters for the key, query, and value projections plus output projection, and the final linear classifier Wcls ∈ R4×1024 contributes 4,096 parameters plus a learned query vector of 1,024 and LayerNorm parameters.
B
G RADIENT F LOW VALIDATION : F ULL D ETAILS
Section 5 summarizes the gradient flow validation experiment. We provide full experimental details here for reproducibility. 9
Preprint. Work in progress.
Table 4: Trainable parameter breakdown. The five projection matrices and output node together constitute 17,580,036 parameters — approximately 0.15% of the total frozen parameter count across all five constituent models (≈12B). Component Dimensions Parameters W1 (Llama-3.2-1B → shared) W2 (Qwen2.5-1.5B → shared) W3 (Gemma-2-2B → shared) W4 (Phi-3-mini → shared) W5 (Mistral-7B → shared) Output node (attn + classifier)
1024 × 2048 1024 × 1536 1024 × 2304 1024 × 3072 1024 × 4096 —
Total trainable
17,580,036 —
Frozen (all LLM weights)
B.1
2,098,176 1,573,888 2,360,320 3,146,752 4,195,328 4,205,572 ≈12B
S ETUP
We construct a minimal two-node graph consisting of Llama-3.2-1B-Instruct (source, d = 2048) and Qwen2.5-1.5B-Instruct (destination, d = 1536), connected by a single projection matrix W ∈ R1536×2048 and a four-class linear output head H ∈ R4×1536 . The forward pass proceeds as follows: (1) encode a factual question with Llama-3.2-1B, extracting the final-token hidden state at layer 24 of 32 (depth 0.75); (2) L2-normalize the resulting vector hsrc ∈ R2048 ; (3) project via W to obtain z = W hsrc ∈ R1536 ; (4) inject z into Qwen’s residual stream at layer 20 of 28 (depth 0.75) via a forward hook using α = 0.25; (5) pass the question through Qwen with the modified residual stream; (6) extract the final-token hidden state at layer 25 of 28 (depth 0.90); (7) pass through the output head to obtain four-class logits. We initialize W and H with standard normal weights scaled by 0.01. Both are registered as trainable parameters with requires grad=True. The Llama and Qwen weights are loaded with requires grad=False. B.2
C ROSS -A RCHITECTURE A LIGNMENT
Prior to the gradient test, we verify cross-architecture semantic alignment using Ridge regression. We collect 50 paired forward passes (simple factual questions) through Llama-3.2-1B and Qwen2.51.5B, extract final-token hidden states at depth 0.75 from both, and fit a Ridge regression mapping Llama representations to Qwen representations (λ = 1.0). We fit a second Ridge regression with the rows of the Qwen activation matrix randomly permuted (permutation control), destroying semantic correspondence while preserving marginal statistics. Results: Ridge R2 = 0.299; Permutation R2 = −0.243. The negative permutation R2 confirms the alignment reflects genuine cross-architecture semantic geometry rather than distributional properties. This is lower than the R2 ≈ 0.50 reported by Armstrong et al. (2026) for similar-scale pairs; we attribute this to our smaller validation set (50 vs. 817 questions), smaller model scales (1B/1.5B vs. up to 8B), and cross-family pairing (Llama-Qwen vs. intra-family pairings which achieve R2 = 0.684). Critically, Armstrong et al. (2026) find near-zero correlation (r ≈ −0.07) between R2 and behavioral correction rate, and our end-to-end training optimizes for task performance directly rather than R2 , so the lower offline R2 does not imply inferior learned projections. B.3
G RADIENT N ORMS
A single forward-backward pass through the two-node graph produces the following results. All values are computed under a cross-entropy loss against a dummy four-class target. • W gradient maximum: 2.48 × 10−1 • W gradient mean: 1.81 × 10−3 • Output head H gradient maximum: 1.91 × 100 10
Preprint. Work in progress.
• Output head H gradient mean: 1.39 × 10−2 • Gradient norm ratio ∥∇W ∥F /∥∇H∥F : 0.130 The ratio of 0.130 indicates that W , which must receive gradients through one frozen model boundary (Qwen), receives approximately 13% of the signal strength available at the output head. This is consistent with gradient attenuation in standard feedforward networks of equivalent depth and far above the catastrophic near-zero collapse sometimes hypothesized for frozen transformer boundaries. B.4
S KIP C ONNECTION A BLATION
We test whether a direct skip connection from z to the output head (bypassing Qwen entirely) improves gradient flow to W . With skip connection: gradient norm ratio = 0.130 (identical). Without skip connection: gradient norm ratio = 0.130. Improvement factor: 1.00× (neutral). Skip connections provide no benefit at two-layer depth and are omitted from the final architecture.
C
C ROSS -A RCHITECTURE A LIGNMENT Q UALITY
This section provides extended discussion of our offline cross-architecture alignment quality relative to Armstrong et al. (2026), which was removed from the main paper for space. Our Ridge R2 of 0.299 for the Llama-1B → Qwen-1.5B pair is lower than the R2 ≈ 0.50 reported for similar-scale cross-architecture pairs in prior work. We attribute this to three factors. First, our validation set contains only 50 question-answer pairs, while prior work uses 817 TruthfulQA questions for fitting; larger fitting sets reliably improve R2 . Second, our models are relatively small (1B and 1.5B parameters); prior work finds alignment quality increases with model scale, with 8B-scale teachers achieving the strongest results. Third, our Llama-Qwen pairing is cross-family; intra-family pairings (e.g., Llama-8B → Llama-1B) achieve R2 = 0.684, substantially above crossfamily pairs. Importantly, none of these factors undermine the validity of our end-to-end trained projections. Prior work reports a near-zero correlation (r ≈ −0.07) between offline geometric alignment quality (R2 ) and behavioral correction rate at test time, concluding that directional accuracy in relevant subspaces matters more than global variance explained. Our end-to-end training procedure optimizes projection matrices directly against task performance rather than R2 , and therefore implicitly optimizes for precisely the directional accuracy that matters for downstream behavior. The offline R2 of 0.299 establishes that the cross-architecture geometry is real and non-trivial; the end-to-end training then refines the projection toward task-relevant directions that an offline proxy dataset may not capture.
D
O PEN B OOK QA T RAINING C URVE
Figure 4 shows the validation accuracy trajectory for the frozen LLM graph on OpenBookQA. The network reaches 74.0% validation accuracy within 30 gradient steps, crosses the best single-model baseline (Qwen2.5-1.5B, 76.6% test) before the end of epoch 1, and peaks at 85.4% mid-epoch 2. The curve plateaus between epochs 2 and 3 (test accuracy 82.4% → 82.8%), motivating our decision to stop training at epoch 3.
11
Preprint. Work in progress.
Accuracy (%)
85% 75% 65%
Val (mid-epoch) Test (end of epoch) Best single model (76.6%)
55% 0
400
800 Training step
E2
E3
Figure 4: OpenBookQA validation accuracy over training. The network crosses the best singlemodel baseline (dashed) during epoch 1 and peaks at 85.4% mid-epoch 2. Test accuracy at end of epoch 3 is 82.8% (+6.2pp over best single model). Dotted vertical lines mark epoch boundaries.
12