arXiv (All)
open access
Do Value Vectors in Deep Layers Need Context from the Residual Stream?
The success of the transformer architecture is in large part due to its use of attention layers. An attention layer follows the standard neural network paradigm: it takes the residual stream as input and thereby produces context-dependent query, key, and value vectors. However, we find that model pe
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- Softmax Attention on Gaussian Mixtures: Linear When It Can, Selective When It Must#1249078
- N-gram-like Language Models Predict Naturalistic Reading Time Best#1078962
- Asymptotic Analysis of Discrete-Time Self-Attention Dynamics in Transformers#1297870
- The Ultimate Image#60514
- In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification#1411919
Record · ID 1459212
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive