arXiv (OAI Expanded)
open access
Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat
Learned gates can approximate sparse attention patterns on frozen transformers, yet provide limited benefit over random gates when trained jointly with the model. We investigate this difference in a controlled 31M-parameter transformer and attribute it to routing absorption: model representations co
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning#1129755
- MoRE: Mixture of Reused Experts#1013731
- Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusters#1118989
- Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusters#1108722
- RETENTION: Resource-Efficient Tree-Based Ensemble Model Acceleration with Content-Addressable Memory#171590
Record · ID 1110190
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive