arXiv (All)
open access
Transformer Heads Looking for Order
In this note, we show that the problem of checking, whether a sequence of bits is ordered, is not doable by 1-head 1-layer transformers but is doable by a 2-head 1-layer transformer. Unlike similar previous results, our results assume the model where transformers have an output MLP.
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- General Techniques for Reducing Key-Switching Overhead in Privacy-Preserving Two-Party Transformer Inference#306968
- Unified Position-Invariant Random Access Through Two Compression Layers via Absolute-Offset Coordinates: A Bit-Perfect Device-Resident Proof#303177
- Revisiting Code Debloating with Ground Truth-based Evaluation#120588
- Exploring a Layer-Wise Design Space for KV Cache Eviction#1014745
- S30145 Consistent Maintenance & Inspections Delivery Programme (Phase 2) — National Highways Limited — OVE Arup#1181085
Record · ID 1061226
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive