arXiv (All)
open access
ExpertMuon-Compass: Alignment-Guided Step Sizes for Mixture-of-Experts Training
Mixture-of-experts (MoE) language models send each token to a few experts, so each expert is trained on a different part of the data, and this part changes during training. With a shared learning rate, Muon applies updates of roughly the same size to expert matrices of the same shape, even when an e
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- AMO: Operator-level Adaptive Muon Orthogonalization#1117421
- EssentialGIN: a new approach for gene essentiality prediction based on graph isomorphism neural networks#1014735
- Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining#820339
- GraPhAI: Neural Networks for Solving Centrosymmetric Crystal Structures.#388756
- Investigating Model Compression for Neural Machine Translation in the Biomedical Domain#1216407
Record · ID 1199414
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive