arXiv (All)
open access
Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its success is not well understood. Analyses of stochastic optimization often assume uniformly bounded stochastic gradient variance, yet recent evidence
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- Towards Understanding Momentum Acceleration in River-Valley Loss Landscape#1105516
- Persistent Depth Ordering amid Shifting Block-Bypass Responses in Language Model Pretraining#1156100
- Can a Language Model Learn Facts Continually in Its Weights?#1215427
- From Rollouts to Recipes: Self-Contained Post-Training for LLMs#821357
- A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks#1234202
Record · ID 1186669
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive