arXiv (All)
open access
Understanding the Weight Averaging Mechanism in LLM Training for Post-Training Quantization
Large language models (LLMs) are typically pretrained in high precision but increasingly deployed with low-precision post-training quantization (PTQ). Recent studies have shown that using weight averaging during pretraining can improve PTQ performance compared with learning-rate decay, suggesting th
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals#1035852
- ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs#1155194
- It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention#986688
- RFG: Self-Improving Diffusion Large Language Models with Reward-Free Guidance#1199702
- TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models#1217142
Record · ID 1201342
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive