arXiv (All)
open access
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
Large language models are typically trained under uniform token weighting, which allows frequent and low-information tokens to dominate learning and can increase the tendency to memorize surface-level text spans. To address this, we present an information-weighted cross-entropy loss that rescales to
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models#820996
- SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance#819242
- SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance#809250
- Unspecified Shared-Feature Bridging: Like in Understanding Formation#694003
- Typological Feature Prediction with Large Language Models: An In-Context Learning Approach#926661
Record · ID 998595
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive