arXiv (All)
open access
The Price of Token Boundaries: Compression Certificates and Prediction
Pre-tokenisation restricts which text fragments can become prediction units, but its compression cost is obscured when tokenisers are compared only under the same boundaries. We measure this cost by bounding the minimum token count from both sides, with and without a regular-expression boundary rule
Conceptio Open Archive
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
This document is indexed with metadata only: full text is not available in the archive for this record. Open the official source ↗
Related documents
- Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning#662146
- Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning#661096
- Checking Leakage Witnesses versus Certifying Bounded Non-Leakage#1116465
- CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters#1154944
- Evaluation Choices Shape Biomedical ML Claims: A Pediatric Pneumonia Benchmark Case Study#1142156
Record · ID 1130711
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive