PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes

Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets grow, they often add semantically redundant samples rather than expanding semantic coverage, which deteriorates the quality of the whole dataset. Exis