Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives

Understanding language requires tracking entities across discourse - i.e., knowing where things are and how they change, even when not explicitly stated. Whether language models perform such tracking in a human-like fashion remains unclear, in part because existing evaluations rely on artificial tas