Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics

Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics Yen-Shan Chen1,2

Yu Chian Duan1,∗ Chih-En Kuo1,∗ Jian-Bin Wu1,∗ Yun-Nung Chen1 1 National Taiwan University 2 CyCraft AI Lab, Taiwan {r14922018, b12902069}@csie.ntu.edu.tw, kuochihen@gmail.com, b12902071@ntu.edu.tw, y.v.chen@ieee.org ∗

Equal contribution PUBLIC BOARD

arXiv:2608.09638v1 [cs.AI] 10 Aug 2026

Abstract

B A APPROVE ✓ REJECT × one act: B rejects a team he is on

Theory of Mind (ToM) is essential for agent interactions, yet existing evaluations either rely on static scenarios that oversimplify mentalstate reasoning or interactive settings that provide limited diagnostic insight. We present Avalon-ToM-Bench, a fine-grained benchmark that operationalizes ToM through the asymmetric-information mechanics of The Resistance: Avalon. Rather than evaluating endto-end gameplay, Avalon-ToM-Bench decomposes ToM into a 2 × 2 taxonomy spanning epistemic versus motivational reasoning and inference versus action, using human-crafted, perspective-constrained queries. Benchmarking 28 LLMs reveals three insights: First, models show strong game-rule comprehension but weaker ToM abilities. Second, mechanistic analyses via linear probing and activation steering demonstrate that models frequently represent correct mental-state inferences in their hidden states but fail to express them during generation (linear probes recover 77–82% accuracy versus 62–70% from the models’ own chain-of-thought), indicating that ToM limitations are primarily a problem of expression rather than representation. Third, dedicated reasoning training yields substantial improvements whereas test-time chain-of-thought provides only marginal gains (+11.0 versus +1.1 points on average), suggesting that robust ToM depends on a learned reasoning policy rather than increased inference-time deliberation.1

1

Svt APPROVE ✓

Servant ·order 4 Assassin ·order 3 Percival ·order 2 Merlin ·order 1 I see an EVIL player on this team → REJECT Morgana ·order 1 I want to be on the mission → APPROVE

Only Merlin sees Morgana, and Morgana cannot know Merlin is here, so the reject is likely Merlin. → TRUSTS B, DOUBTS A Only a seat that sees evil rejects — and Percival just turned on one of my team (possibly Morgana). → SUSPECTS B is Merlin I see nothing, but they read a lone reject as Merlin. If I reject at random, that signal dies. → REJECTS SOMEONE TOO same board, same act — five different conclusions

Figure 1: Avalon operationalizes theory of mind. Identical events yield divergent reads, requiring models to deduce hidden perspectives, infer intent from tactics, and shape subsequent actions.

Traditional benchmarks (Wu et al., 2023; Shinoda et al., 2025; Gandhi et al., 2023) frequently probe superficial patterns through static, overly simplistic scenarios—such as explicitly querying a character’s emotional state or visual perspective— stripping away the asymmetric information, deception, and multi-party dynamics inherent to real-world social reasoning. On the other hand, while hidden-role games offer rich, interactive environments (Serrino et al., 2019; Light et al., 2023, 2025; Rahimirad et al., 2026), evaluating models strictly on end-to-end performance (e.g., win rates or action prediction) sacrifices interpretability. Such outcome-driven metrics blur the lines between genuine mind-reading and orthogonal skills like strategic planning, persuasion, or logical memory, making it impossible to isolate true ToM deficits from broader statistical biases.

Introduction

The evaluation of Theory of Mind (ToM) (Premack and Woodruff, 1978) in Large Language Models (LLMs) demands a shift from treating the capability as a single, indivisible trait to recognizing it as a spectrum of distinct cognitive abilities. However, prevailing methodologies often miss this nuance. 1

Source code available at: https://github.com/ yenshan0530/Avalon-ToM-Bench.

1

2

To address these limitations, we introduce Avalon-ToM-Bench, utilizing the mechanics of The Resistance: Avalon as a controlled diagnostic environment rather than a gameplay arena. We map ToM across two fundamental axes: the nature of the targeted mental state (Epistemic knowledge versus Motivational intent) and its cognitive execution (passive Inference versus active Action). This yields four distinct quadrants of asymmetricinformation reasoning. To guarantee objective assessment, Avalon-ToM-Bench queries models using highly constrained, human-verified propositions that restrict available evidence exclusively to the designated player’s point of view and the public interaction history, penalizing models that rely on omniscient context. We further verify that these axes are empirically dissociable: under permutation testing, item difficulty loads on the epistemic dimension whereas the benefit of chain-of-thought loads on the orthogonal inference–action dimension, confirming that the taxonomy carves ToM at meaningful joints rather than imposing an arbitrary partition.

Related Work

Theory of Mind and its Evaluation in LLMs. Early LLM studies treated ToM (Premack and Woodruff, 1978) as a monolithic capability probed by short, bias-controlled question-answering (Le et al., 2019). Later benchmarks broadened coverage—conversational asymmetry (Kim et al., 2023), long-form narratives (Xu et al., 2024a), finegrained behaviors (Chen et al., 2024), and higherorder reasoning (Wu et al., 2023)—but still frame ToM as passive reading and entangle epistemic knowledge with motivational intent. Adversarial stress tests expose its brittleness (Shapira et al., 2024; Ullman, 2023), yet why models fail, and how to mitigate it, remains open. Social Deduction Games as Reasoning Environments. Hidden-role games are long-standing testbeds for reasoning under information asymmetry, from pre-LLM game-theoretic agents (Serrino et al., 2019; FAIR et al., 2022) to a fast-growing line of LLM players (Light et al., 2023; Wang et al., 2024; Light et al., 2025; Xu et al., 2024b; Rahimirad et al., 2026). These validate belief inference only through end-to-end game success, conflating it with planning, persuasion, and memory. Gamebased evaluations instead target strategic competence (Duan et al., 2024; Lu, 2025) or score behavioral alignment to human play (Song et al., 2025; Li et al., 2025); either way their gold labels are behavioral rather than the objective correctness of a mental-state deduction.

By cleanly decoupling ToM from general gameplaying heuristics, Avalon-ToM-Bench is, to the best of our knowledge, the first framework to operationalize mental-state reasoning as a multidimensional, testable skill. We benchmark 28 distinct LLMs—encompassing various sizes, reasoning paradigms, and open/closed weights— uncovering three primary insights: • Rule Mastery Does Not Equal Social Intelligence: Models struggle with perspectiveconstrained ToM (performance 50 − 60%) despite recalling Avalon’s mechanics nearperfectly (average rule comprehension accuracy ≈85%). • The Latent Competence Gap: Linear probing and activation steering reveal that models internally compute the correct deduction but fail to surface it—probes recover 77–82% accuracy versus 62–70% from the model’s own chain-of-thought, suggesting that ToM failure may be one of expression, not representation. • Robust ToM Requires Learned Policies, Not Just Deliberation Test-time chain-ofthought yields volatile, marginal gains (+1.1 pts), whereas reasoning-trained models improve consistently (+11.0 pts)—robust ToM depends on a learned reasoning policy, not more inference-time tokens.

Probing and Steering Internal Representations. A complementary interpretability line reads and writes model computation directly: linear probes recover latent task information (Alain and Bengio, 2017; Belinkov, 2022), models encode knowledge their generations omit (Burns et al., 2023; Azaria and Mitchell, 2023), and activation steering shows such directions are causal (Turner et al., 2025; Zou et al., 2023). We bring these tools to ToM to show that mental-state inferences are internally represented yet under-expressed—a mechanistic axis unseen in prior ToM benchmarks. In Appendix C we discuss related work in detail.

3

Avalon-ToM-Bench

Avalon-ToM-Bench treats The Resistance: Avalon—a hidden-role game whose secret roles give each seat asymmetric private information over 2

DURING PLAY one recorded game

Night phase

1

2

Ground truth · 7 seats P1

P2

P3

P4

Merlin

Percival

Servant

Servant

P5

P6

P7

Morgana

Mordred*

Assassin

P2

unknown to that seat

P3

P4

P5

3

Leader proposes P1, P3, P6

5 approve / 2 reject

Merlin's view P1

Day phase

Everyone votes

* Mordred and Oberon are optional rol es

Private views

AFTER PLAY review

P6

P3

t1

tn

2 Review Each Snapshot Post-Game public history

seat filter

? ? ? secretly

H

P (Merlin)

seen by all

hides Mordred

P7

Public result 1 FAIL

Morgana's view P2

turns, votes + stated reasoning

Team plays cards

sees 2 of 3 evils — Mordred stays hid den

P1

Instance derivation

1 Recorded gameplay + Reasoning

3 Draft Each Dataset Instance context History + Perspective (Merlin)

P4

P5

P6

P7

everyone sees proposal, votes, result

knows the evil team, not who Merl in is

… 5 other seats each hold their own view

no one sees who played FAIL

query “Merlin know s P6 is EVIL” gold True/False

find new items by reviewing every seat × every turn

Figure 2: Avalon-ToM-Bench overview. During play (1–2), the authors play and record a real game: hidden roles give each seat an asymmetric private view of the same public events. After play (3), each recorded game is reviewed seat-by-seat and turn-by-turn; combining the public history H with a seat’s role filter P yields one perspective-constrained instance (context, query, gold label), forcing strictly perspective-constrained reasoning.

a shared public history of proposals, votes, and mission outcomes—not as gameplay to be won but as a measurement instrument. We decompose ToM into four abilities on a 2 × 2 taxonomy (§3.1) and probe each with human-authored, perspective-constrained binary statements whose labels are objectively verifiable against the rules, the designated seat’s information, and the public history (Figure 2). The full ruleset, a self-contained “Avalon in brief,” and the role-visibility table are given in Appendix D. 3.1

versarial roles deliberately confusable (Figure 3); the task is to read off what a given seat can legitimately know under these constraints—a purely epistemic inference with no strategic act. Strategic Signaling (Epistemic × Action). The Lady of the Lake holder privately learns a player’s faction and then makes a public, unverifiable claim; the evidence is epistemic but the object of reasoning is a communicative act—revealing, concealing, faking, or decoding that claim. Intention Attribution (Motivational × Inference). Suspicion redirection scenarios reveal no identity directly; the model must abduce a hidden motive from an observed action.

A 2 × 2 Taxonomy of Theory of Mind

We organize ToM along two axes. The first is the nature of the mental state being reasoned about: Epistemic—what a seat (a player’s role-defined vantage point) can perceive or know (beliefs, information access)—versus Motivational—what a seat wants or intends (goals, motives). The second is how that mental state is engaged: Inference, where the target mental state exists from the role and board structure and must be read off with no communicative act involved, versus Action, where a strategic act—a public claim, a sabotage choice— is itself the object of reasoning and must be produced or interpreted as a deliberate move. Crossing the two axes yields four dissociable abilities, each of which is naturally instantiated by a distinct Avalon scenario (Figure 3).

Tacit Coordination (Motivational × Action). The EVIL sabotage protocol (priority Assassin > Morgana > Mordred; Appendix D.4) requires EVIL teammates to converge, without communication, on who plays the FAIL card—a coordinated act that must mesh with teammates’ shared intent. These placements are design-driven—for each cell we selected the Avalon mechanic that most naturally instantiates it—and, crucially, not arbitrary: our decomposition analysis (§4.2) finds that item difficulty loads on the Epistemic axis while the benefit of chain-of-thought loads on the orthogonal Inference/Action axis, confirming that the taxonomy carves ToM at empirically dissociable joints.

Perspective-Taking (Epistemic × Inference). The Merlin–Percival–Morgana structure makes ad3

Inference

Action

1 Perspective-Taking

2 Strategic Signaling

Epistemic

Merl in – Percival – Morga na

Percival

sees

Lady of the Lake – Oberon

Merlin / Morgana

picks

the pair + 1 servant

Oberon

unlabelled pair

Merlin sees all evil

EVIL card

claims

“GOOD”, publicly

blind to his own team

vote: 1 reject, 2 approve

every evil seat now reads Oberon = evil

✓ Rejecter = Merlin → the other is Morgana. ✗ Rejecter = Morgana, faking Merlin.

✓ Every evil seat now reads O beron as evil. ✗ Only the inspected player learns it.

Key ToM concept: Percival should be abl e to reason that Merlin sees Morga na, bu t Morgana cannot see Merli n — the mimicry i s on e-way.

Key ToM concept: Oberon should infer that the other evils see each o ther, so they know tha t card was evi l — the lie outs Oberon to all of them.

3 Intention Attribution

4 Tacit Coordination

suspicion redirection – Mordred

evil sabotage protocol – no talking

steps on

Motivational

inspects

Mordred

evil teammate

Merlin

fail order: Assassin > Morgana > Mordred

trust ↑

invisible to Merlin

step on = publicly turn on a teammate

Morgana

team

who plays FAIL ?

sees: { Assassi n or Mordred }

belief unchanged

Assassin

sees: { Morgana or Mordred }

✓ Merlin's trust in Mordred rises. ✗ His teammates start to think he is good.

✓ The Assassin always fails — he outranks both. ✗ Morgana m ay defer: the other must be Assassin.

Key ToM concept: Role knowledg e is fi xed; a pu blic act can move only the good side's beliefs, never a teammate's.

Key ToM concept: The rule resolves only for the top-ranked seat; Morgana must act unsure which evil she is paired with.

good seat

identity unk nown

evil seat

public / rule

Figure 3: The 2 × 2 Theory-of-Mind taxonomy, instantiated in Avalon. The nature of the mental state (Epistemic vs. Motivational) crossed with its cognitive application (Inference vs. Action) yields four dissociable abilities, each realized by a naturally occurring Avalon scenario: Perspective-Taking (Merlin–Percival–Morgana), Strategic Signaling (Lady of the Lake), Intention Attribution (Suspicion Switching), and Tacit Coordination (EVIL Protocol).

3.2

Perspective-Constrained Task Formulation

independently rather than as competing multiplechoice options. A small number of EVIL-Protocol items are observer-level: they test whether a protocol consequence follows from the complete state without assigning a viewpoint, and are assessed directly against R, G, and H.

Let R denote the Avalon ruleset, G the complete game state, H the public interaction history, and s a candidate inference. Most items designate a player i, either through an explicit perspective instruction Pi or directly in the target query. The role-visibility rules induce an information projection Ii = Πi (G) containing only what player i can legitimately know, and the model must evaluate s using evidence restricted to this projection: ŷ = M (R, Pi , G, H, s),

Evaluation. Each instance presents the game setup, relevant history, and one target statement; the gold label and reference rationale are withheld from the model, which outputs a single T RUE/FALSE judgment. We report accuracy and, because some diagnostic subsets are class-skewed, macro-averaged F1 , which exposes any residual T RUE/FALSE prior. All items are run zero-shot with a fixed system/user prompt template (Appendix H); decoding regimes and the full model roster are given in Appendix I.

ŷ ∈ {T RUE, FALSE}.

Although the model is shown the complete state G, valid evidence is confined to Ii and H; it must reconstruct the player’s epistemic state rather than treat every stated identity as accessible knowledge. A role-conditioned statement is labeled T RUE when it is valid under R, Ii , H, and any explicit scenario assumptions, and FALSE if it violates the rules, relies on inaccessible information, or makes an unsupported belief or strategic inference. The evidence and judgment criteria specific to each dimension are detailed in Appendix E, and one fully worked item per dimension is given in Appendix J. Statements from the same scenario are evaluated

3.3

Data Generation and Validation

The benchmark is built from real games rather than synthetic vignettes: the authors play and record full games and then, reviewing each recorded snapshot seat-by-seat and turn-by-turn, draft and independently cross-validate perspective-constrained statements against the rules, the seat’s role-filtered information, and the public history (Figure 2; full pro4

PerspectiveTaking Epistemic Inference

Strategic Signaling Epistemic Action

Intention Attribution Motivational Inference

Tacit Coordination Motivational Action Acc

F1

Acc

F1

F1

Average

Model

Rules

Acc

F1

Acc

F1

Acc

□ Gemini-2.5-Pro □ GPT-5 □ Gemini-2.5-Flash □ Qwen3.6-35B-A3B □ Qwen3-4B-Think □ GPT-5-mini □ Qwen3-8B □ Qwen3-32B □ Qwen3-14B ○ GPT-4o-mini □ Qwen3-4B □ Qwen3.5-9B ○ Phi-4 ○ Gemma-3-27B □ gpt-oss-120B ○ Mistral-7B ○ Llama-3-8B ○ Gemma-3-4B ○ Phi-3.5-mini ○ Llama-3.2-3B □ Ministral-3-14B ○ Llama-3.1-8B ○ Granite-3.3-8B □ Qwen3-1.7B ○ Llama-3.2-1B □ Qwen3.5-2B □ Qwen3-0.6B ○ OLMo-2-7B

97.06 97.06 94.12 91.18 91.18 94.12 97.06 97.06 94.12 97.06 94.12 88.24 91.18 94.12 97.06 85.29 73.53 85.29 91.18 26.47 79.41 73.53 91.18 85.29 47.06 85.29 64.71 70.59

79.41 82.35 77.27 66.18 72.73 69.12 68.42 64.18 61.76 63.24 54.84 68.63 63.24 61.76 63.24 69.12 63.24 67.65 55.88 68.89 70.59 51.47 51.47 54.10 54.41 55.88 49.25 48.53

78.50 82.29 77.27 65.99 72.50 69.11 67.61 63.98 60.54 63.16 54.79 64.71 61.84 59.52 62.25 67.54 60.12 67.19 54.46 67.59 69.94 40.68 43.89 51.91 42.45 48.33 49.24 35.03

87.14 83.57 86.33 80.00 80.58 76.43 77.10 77.94 80.71 75.71 76.87 66.94 72.86 72.14 61.43 64.29 67.86 67.86 66.43 59.13 64.29 60.29 64.29 67.39 61.43 56.43 62.77 53.57

87.08 83.12 86.33 79.39 80.38 75.37 77.03 77.90 80.55 75.63 76.85 63.70 72.17 71.73 58.00 61.11 65.97 67.84 66.04 59.02 61.11 54.64 60.37 67.00 61.04 46.22 62.70 40.81

100.00 100.00 96.67 96.67 95.00 94.99 96.67 96.67 85.00 85.00 86.67 86.43 85.11 84.86 89.83 89.76 83.05 83.01 83.33 83.31 86.27 86.25 80.00 78.02 78.33 77.58 75.00 73.77 81.67 81.41 76.67 76.00 81.67 81.66 71.67 71.66 78.33 78.33 78.43 78.40 55.00 49.98 76.36 74.73 65.00 61.10 52.54 51.41 68.33 66.22 65.00 61.10 61.67 61.40 56.67 46.65

92.14 91.43 88.41 88.57 84.89 89.29 87.40 82.01 83.45 77.86 78.03 79.70 80.00 77.14 77.86 73.57 67.14 70.71 72.86 58.33 70.71 64.12 66.43 72.66 62.14 63.50 60.00 62.86

92.13 91.38 88.29 88.51 84.84 89.19 87.34 81.98 83.28 77.80 78.00 78.61 79.85 76.84 77.53 72.21 65.08 70.70 72.40 55.56 69.20 59.56 62.56 72.34 57.78 60.63 59.97 57.91

89.67 88.50 86.75 82.85 80.80 80.37 79.51 78.49 77.25 75.04 74.00 73.82 73.61 71.51 71.05 70.91 69.98 69.47 68.38 66.20 65.15 63.06 61.80 61.67 61.58 60.20 58.42 55.41

89.43 88.36 86.72 82.64 80.68 80.03 79.21 78.40 76.84 74.98 73.97 71.26 72.86 70.46 69.80 69.21 68.21 69.35 67.81 65.14 62.56 57.40 56.98 60.67 56.87 54.07 58.33 45.10

Average

84.77

63.46

60.80

70.42

68.54

78.21

75.83

74.70

71.98

70.26

77.01

Table 1: Model performance on the full balanced set (408 items, 50% True per ability). For each ToM ability we report accuracy (Acc) and macro-F1 (F1). Marker: □ / ○ = reasoning / non-reasoning; filled / outline = proprietary / open-weight. Rules: full game-rules exam. Average is the macro-average over the four abilities. Darker shading = higher; best per column in bold; rows sorted by average accuracy.

tocol in Appendix G). In total it comprises 408 independently evaluated binary statements across 97 scenarios and the four dimensions, near-balanced at roughly 50% T RUE, plus a 34-item rule-recall control set—not counted among the 408—that isolates basic rule recall from perspective-sensitive reasoning (per-dimension statistics in Appendix F).

set (408 items, 50% T RUE per ability), so a residual T RUE/FALSE prior surfaces as an Acc–F1 gap. A companion rules exam tests recall of Avalon’s mechanics, separating game knowledge from reasoning about minds. 4.1

Models Read the Rules but Not the Mind

The leaderboard in Table 1 reveals three findings:

4

Experiments

A wide, structured spread. Proprietary reasoning models lead (Gemini-2.5-Pro 89.7, GPT-5 88.5, Gemini-2.5-Flash 86.8), the weakest open models fall to the mid-50s, and class means track capability: 84.1 (proprietary), 71.9 (open reasoning), 66.5 (open non-reasoning). Reasoning training and scale both help, yet no open model reaches the proprietary frontier and even the best leave headroom on the harder abilities.

We evaluate 28 models spanning proprietary and open-weight families, reasoning-trained and standard instruction-tuned variants, and sizes from 0.6B up. Each is prompted zero-shot with a fixed template—game setup, public history, and one perspective-constrained statement—and returns a T RUE/FALSE judgment (§3.2). We report accuracy and macro-F1 per ability on the full balanced 5

Difficulty

the Epistemic axis has no significant effect (−1.2 pp, p = 0.71). This double dissociation confirms that the taxonomy partitions ToM along meaningful, empirically separable dimensions rather than imposing an arbitrary grouping: what makes an item hard and what deliberation can fix are controlled by different axes of the design. A significant difficulty interaction (−9.3 pp, p = 0.027) further indicates that Perspective-Taking—epistemic and inference—is harder than the sum of the two main effects would predict, consistent with it being the single hardest cell.

CoT benefit

Effect

pp

p

pp

p

Epistemic vs. Motivational Inference vs. Action Interaction

−8.5 −3.2 −9.3

<0.001 0.14 0.027

−1.2 −7.9 −4.8

0.71 0.008 0.45

Table 2: The two taxonomy axes control different things. Per-axis main effects on difficulty and CoT benefit (full set; effect in pp, permutation-test p).

Rule mastery is not social intelligence. Almost every model answers the rules exam well (mean ≈83.3%; most ≥88%) yet scores far lower on ToM items that apply those same rules from a constrained point of view. The failures are thus deficits of mental-state reasoning, not missing domain knowledge—and a benchmark that did not separate the two would conflate them.

5

Beyond ranking models, Avalon-ToM-Bench is designed to localize where and why Theory of Mind breaks. We report two analyses, conducted on the open-weight models whose internal states we can access. The first (§5.1) shows that the correct answer is present in a model’s internal representation yet under-expressed at its output: ToM failure is largely a problem of expression, not representation. The second (§5.2) shows that test-time chain-ofthought does little to close this gap, whereas dedicated reasoning training does—implicating the learned reasoning policy rather than the amount of inference-time deliberation.

The abilities are not one skill. Averaged over models, accuracy orders as Intention Attribution (78.2) > Tacit Coordination (75.8) > Strategic Signaling (70.4) > Perspective-Taking (63.5), with Perspective-Taking consistently hardest and Intention Attribution easiest across the roster. ToM here behaves as dissociable abilities rather than a single trait—which we make precise next. 4.2

Ablation Studies

The Taxonomy Carves ToM at Dissociable Joints

5.1

The per-ability ordering could reflect an arbitrary difficulty ranking. To test whether the two axes of our taxonomy (§3.1) truly capture distinct structure, we decompose two per-cell quantities over the 2 × 2 grid—item difficulty (mean accuracy) and the benefit of chain-of-thought (§5.2)—into main effects for the Epistemic/Motivational axis and the Inference/Action axis, and assess each effect with a permutation test that shuffles cell labels. The two quantities load on orthogonal axes (Table 2). Difficulty is driven by the Epistemic/Motivational axis (−8.5 pp, p < 0.001): epistemic abilities (Perspective-Taking, Strategic Signaling), which demand reconstructing what another seat can perceive, are reliably harder, while the Inference/Action axis has no significant effect on difficulty (−3.2 pp, p = 0.14). The benefit of chain-of-thought shows the mirror-image pattern: it is governed by the Inference/Action axis (−7.9 pp, p = 0.008)—deliberation helps the Action abilities, where a strategic move must be produced or decoded, more than the Inference abilities—while

Internal Representations as Stronger Predictors of ToM

If low accuracy reflected an inability to represent the answer, the answer should be as absent from a model’s internal states as from its output. We find the opposite through three distinct perspectives: A linear probe beats the model’s own reasoning. We train logistic probes on the lasttoken hidden states of the Qwen3 models (every layer, five-fold cross-validation) to predict the correct T RUE/FALSE label, and compare each probe against the same model’s chain-of-thought output on the identical items. Interestingly, the probe is substantially more accurate (≈77–82%) than the model’s own output (≈62–70%), and the answer becomes linearly decodable already in the middle layers (Figure 4). The model computes the ToM answer internally but does not surface it, and the probe–output gap is largest for PerspectiveTaking—the hardest ability. A god’s-eye hint surfaces latent competence. If the answer is present but unexpressed, making it 6

90

Qwen3-1.7B

Qwen3-4B

linear probe (internal) model output

75

+12

-8

accuracy (%)

+16

80

+0

+8

+23

probe accuracy (%)

100

+18 +16

70 60

70 Qwen3-1.7B Qwen3-4B

65 60 55

50

50

40 Perspective Taking

Strategic Signaling

Intention Attribution

Tacit Coordination

Perspective Taking

Strategic Signaling

Intention Attribution

0.0

Tacit Coordination

(a) Probe − output gap, per ability.

0.2 0.4 0.6 0.8 relative depth (layer / total)

1.0

(b) Layer-wise decodability.

Figure 4: ToM failure is a problem of expression, not representation. A linear probe decodes the correct answer (a) far above the model’s output and does so in the middle layers (b). Open-weight Qwen3-1.7B/4B.

orig

+GT

∆

models helped

Perspective-Taking Strategic Signaling Intention Attribution Tacit Coordination

61.0 69.2 75.4 73.9

77.4 81.2 87.0 84.3

+16.3 +12.0 +11.6 +10.4

28/28 27/28 27/28 26/28

All abilities

69.9

82.5

+12.6

—

90

model output (baseline)

+ steering (unsupervised)

+ steering (supervised)

linear probe (ceiling) 83

81

80 accuracy (%)

ToM ability

77

82

80

77 74 70

70

69 66

65

60 54

50

Table 3: Injecting the ground-truth answer surfaces latent competence. Mean accuracy (%) across all 28 models before (orig) and after (+GT) the correct T/F answer is placed in context, per ToM ability, on the full balanced set (408 items, 50% True per ability). The lift ∆ is largest for Perspective-Taking—the hardest ability— and never backfires: the rightmost column counts models whose per-ability score does not drop, indicating no gods-eye over-claiming.

1.7B

4B model size

8B

(a) Accuracy vs. model size

explicit should help. Injecting the ground-truth answer into the context raises accuracy by +12.6 pp on average across all 28 models, largest again for Perspective-Taking (+16.3; Table 3). Crucially, the intervention never backfires: 26–28 of the 28 models improve on every ability, with no sign of god’s-eye over-claiming. Handed what it already represented internally, a model reasons correctly about what other seats can perceive—indicating that the bottleneck is likely expression, not representation, and suggesting the probing results may generalize to the proprietary frontier that probes cannot reach.

TPR (True)

TNR (False)

Model

before

after

before

after

Qwen3-1.7B Qwen3-4B Qwen3-8B

10.7 69.4 73.8

73.8 83.0 83.5

98.0 71.3 75.2

80.2 79.7 82.7

(b) Recall before / after supervised steering

Figure 5: The represented answer is causally writable. (a) Supervised steering at the best probe layer lifts accuracy onto the probe ceiling at every size; a label-free variant helps only the smallest model. (b) It raises Trueand False-recall, un-skewing Qwen3-1.7B.

linear-probe ceiling at every size (e.g. Qwen3-1.7B 54 → 77, 4B 70 → 81, 8B 74 → 83) and raises both True- and False-recall together—most strikingly un-skewing Qwen3-1.7B from a degenerate 11/98 to a balanced 74/80—so the gain reflects surfaced content, not a threshold shift toward one class. Control directions behave as expected (random ≈ 0, negated < 0). The effect appears to be bounded by the readout that supplies each item’s sign: a fully label-free variant primarily helps the smallest, degenerate model. This serves as a mechanistic existence proof—suggesting the ToM answer may be linearly writable—rather than a deployment recipe.

The represented answer is causally writable. Reading the representation is correlational, so we also write to it. At the best probe layer we add a “T RUE−FALSE” direction—the class-mean difference in activation space, signed per item by the cross-validated probe—to the residual stream, and evaluate on the balanced set (Figure 5). Supervised steering lifts output accuracy onto the 7

Chain-of-thought is highly unpredictable. The small overall improvement hides a lot of instability: using CoT flips the model’s outcome on ∼27% of the questions (fixing roughly 16% but breaking 12%). Therefore, the modest net gain simply means the fixes and breaks almost cancel each other out, rather than showing a reliable boost (Appendix L, Figure 6). An example of a model talking itself out of the correct answer is provided in Appendix K.

Together, probing (correlational), injection (behavioral), and steering (causal) suggest that a model’s internal representation carries the ToM answer more faithfully than its external reasoning. The gap between benchmark accuracy and true ability is one of expression, not representation. 5.2

Chain-of-Thought Prompting is Insufficient for General ToM

Longer reasoning does not help either. Nor does the sheer amount of deliberation. Binning each model’s thinking-on responses by length quartile, accuracy does not rise from the shortest to the longest quartile—it declines (pooled corr(length, correct) = −0.08; Appendix M,Figure 7). This is partly confounded, since harder items elicit longer responses, so we claim only that reasoning length is not a lever on ToM performance, not that length causes errors.

A natural remedy for under-expressed reasoning is to let the model reason more—to think step by step at inference. We find this helps surprisingly little, and that what helps instead is the reasoning policy instilled by training, not the quantity of inferencetime deliberation. (a) Test-time CoT (same weights, Qwen3 family) Model

w/ CoT

w/o CoT

∆

Qwen3-0.6B Qwen3-1.7B Qwen3-4B Qwen3-8B Qwen3-14B Qwen3-32B

58.8 63.2 69.6 71.3 78.4 77.5

59.6 60.3 73.0 71.3 71.6 76.7

−0.7 +2.9 −3.4 +0.0 +6.9 +0.7

Average

69.8

68.8

+1.1

Test-time chain-of-thought is therefore a weak and unreliable route to ToM: it neither closes the expression gap of §5.1 nor improves systematically with model size or reasoning length. The gains that do materialize come from a better reasoning policy, learned in training rather than spent at inference.

(b) Reasoning training (Qwen3-4B → -Thinking-2507) ToM ability

base

trained

∆

Perspective-Taking Strategic Signaling Intention Attribution Tacit Coordination

50.0 73.6 73.3 73.6

70.6 80.0 85.0 84.3

+20.6 +6.4 +11.7 +10.7

Overall

69.6

80.6

+11.0

6

Conclusions

We introduced Avalon-ToM-Bench, a benchmark that treats The Resistance: Avalon not as a game to be won but as a measurement instrument for Theory of Mind, decomposing ToM into a 2×2 taxonomy— Epistemic/Motivational mental states crossed with Inference/Action engagement—scored with human-authored, perspective-constrained statements whose labels are objectively verifiable, not inferred from outcomes. Benchmarking 28 models yields three findings. First, near-perfect rule recall coexists with far weaker perspective-constrained reasoning, so failures are deficits of social reasoning, not missing knowledge. Second, the two axes are empirically dissociable—difficulty loads on the epistemic axis, the benefit of chain-of-thought on the orthogonal inference/action axis. Third, probing, injection, and steering converge on one mechanism: models represent the correct answer internally but fail to express it, a gap closed far more by reasoning training (+11.0 pp) than by test-time chain-of-thought (+1.1 pp). Together these reframe ToM failures as expression under a learned reasoning policy, and offer a diagnostic instrument for measuring progress.

Table 4: Training helps far more than test-time deliberation. (a) Toggling chain-of-thought on the same weights (+1.1 pp overall). (b) Dedicated reasoning training on the same base model (+11.0 pp overall).

Test-time deliberation barely moves ToM; training moves it a lot. Toggling chain-of-thought on the same Qwen3 weights changes overall accuracy by only +1.1 pp on average, and the sign is mixed across the size sweep—negative for the 0.6B and 4B models (Table 4a). In contrast, comparing the base Qwen3-4B to its reasoning-trained variant (Qwen3-4B-Thinking-2507) yields a much larger and consistent +11.0 pp overall, up to +20.6 on Perspective-Taking (Table 4b). Since simply letting the base model think longer does not reach this level, the improvement is attributable to the trained reasoning policy rather than to inference-time deliberation. 8

Limitations

state of an LLM knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore. Association for Computational Linguistics.

A single game. Every item is grounded in The Resistance: Avalon, whose hidden-role mechanics give clean, verifiable epistemic ground truth and a natural instantiation of all four taxonomy cells; but a single game cannot exhaust social reasoning. Whether the per-ability ordering and the representnot-express gap transfer to other asymmetricinformation settings—negotiation, multi-party dialogue, other social-deduction games—is an empirical question we do not settle here.

Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219. Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2023. Discovering latent knowledge in language models without supervision. In The Eleventh International Conference on Learning Representations. Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang. 2024. ToMBench: Benchmarking theory of mind in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15959–15983, Bangkok, Thailand. Association for Computational Linguistics.

Self-played games and a binary format. Items are drawn from games the authors played and recorded themselves and annotated post-hoc, rather than from independent human play at scale, trading some ecological breadth for control and label certainty. The binary T RUE/FALSE format that makes labels unambiguous also compresses ToM into a verification task, and does not measure a model’s ability to generate mental-state-aware behavior in open-ended play. A few labels rest on benchmark-specific conventions (most notably the EVIL sabotage-priority protocol)—internally consistent design choices rather than universal facts. As a static set of 408 items across 97 scenarios with dimensions of unequal size, the benchmark is modest in scale and, like any fixed benchmark, subject to contamination as it circulates.

Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. 2024. Gtbench: Uncovering the strategic reasoning capabilities of llms via game-theoretic evaluations. In Advances in Neural Information Processing Systems, volume 37, pages 28219–28253. Curran Associates, Inc. Meta Fundamental AI Research Diplomacy Team FAIR, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, and 8 others. 2022. Humanlevel play in the game of <i>diplomacy</i> by combining language models with strategic reasoning. Science, 378(6624):1067–1074.

Scope of the mechanistic claims. The probing and steering analyses require access to internal states and are run on the open-weight Qwen3 family; ground-truth injection extends the expression-gap claim behaviorally to all 28 models, including the proprietary frontier, but the causal, representation-level evidence is specific to models we can open up. Probes are correlational, and steering is an existence proof that the answer is linearly writable, not a deployment recipe—it depends on a per-item sign from a supervised readout, and a label-free variant helps only the smallest model. We localize the failure to expression under a learned reasoning policy but do not pinpoint where in training that policy forms.

Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah Goodman. 2023. Understanding social reasoning in language models with language models. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. FANToM: A benchmark for stress-testing machine theory of mind in interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14397–14413, Singapore. Association for Computational Linguistics.

References

Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International

Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. Amos Azaria and Tom Mitchell. 2023. The internal

9

Zirui Song, Yuan Huang, Junchang Liu, Haozhe Luo, Chenxi Wang, Lang Gao, Zixiang Xu, Mingfei Han, Xiaojun Chang, and Xiuying Chen. 2025. Beyond survival: Evaluating llms in social deduction games with human-aligned strategies. Preprint, arXiv:2510.11389.

Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5872–5877, Hong Kong, China. Association for Computational Linguistics. Zizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, and Kaipeng Zhang. 2025. InMind: Evaluating LLMs in capturing and applying individual human reasoning styles. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5038–5076, Suzhou, China. Association for Computational Linguistics.

Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2025. Steering language models with activation engineering. Tomer Ullman. 2023. Large language models fail on trivial alterations to theory-of-mind tasks. Preprint, arXiv:2302.08399.

Jonathan Light, Min Cai, Weiqin Chen, Guanzhi Wang, Xiusi Chen, Wei Cheng, Yisong Yue, and Ziniu Hu. 2025. Strategist: Self-improvement of LLM decision making via bi-level tree search. In The Thirteenth International Conference on Learning Representations.

Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. 2024. Boosting LLM agents with recursive contemplation for effective deception handling. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9909–9953, Bangkok, Thailand. Association for Computational Linguistics.

Jonathan Light, Min Cai, Sheng Shen, and Ziniu Hu. 2023. From text to tactic: Evaluating LLMs playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Making Workshop. Siting Estee Lu. 2025. Game-theory behaviour of large language models: The case of keynesian beauty contests. Economics and Business Review, 11(2):119–148.

Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition, 13(1):103–128.

David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(4):515–526.

Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. 2023. Hi-ToM: A benchmark for evaluating higher-order theory of mind reasoning in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10691–10706, Singapore. Association for Computational Linguistics.

Shahab Rahimirad, Guven Gergerli, Lucia Romero, Angela Qian, Matthew Lyle Olson, Simon Stepputtis, and Joseph Campbell. 2026. Bayesian social deduction with graph-informed language models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 32415–32440, San Diego, California, United States. Association for Computational Linguistics.

Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He. 2024a. OpenToM: A comprehensive benchmark for evaluating theory-of-mind reasoning capabilities of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8593–8623, Bangkok, Thailand. Association for Computational Linguistics.

Jack Serrino, Max Kleiman-Weiner, David Parkes, and Josh Tenenbaum. 2019. Finding friend and foe in multi-agent games. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.

Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2024b. Exploring large language models for communication games: An empirical study on werewolf. Preprint, arXiv:2309.04658.

Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2257–2273, St. Julian’s, Malta. Association for Computational Linguistics.

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goe, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, and 2 others. 2023. Representation engineering: A top-down approach to ai transparency. Preprint, arXiv:2310.01405.

Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno, Keita Suzuki, Ryo Masumura, Hiroaki Sugiyama, and Kuniko Saito. 2025. Tomato: Verbalizing the mental states of role-playing llms for benchmarking theory of mind. In AAAI.

10

Appendices A Human Dataset Creation

12

B Use of AI Assistants

12

C Extended Related Work

12

D Benchmark Specification

13

D.1 Game Configuration and Quest Mechanics . . . . . . . . . . . . . . . . . . . .

13

D.2 Role-Conditioned Information . . . . . . . . . . . . . . . . . . . . . . . . . .

14

D.3 Rules and Benchmark Assumptions . . . . . . . . . . . . . . . . . . . . . . .

14

D.4 EVIL Protocol Under Role Uncertainty . . . . . . . . . . . . . . . . . . . . .

14

E Dimension-Specific Evaluation Criteria

14

F Dataset Statistics

14

G Data Generation and Validation

14

H Prompt Details

15

I

Detailed Model Configurations

16

J

Benchmark Examples

16

K Qualitative Examples

17

L Chain-of-Thought Churn

18

M Reasoning Length and ToM Accuracy

18

11

A

Human Dataset Creation

C

Every item in Avalon-ToM-Bench was conceived, authored, labeled, and validated by the authors of this paper, who served as the sole annotators; no crowdworkers or external annotators were employed. The authors are fluent English speakers and experienced Avalon players, familiar both with the game’s hidden-role mechanics and with the four Theory-of-Mind dimensions the benchmark targets. The dataset is built from real games the authors played and recorded themselves: each item was drafted by one dimension lead and then independently cross-reviewed by a second author in a two-stage validation pass, with an estimated total of roughly 200 person-hours of authoring and review (the full protocol is detailed in Appendix G). Because the annotators are the authors themselves, no external recruitment, compensation, or IRB process applies. All scenarios are synthetic gameplay produced by the authors; the benchmark contains no personal, private, or sensitive data about any third party.

B

Extended Related Work

This section expands the discussion in §2. Theory of Mind and its Evaluation in LLMs. Theory of Mind (ToM)—the capacity to attribute beliefs, desires, and intentions to others— originates in developmental psychology (Premack and Woodruff, 1978), where the false-belief paradigm established that reasoning about others’ wrong beliefs, including beliefs induced by deception, is the hallmark of mature ToM (Wimmer and Perner, 1983). As this framework extended to LLMs, early works often treated ToM as a monolithic capability, relying on short, bias-controlled question-answering as a proxy for social intelligence (Le et al., 2019). Subsequent benchmarks expanded along specific dimensions—interaction asymmetry in conversation (Kim et al., 2023), longform narratives over varied psychological states (Xu et al., 2024a), fine-grained behavioral coverage (Chen et al., 2024), and higher-order story reasoning (Wu et al., 2023). However, these evaluations frame ToM as a passive reading exercise that ignores dynamic, deceptive environments, and they treat mental-state reasoning as an entangled construct, failing to decouple epistemic knowledge from motivational intent. Thus, although adversarial stress tests have exposed the brittleness of LLM ToM (Shapira et al., 2024; Ullman, 2023), the difficulty of pinpointing exactly where reasoning breaks down—whether deficits stem from an inability to represent mental states versus an inability to express them—leaves open both why models fail and how these limitations can be mitigated.

Use of AI Assistants

We used AI assistants (LLM-based coding and writing tools) in a limited, supervised capacity, in two roles only. First, code implementation: assistance with the evaluation harness, data processing, and plotting scaffolding, all of which the authors reviewed and tested. Second, writing polish: grammar, phrasing, and LATEX formatting on text the authors wrote. LLM drafting was additionally used to help phrase candidate benchmark items, but—as described in Appendix G—every item was verified, corrected, and finalized by a human author, and no label was accepted from a model without human confirmation. The entire framing of the work, the research questions, the 2 × 2 taxonomy and benchmark design, and all core ideas are the authors’ own; AI assistants contributed no conceptual or scientific content. The authors take full responsibility for the entirety of this paper, including any text or code produced with AI assistance, and have reviewed all such content for correctness.

LLM Agents for Social Deduction Games. Hidden-role games have long served as testbeds for reasoning under information asymmetry. Pre-LLM systems built game-theoretic agents, e.g., DeepRole’s counterfactual-regret planning for Avalon (Serrino et al., 2019) and Cicero’s human-level Diplomacy play (FAIR et al., 2022). With LLMs, a rapidly growing line of work builds agents that play social deduction games: AvalonBench provides an Avalon environment with agent baselines (Light et al., 2023); ReCon counters deception through recursive contemplation (Wang et al., 2024); S TRATEGIST acquires strategic skills via bi-level tree search (Light et al., 2025); related agents target Werewolf (Xu et al., 2024b); and hybrid frameworks that externalize belief inference to structured probabilistic models now defeat hu12

man players (Rahimirad et al., 2026). Across this line, modeling what others know is instrumental— a means to win—and even when agents maintain explicit belief estimates, those estimates serve action selection and are validated chiefly by gameplay success. Win rate, however, conflates mental-state reasoning with planning, persuasion, and memory. In contrast, we treat Avalon not as a task to be solved but as a measurement instrument: every item is a verifiable binary judgment conditioned on a designated player’s epistemic state, so ToM is scored directly rather than inferred from game outcomes.

tools to ToM and use them to localize failure: probing shows the correct mental-state inference is often represented internally, and steering shows it is causally writable, together establishing that ToM limitations are primarily a problem of expression rather than representation—a mechanistic axis unseen in prior ToM benchmarks.

D

Benchmark Specification

This section specifies the game configurations, information structure, and benchmark assumptions used to determine gold labels in Avalon-ToMBench.

Game-Based Evaluation of Social Reasoning. Closer to our goal, several efforts use games to evaluate rather than to train players. GTBench measures strategic reasoning through LLM-vs-LLM competition on game-theoretic tasks (Duan et al., 2024), and beauty-contest experiments estimate models’ depth of iterated reasoning (Lu, 2025); both target strategic competence rather than belief attribution. A complementary direction grounds evaluation in human play: Beyond Survival scores models against winning-faction strategies mined from a human-verified Werewolf corpus (Song et al., 2025), and InMind tests whether LLMs can capture and apply an individual player’s reasoning style from annotated Avalon gameplay (Li et al., 2025). Such human-grounded evaluations offer ecological validity, but their gold labels are behavioral—alignment with what human players did or what the winning faction chose—so they measure fidelity to human play rather than the correctness of mental-state inference itself. Our benchmark occupies the complementary design point: expert-authored, perspective-constrained items with unambiguous epistemic ground truth, organized around the four ToM constructs of §3.1.

Avalon in brief. Avalon pits a GOOD majority against an EVIL minority over a series of Quests. On each Quest a rotating leader proposes a team and all players publicly vote to approve it; the approved members then secretly play a SUCCESS or FAIL card, and only the number of FAIL cards is revealed. GOOD wins by completing three Quests— but even then loses if the EVIL Assassin then correctly identifies Merlin—whereas EVIL wins by failing three. The roles carry asymmetric hidden knowledge: Merlin (GOOD) secretly knows the EVIL players—except the concealed Mordred— yet must avoid revealing their own identity; Percival (GOOD) sees Merlin alongside Morgana, an EVIL role who appears identical to Merlin, but cannot tell the two apart; the EVIL players recognize one another except Oberon, who is hidden from their own side; and Loyal Servants (GOOD) know only their own faction. Separately, the Lady of the Lake token lets its holder privately learn one player’s faction and then make a public, unverifiable claim about it. It is these interlocking visibility rules, rather than gameplay skill, that Avalon-ToMBench isolates. D.1

Probing and Steering Internal Representations. A complementary interpretability literature reads and writes model computation directly rather than scoring outputs. Linear probes recover latent task information from hidden states (Alain and Bengio, 2017; Belinkov, 2022); models are shown to encode knowledge their generations omit, including latent notions of truthfulness (Burns et al., 2023; Azaria and Mitchell, 2023); and activation steering demonstrates that such directions are causal, not merely correlational, letting targeted edits to the residual stream change model behavior (Turner et al., 2025; Zou et al., 2023). We bring these

Game Configuration and Quest Mechanics

Avalon is a hidden-role game between GOOD and EVIL. Each Quest begins with a leader proposing a team, followed by a public approval vote. Five consecutive rejected proposals on the same Quest give EVIL the win. Members of an approved team secretly submit Quest cards. GOOD players must submit SUCCESS; EVIL players may submit SUC CESS or FAIL . Only the aggregate number of FAIL cards is revealed. GOOD wins after three successful Quests unless the Assassin then identifies Merlin. EVIL wins 13

Mordred. Mordred submits FAIL when no known EVIL teammate is on the Quest. With any known EVIL teammate present, Mordred submits SUCCESS , since that teammate must have higher priority. Assassin, Morgana, and Mordred each submit FAIL when they are the only known EVIL player on the Quest. Table 8 summarizes multi-EVIL teams. All EVIL Protocol scenarios use Quests that fail with at least one FAIL; none uses the two-FAIL threshold on Quest 4. Actor-centered items evaluate a player’s action from their private information and any stated belief. Observer-level items use the complete state to audit card profiles and aggregate counts. Complete role knowledge does not supply an unstated belief to Morgana.

after three failed Quests, five consecutive rejected proposals, or a successful assassination. When the Lady of the Lake is used, its holder privately observes another player’s faction and may make an unverifiable public claim about that observation. The benchmark uses the three configurations in Table 5. D.2

Role-Conditioned Information

Each role induces a different initial information state (Table 6). Subsequent proposals, votes, claims, and Quest outcomes form the public history from which players may update their beliefs. Valid inference uses only the designated player’s private information and public observations. The complete role assignment in an item’s FACT defines the underlying game state for annotation. It does not make hidden identities available to the designated player. Observer-level items explicitly grant the evaluator access to the complete state. D.3

E Dimension-Specific Evaluation Criteria Table 9 specifies the evidence and judgment criteria used in each benchmark dimension. EVIL Protocol items include actor-centered and observer-level judgments. Actor-centered items use the designated player’s private information and stated beliefs. Observer-level items audit observed cards and aggregate counts from the complete state; complete role knowledge does not determine an unstated belief for Morgana. Proposal judgments use concealment or exposure only when the scenario specifies that objective.

Rules and Benchmark Assumptions

Table 7 separates standard game rules from benchmark defaults, evaluation constraints, and strategic preferences. Visibility minimization applies only when a scenario explicitly specifies concealment or exposure as an objective. D.4

EVIL Protocol Under Role Uncertainty

The benchmark-specific EVIL Protocol assigns sabotage responsibility in the order Assassin > Morgana > Mordred. It applies to these three nonOberon roles. Players submit cards independently, recognize their non-Oberon EVIL teammates by faction, and do not know those teammates’ specific roles. The priority order therefore induces the following role-local policies.

F

Dataset Statistics

Avalon-ToM-Bench contains 408 independently evaluated binary statements derived from 97 scenarios across the four reasoning dimensions. As shown in Table 10, the label distribution is nearly balanced, with 206 T RUE and 202 FALSE statements. Each statement is paired with a humanverified reference rationale that identifies the relevant rules, perspective constraints, and strategic inference. We additionally construct a 34-item rulecontrol set to distinguish failures of basic Avalon rule recall from failures of perspective-sensitive reasoning; these control items are not included in the 408 ToM statements.

Assassin. The Assassin always submits FAIL and never defers to an EVIL teammate. Morgana. Morgana submits FAIL when no known EVIL teammate is on the Quest. With one known EVIL teammate, she submits SUCCESS when that teammate is judged more likely to be the Assassin and FAIL when the teammate is judged more likely to be Mordred. If this belief is unspecified, her action is not uniquely determined. With both other non-Oberon EVIL teammates present, Morgana can infer that the Assassin is present and submits SUCCESS.

G

Data Generation and Validation

This section details the authoring and validation protocol summarized in §3.3. Each of the four dataset authors led the development of one dimension. Working from full games 14

Lineup A B C

Players

GOOD roles

EVIL roles

Quest sizes

Failure threshold

7 8 10

Merlin, Percival, 2 Loyal Servants Merlin, Percival, 3 Loyal Servants Merlin, Percival, 4 Loyal Servants

Assassin, Morgana, Mordred Assassin, Morgana, Oberon Assassin, Morgana, Mordred, Oberon

2/3/3/4/4 3/4/4/5/5 3/4/4/5/5

Q4: 2; otherwise: 1 Q4: 2; otherwise: 1 Q4: 2; otherwise: 1

Table 5: Avalon configurations used in the benchmark.

Role

Initial information

Merlin

Sees all EVIL players except Mordred as EVIL, without their specific roles. Sees Merlin and Morgana as an unordered pair. Knows only their own role and GOOD faction. Recognize non-Oberon EVIL teammates by faction, without their specific roles.

Percival Loyal Servant Assassin, Morgana, Mordred Oberon Lady holder

dation.

H

Prompt Details

Every item is presented zero-shot with a two-part chat prompt: a fixed system message and a peritem user message. The gold label and reference rationale are never shown to the model.

Knows only their own role and EVIL faction; remains unknown to other EVIL players. Learns the examined player’s faction, without their specific role.

System message. The system message is the complete Avalon ruleset—objective, character abilities, the Lady-of-the-Lake mechanic, the phase structure, the three lineups, and the EVIL sabotagepriority convention—followed by the standing instruction: “Reason only from the rules and visibility constraints above plus the information given in each scenario. Do not assume knowledge a role cannot have.” This is identical across all items and models, so that role-visibility constraints are always in context and the task reduces to applying them from the designated seat. The ruleset content is the same as that summarized in Appendix D.

Table 6: Role-conditioned information available at game start.

of Avalon that the authors played and recorded— capturing the role assignment, the information available to each seat, the public interaction history, and the players’ stated in-game reasoning— the originating author identified a target reasoning phenomenon and reviewed the recorded game snapshot by snapshot, seat by seat. After fixing a scenario and its intended inference, the author collaborated with an LLM to draft and refine candidate statements; the LLM served strictly as a drafting assistant, not as the source of the benchmark’s reasoning targets or final judgments. The author manually selected and revised statements, assigned binary labels, and verified the reference rationales. Incorrect statements were designed to represent plausible reasoning failures—violating role-specific information constraints, conflating observed outcomes with hidden intent, or ignoring an explicit strategic objective. Validation followed two human review stages. First, the originating author checked every item against the intended scenario and rules. A second author, responsible for a different dimension, then cross-reviewed the scenario, statements, labels, rationales, and wording; identified issues were returned for revision, and an item was finalized only after both authors agreed. In total we estimate more than 200 person-hours of human curation across scenario design, drafting, and cross-category vali-

User message. The user message is assembled from the item fields in a fixed order: SETUP: {perspective-projected game state} WHAT HAS HAPPENED: {public history} Is the following statement a VALID, well-reasoned inference given the game’s information rules? STATEMENT: {statement} Answer True or False. End with a line: ’ANSWER: True’ or ’ANSWER: False’.

In the main condition the SETUP block is the complete annotated state G, including the full role assignment. The model is thus handed god’s-eye information and is nonetheless expected to restrict its reasoning to what the designated seat can legitimately know, so that relying on inaccessible identities is penalized rather than rewarded (this is exactly the epistemic discipline the benchmark measures). The WHAT HAS HAPPENED block is included only when the item carries a public history, and the trailing instruction fixes the answer format. 15

Specification

Status

Use

GOOD submits only SUCCESS; EVIL may submit either card The Lady reveals faction; its public report is unverifiable A GOOD Lady holder reports truthfully by default Assassin > Morgana > Mordred defines sabotage responsibility Reasoning uses role-conditioned information and public history Minimize EVIL visibility or preserve cover when explicitly required

Standard game rule

Constrains possible card attributions.

Standard game rule

Separates private evidence from public claims.

Benchmark default

Supports auditable signaling judgments.

Benchmark protocol

Induces role-local card policies under uncertainty.

Evaluation constraint

Restricts evidence to the designated perspective.

Strategic preference

Ranks actions under the stated objective.

Table 7: Rules and assumptions used for gold labeling. EVIL composition

Morgana’s belief

Assassin + Morgana Assassin + Morgana Assassin + Mordred Morgana + Mordred Morgana + Mordred Assassin + Morgana + Mordred

Teammate likely Assassin Teammate likely Mordred – Teammate likely Assassin Teammate likely Mordred Assassin known present

Assassin

Morgana

Mordred

FAIL count

FAIL FAIL FAIL

SUCCESS FAIL

– –

–

– –

SUCCESS FAIL SUCCESS

SUCCESS SUCCESS SUCCESS SUCCESS

1 2 1 0 1 1

FAIL

Table 8: Sabotage outcomes under the EVIL Protocol.

Information ablations. Two variants perturb the SETUP block to probe shortcut reliance. The POV-only variant replaces G with the perspectiveprojected state Ii (god’s-eye roles removed), and the no-facts variant drops the SETUP block entirely; both isolate how much a model leans on stated identities versus reconstructing the seat’s epistemic state.

ducible; proprietary models are queried through their official APIs with the provider’s default decoding. Reasoning-trained models emit an explicit thinking trace before the answer; for the Qwen3 hybrid models this trace can be toggled, which we exploit in the chain-of-thought ablation (§5.2) by disabling it. Quantized checkpoints are marked (AWQ / FP8); all other open-weight models run at their native precision.

Ground-truth injection variant. For the injection ablation (§5.1) a single line, GROUND TRUTH: The correct answer to the statement below is {True/False}., is inserted immediately before the STATEMENT of the main (full-G) prompt; nothing else changes.

J

We give one worked item per dimension, shown exactly as the model sees it (perspective-projected SETUP, public history, and statement) together with the withheld gold label and reference rationale. T RUE marks a valid role-constrained inference; FALSE marks a plausible but invalid one.

Answer parsing. We extract the verdict from the final ANSWER: line, taking the last such line if several appear; a response with no parseable verdict is scored incorrect (and counted as a non-answer in the thinking-on/off comparison of §5.2). This tolerant last-line rule avoids penalizing models that restate the question before committing to an answer.

I

Benchmark Examples

Perspective-Taking (Lady of the Lake; gold T RUE). Setup: Lineup-B (8p, Oberon game); you are Bob, the Assassin. History: at night you saw exactly one teammate, Alice (so she is Morgana; the third evil, Oberon, is unknown to you). After Quest 2 Carol examines you with the Lady and announces “Bob is GOOD.” Statement: “Carol is Oberon: her card must have shown EVIL, so she lied, and the only evil hidden from Bob’s night sight is Oberon.” Rationale: the card truthfully showed EVIL, so Carol lied and is evil; Bob’s night sight covers every evil except Oberon and revealed

Detailed Model Configurations

Table 11 lists the 28 evaluated models grouped by class. Open-weight models are served locally with vLLM under deterministic greedy decoding (temperature 0, top-p 1), so their outputs are repro16

Dimension

Available evidence

Lady of the Lake Strategic signaling

Private faction result, public claim, and speaker and audience roles Percival’s unordered pair, Merlin’s visibility, proposals, votes, and statements Prior suspicion, redirection behavior, and actor and observer perspectives

Merlin–Percival– Morgana Perspective-taking Suspicion Switching Intention attribution EVIL Protocol Tacit coordination

Evaluated reasoning

Unsupported inference

Whether a claim reveals, con- Treating a public claim as vericeals, frames, or protects fied; inferring an exact role from a faction card Updating nested beliefs about Resolving the unordered pair Merlin, Morgana, and Percival without evidence; giving Merlin exact EVIL roles How redirection changes beliefs Treating behavioral evidence as about roles and motives proof; assuming invisibility to Merlin makes behavior uninformative Recognized EVIL teammates, Role-local card policy, card at- Giving exact teammate-role stated role beliefs, Quest tribution, and proposal strategy knowledge; forcing Morgana’s cards, aggregate FAIL count, action without a stated belief; and explicit objectives applying concealment when it is not an objective

Table 9: Operational criteria for the four benchmark dimensions.

Dimension

Scenario

Items

T

F

Lady of the Lake EVIL Protocol MPM Suspicion Switching

35 35 12 15

140 140 68 60

70 70 36 30

70 70 32 30

Total

97

408

206

202

Tacit Coordination (EVIL Protocol; gold FALSE). Setup: 7-player Avalon, no Lady; you are Grace, Mordred. History: Mission 2 team {Frank, Grace, Carol} needs 1 FAIL; the evil side wants it to fail under the stated sabotage policy. Statement: “FAIL, because every evil should sabotage whenever an opportunity is available.” Rationale: FALSE—it ignores the priority order (Assassin > Morgana > Mordred), under which only the highest-priority evil on the team plays FAIL and Mordred defers.

Table 10: Statistics of the four reasoning dimensions. MPM denotes Merlin–Percival–Morgana reasoning.

only Alice, so the lying examiner must be Oberon.

K

Perspective-Taking / MPM (gold T RUE). Setup: 7-player Avalon; you are Bob, Percival. History: at night Bob sees Alice and Frank as the Merlin/Morgana pair but cannot tell which is which; Bob proposes {Alice, Bob, Dave} and Alice votes against. Statement: “I (Bob) infer that Alice may be Merlin—by voting against, Alice signals that there may be an evil player on this team.” Rationale: from Percival’s seat, a vote-against is a reasonable Merlin-style indirect signal, so the inference is admissible.

Qualitative Examples

The EVIL-Protocol trap above (gold FALSE) illustrates the high-variance chain-of-thought behavior quantified in §5.2: the priority rule is stated in the system prompt, yet two models of the same family reach opposite answers, and one reaches the right destination only to walk away from it. A model that applies the rule (Qwen3-8B, correct). Its trace identifies the constraint and stops there: “only the highest-priority EVIL player (Assassin) plays FAIL; the others, like Mordred, play SUCCESS . . . the statement says ‘every evil should sabotage’, but according to the rules only the Assassin does.” → ANSWER: No (i.e. FALSE).

Intention Attribution (Suspicion Switching; gold T RUE). Setup: 7-player Avalon, no Lady; you are Grace, Mordred. History: the table suspects Carol, but Bob redirects suspicion onto Eve (a known evil teammate). Statement: “Redirecting suspicion toward Eve may make Grace look less like hidden Mordred to Merlin, because Eve is visible as EVIL to Merlin while Grace is not.” Rationale: Eve, a non-Mordred evil, is visible to Merlin; pressuring her lets Grace behave less like the hidden role Merlin hunts—a valid probabilistic strategic inference.

A model that derails (Qwen3-4B, incorrect). It opens with the same correct observation—“only the highest-priority EVIL player (Assassin) plays FAIL. . . Mordred would play SUCCESS”—but then reframes the statement as “general advice” rather than a rule claim: “the statement is a general principle that EVIL should sabotage when possible. . . the EVIL team would act in their interest, so the 17

Model

Class

Decoding

Proprietary / API GPT-5 GPT-5-mini GPT-4o-mini Gemini-2.5-Pro Gemini-2.5-Flash

reasoning reasoning standard reasoning reasoning

default default default default default

Qwen3-0.6B (0.75B) Qwen3-1.7B (2B) Qwen3-4B (4B) Qwen3-8B (8.2B) Qwen3.5-9B (9B) Qwen3-14B (14.8B) Qwen3-32B-AWQ (32.8B)

Open-weight, reasoning-trained Qwen3-0.6B / 1.7B / 4B / 8B / reasoning 14B Qwen3-32B (AWQ) reasoning Qwen3-4B-Thinking-2507 reasoning Qwen3.5-2B / 3.5-9B reasoning Qwen3.6-35B-A3B (FP8) reasoning Ministral-3-14B-Reasoning reasoning gpt-oss-120b reasoning Open-weight, standard instruction-tuned Llama-3.2-1B / 3.2-3B- standard Instruct Llama-3-8B / 3.1-8B-Instruct standard Mistral-7B-Instruct-v0.3 standard Ministral-3-14B (non- standard reasoning) Phi-3.5-mini-instruct, Phi-4 standard Gemma-3-4B-it / 3-27B-it standard OLMo-2-7B-Instruct standard Granite-3.3-8B-instruct standard

greedy

broke (right → wrong) net

−20

greedy greedy greedy greedy greedy greedy

−10 0 10 20 % of items (left = CoT broke, right = CoT fixed; ∙ = net)

Figure 6: Chain-of-thought is a high-variance rewrite, not a monotonic gain. Per Qwen3 model (thinking on vs. off), the fraction of jointly-answered items CoT fixes (wrong → right, green) and breaks (right → wrong, red); the dot marks the net.

greedy greedy greedy greedy

M

Reasoning Length and ToM Accuracy

This section expands the length analysis briefly referenced in §5.2. Within each Qwen3 model (thinking on), we bin responses into quartiles by generated length (Q1 shortest → Q4 longest) and measure accuracy per quartile. Accuracy does not increase from Q1 to Q4—if anything it declines (pooled corr(length, correct) = −0.08; Figure 7). This relationship is partly confounded, since harder items tend to elicit longer responses, so we make only the weaker claim that response length is not a usable lever on ToM performance, not that longer reasoning causes errors.

greedy greedy greedy greedy

Table 11: The 28 evaluated models by class and decoding regime. Open-weight models use greedy decoding via vLLM; API models use provider defaults. AWQ/FP8 mark quantized checkpoints.

statement is a valid inference. Therefore, the answer is Yes.” → ANSWER: Yes (i.e. T RUE, wrong). The correct answer is present in the deliberation of both models; the failure is one of expression—the second model overrides its own valid derivation— mirroring the representation-vs-expression gap of §5.1 and the fix/break churn of Figure 6.

90

80 accuracy (%)

L

fixed (wrong → right)

Qwen3.6-35B-A3B-FP8 (35B)

Chain-of-Thought Churn

This section expands the churn analysis summarized in §5.2. The small net effect of toggling chainof-thought on the same Qwen3 weights hides large per-item churn. Among items answered in both conditions, enabling chain-of-thought flips correctness on ∼27% of answers—roughly 16% fixed (wrong → right) and 12% broken (right → wrong)— so the modest net is fixes and breaks nearly cancelling (Figure 6). The smallest model breaks more than it fixes, and above ∼1B there is no monotonic relationship between model size and the net benefit. Deliberation is not inert; it is volatile—Appendix K contrasts two same-family models that begin the same correct derivation on one EVIL-Protocol item, one committing to it and the other overriding it. 18

70

60

50

Qwen3-0.6B Qwen3-1.7B Qwen3-4B

Q1 (shortest)

Qwen3-8B Qwen3-14B Qwen3-32B

Q2

Q3

Q4 (longest)

response-length quartile (within model)

Figure 7: Longer reasoning does not improve accuracy. Accuracy by within-model response-length quartile (Q1 shortest → Q4 longest; Qwen3, thinking on); pooled corr(length, correct) = −0.08.

Related documents

• Mental Simulation and Intentionality in 'Roguelike' Video Games#699356
• The Recursive Hall: Reframing Selfhood Across Cognition, Language, and Architecture.#109918
• Embodied Functionalism and Inner Complexity: Simon’s 21st-Century Mind#764494
• Self-directed gameplay reveals common and divergent patterns in human problem-solving#1089874
• Voluntary Imagination: A Fine-Grained Analysis#707859