Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games Keyang Zhong Junlin Xie Sun Yat-sen University The Chinese University of Hong Kong, Shenzhen [email protected] [email protected] Hefeng Wu Sun Yat-sen University [email protected]
Haofeng Li Sun Yat-sen University [email protected]
Abstract
arXiv:2604.11741v1 [cs.AI] 13 Apr 2026
Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multi-player game settings with imperfect and deceptive information. In this paper, we pick up a representative multi-player task, Murder Mystery Games, which require to infer hidden truths based on partial clues provided by the roles of different intentions. To address this challenge, we propose a collaborative multiagent framework for evaluating and synthesizing high-quality, role-driven multi-player game scripts, enabling fine-grained interaction patterns tailored to character identities (i.e., murderer vs. innocent). Our system generates rich multimodal contexts—including character backstories, visual/textual clues, and multi-hop reasoning chains—through coordinated agent interactions. We design a two-stage agentmonitored training strategy to enhance the reasoning ability of VLM: (1) Chain-of-Thought based fine-tuning on curated and synthetic datasets that model uncertainty and deception; (2) GRPO-based Reinforcement Learning with agent-monitored reward shaping, encouraging the model to develop character-specific reasoning behaviors and effective multi-modal multihop inference. Extensive experiments demonstrate that our method significantly boosts the performance of VLM in narrative reasoning, hidden fact extraction, and deception-resilient understanding. Our contributions offer a scalable solution for training and evaluating VLMs under uncertain, adversarial, and socially complex conditions, laying the groundwork for future benchmarks in multimodal multi-hop reasoning under imperfect information.
1
Introduction
Vision-language models (VLMs) have demonstrated impressive capabilities in foundational perceptual tasks such as image captioning and visual question answering (VQA), as well as in more
Guanbin Li Sun Yat-sen University [email protected]
complex reasoning tasks through chain-of-thought (CoT) prompting, leveraging their ability to align and integrate information across visual and linguistic modalities (OpenAI, 2023; Google, 2023; Li et al., 2025b; Wang et al., 2025). However, tasks that demand sophisticated reasoning particularly those involving multi-hop inference, imperfect or deceptive information, and dynamic social interactions—remain challenging (Yang et al., 2018; Chen et al., 2024a,b). Advancing VLMs in such settings necessitates evaluation and training environments that require not only perception and knowledge, but also deeper reasoning and adaptability under imperfect information. In real life, many practical tasks involve a multiplayer game-theoretic process using imperfect information. For example, in judicial proceedings, judges, prosecutors, defense lawyers, witnesses, and juries engage in multiple rounds of social interaction from their respective perspectives, and make multi-hop inferences and decisions based on incomplete information, ultimately attempting to finish their own task. To study such imperfect-information multiplayer reasoning in vision-language models, we adopt Murder Mystery as a representative test environment. Murder Mystery is a social deduction role-playing game in which players assume predefined identities and collaboratively infer the hidden murderer, making it a typical yet tractable setting for modeling multi-agent interaction and reasoning under uncertainty. With access to public and private textual and visual clues, players engage in structured dialogue to reason about motives and inconsistencies, and infer the murderer amid adversarial deception. The game proceeds through four key phases: 1. Role Setup and Clue Absorption: Players receive the rules and character backgrounds, followed by textual and visual clues. They
CriticAgent
Logic
Evaluation agents
ScoreAgent
Init-VLM
Coherence Feedback
SFT
Generation agents OutlineAgent
CharacterAgent
ClueAgent
RoleplayAgent
Role-playing data
User settings
Monitor
RL
Text Clue Pool
outline
Role-scripts
Image Clue Pool
Game scripts
QaAgent
Evaluation QA data
EnhancedVLM
Figure 1: Overview of the proposed framework. It employs evaluation agents and generation agents to collaboratively generate logically coherent game scripts and instructs a pretrained VLM via a two-stage training strategy under agent monitoring to enhance the target model’s reasoning capability under imperfect information.
then provide in-character self-introductions as the begin. 2. Interactive Discussion: Players engage in question-and-answer interaction with each other based on their clues and suspicions, emphasizing social inference, inconsistency detection, credibility assessment, and information selection. 3. Hypothesis Generation: Integrating accumulated clues and dialogue, players generate reasoning chains to infer motives and methods. This phase requires multi-hop multimodal reasoning across narrative and visual content. 4. Final Decision: Each player makes a final judgment regarding the murderer’s identity. This setting embodies key challenges such as imperfect information, inconsistency detection, and strategic social interaction, making it a suitable testbed for evaluating multi-hop multimodal reasoning in vision-language models. In addition, the task probes models’ abilities in long-form narrative understanding, multimodal evidence integration, and the synthesis of textual and visual information through multi-step inference. Although Murder Mystery is a representative task for modeling multi-player game process, there is still a lack of large-scale datasets for fine-tuning and evaluating models. Large-scale production of high-quality murder mystery scripts is expensive and impractical. To address this challenge, we design a multi-agent simulation framework
where powerful LLMs (e.g., Gemini 2.5Pro (Gemini Team, 2025)) act as autonomous agents to collaboratively synthesize diverse Murder Mystery game scripts, producing challenging questionsanswering pairs and multi-player interactive dialogue as training datasets. To enable the fine-tuning of VLMs on complex and adversarial examples, we build a new paradigm to automatically generate reasoning chains based on incomplete information. Lastly, we adopt a scalable two-stage training pipeline to learn VLMs, via combining highquality, auto-generated cases with curated training data. Our main contributions are summarized as follows: • Multi-Agent Script Synthesis Framework: We propose a scalable multi-agent framework to automatically generate diverse, high-quality multi-player game scripts. This framework simulates realistic character roles, player interactions and multimodal clues. • Training Data Construction and Learning under Imperfect Information: We develop a novel paradigm for generating reasoning chains under imperfect information, enhancing model learning via a two-stage agentmonitored strategy. • Performance Enhancement under Imperfect Information: Our method demonstrates consistent performance gains in reasoning and role-playing for vision-language models at both the 3B and 7B scales (e.g., Qwen2.5VL-3B-Instruction and its 7B counterpart) in
Murder Mystery scenarios involving imperfect and deceptive information.
2
Related Work
Social Reasoning Games as VLM Evaluation Platforms Social reasoning games, such as Murder Mystery and Werewolf, have become prominent platforms for evaluating the reasoning capabilities of VLMs in settings characterized by imperfect information, multi-agent interactions, and deception. These games provide a robust framework for assessing cognitive resilience in complex, multimodal scenarios (Zhu et al., 2025; Wu et al., 2024). WhodunitBench, offers 50 murder myster scripts with both multiple-choice and open-ended questions to facilitate multi-agent reasoning assessment (Xie et al., 2024). Frameworks such as MultiMind extend the evaluation to non-verbal modalities, incorporating facial expressions and intonation (Zhang et al., 2025d). The SocialMaze benchmark focuses on VLM reasoning in static social contexts, explicitly excluding deceptive elements (Xu et al., 2025b). Other frameworks, including BALROG, KORGym, and VS-Bench, assess multimodal reasoning in dynamic game environments but do not explicitly target social interaction capabilities (Paglieri et al., 2024; Shi et al., 2025; Xu et al., 2025a). Multi-Agent Synthetic Data The scarcity of high-quality multimodal training data remains a significant bottleneck for VLM development. Synthetic data generation, particularly through multiagent systems, has emerged as a scalable solution that enhances dataset diversity and reasoning complexity while reducing reliance on manual annotation. AgentInstruct utilizes a hierarchical multiagent workflow to automatically produce synthetic instruction-response data with minimal human involvement (Mitra et al., 2024). Similarly, MATRIX simulates multi-agent social scenarios to generate data for alignment and instruction tuning (Tang et al., 2025). AudioGenie uses a dual-team multiagent framework consisting of a "generation team" and a "supervision team" to generate diverse audio from multimodal inputs(Rong et al., 2025). Frameworks such as GenArtist and LayerCraft operate on similar principles. GenArtist decomposes complex text prompts into sub-tasks using a VLM-based agent, constructs detailed planning trees, and leverages external tools (e.g., SDXL, DALL-E 3) for image generation and editing. Iterative verification
and self-correction further enhance output fidelity (Wang et al., 2024; Zhang et al., 2025b,c). Recent works on composed image retrieval further highlight the importance of modeling fine-grained modification signals and compositional semantics for generating high-quality multimodal data (Li et al., 2026; Chen et al., 2025, 2026; Zhang et al., 2026a; Qiu et al., 2026). Training Pipelines for Reasoning-Enhanced VLMs Recent research frequently adopts a supervised fine-tuning (SFT) followed by reinforcement learning (RL) pipeline to enhance VLM reasoning. Both Reason-RFT (Tan et al., 2025) and SRPO (Zhang et al., 2025a) employ this twostage approach: SFT is used to instill structured chain-of-thought reasoning, while RL further optimizes reasoning quality and generalization. In a curriculum-based paradigm, Infi-MMR (Liu et al., 2025) progressively transitions from textual to multimodal and caption-free reasoning using sequential RL, achieving strong results on multimodal math benchmarks. VILASR introduces a “drawing-toreason” paradigm, utilizing simple visual operations (e.g., auxiliary lines) to articulate spatial relationships, and employs a three-stage training process—synthetic data pre-training, reflective rejection sampling, and RL—to improve self-correction and generalization (Wu et al., 2025). Recent efforts also explore retrieval-augmented and experiencedriven learning paradigms to improve long-horizon reasoning and interaction efficiency. For instance, ExpSeek proposes a self-triggered experience seeking mechanism for web agents, enabling adaptive data acquisition and policy refinement during training (Zhang et al., 2026b).
3
Method
This section first describes our multi-agent framework, and then takes the Murder Mystery Games as the application scenario to depict the process of applying our framework. 3.1
Overview of Proposed Multi-Agent Framework
The proposed collaborative multi-agent framework aims to leverage collaborative agents to generate high-quality training data and instruct a pretrained VLM to enhance its reasoning under imperfect information in game-theoretic tasks.Our multi-agent framework includes two types of agents, i.e., generation agents and evaluation agents. While gen-
eration agents simulate realistic, interactive game processes to generate game-script data, evaluation agents focus on assessing the quality of these generated outputs and offering constructive feedback for improvement. As shown in Figure 1, the framework includes generation agents such as the OutlineAgent, which produces story outlines with background and role summaries; the CharacterAgent, which creates detailed role scripts; and the ClueAgent, which generates multimodal clues that convey key environmental information. Building upon these elements, the RoleplayAgent produces role-playing data for specific scenarios, while the QaAgent constructs question–answer pairs to assess and strengthen the model’s reasoning ability. To ensure quality, the CriticAgent evaluates the generated scripts for logical coherence and behavioral consistency. During training, the ScoreAgent assesses the model’s role-specific behaviors, measuring how well its interactions align with the designated roles, and uses this feedback to facilitate model improvement. All agents interact through shared game scripts, working collaboratively to enhance imperfect-information reasoning, and the framework remains extensible for diverse gametheoretic tasks by adapting or adding specialized agents.
3.2
Agent-Driven High-Quality Data Generation
Each agent is instantiated via carefully designed prompts to a strong proprietary model(Figure 2). OutlineAgent first constructs the crime-day narrative with basic motives and secrets. CharacterAgent elaborates detailed daily actions and interactions while maintaining suspense. CriticAgent evaluates the resulting scripts across four dimensions: plot complexity, character development, difficulty, and logical rationality, and gives feedback for refinement. ClueAgent produces multimodal clues—visual or textual—that aid deduction without revealing the culprit. RoleplayAgent then simulates multi-turn dialogues, while QaAgent generates reasoning chains and QA pairs (from one-hop to multi-hop) with annotated step-by-step reasoning and supporting evidence. The resulting training corpus consists of two components: interactive role-playing data, providing context-rich dialogue trajectories, and structured QA data covering perception and cognition, with explicit reasoning and evidence grounding. Together, these data support subsequent agentmonitored model enhancement, enabling robust and multifaceted capability injection into the target VLM. Detailed agent specifications and dataset descriptions are provided in Appendices A and B.
Application to Murder Mystery Games Although game-theoretic tasks are widely present in social life, there are rare well-defined benchmarks for enhancing VLMs’ reasoning ability in such scenarios. Recently, a benchmark (Xie et al., 2024) rooted from Murder Mystery Games has emerged as a VLM evaluation platform. Though its data remains insufficient, it offers a well-defined game-theoretic scenario. Therefore, we validate the soundness of the proposed framework by training models on data synthesized by our framework and evaluating their effectiveness in Whodunitbench.
To effectively enhance the target model’s reasoning capability under imperfect information, we adopt a two-stage training strategy: (i) direct fine-tuning with synthetic offline data to establish basic roleplaying and reasoning capabilities in Murder Mystery Games, and (ii) GRPO-based reinforcement learning monitored by ScoreAgent to incentivize reasoning potentials, as illustrated in Figure 3.
Under the Murder Mystery Games evaluation protocol, a VLM receives a context defined as C = {B, I, T, D}, where B denotes character backgrounds, I = {In }N n=1 is a set of image-based M clues, T = {Tm }m=1 comprises public textual clues, and D captures the dialogue history. The model is required to demonstrate role-playing fidelity, detect deception by other participants, and execute sophisticated reasoning over multimodal clues.
Since the training data are synthesized by agents built on powerful large-scale VLMs, fine-tuning allows a smaller target model to inherit structured reasoning patterns and role-playing interaction behaviors, leading to improved performance on complex multimodal inference tasks. In practice, we apply parameter-efficient fine-tuning with LoRA(Hu et al., 2021) to the pretrained VLM using standard autoregressive supervision over the generated answers and reasoning traces.
3.3
3.3.1
Agent-Monitored Model Enhancement
Supervised Fine-tuning
Feedback Improvement
Role-scripts and Clues Generation
Multi-modal Clues Pool
Role-scripts
Text Clue Pool
OutlineAgent
CharacterAgent Details
Theme
User settings
Timeline
Behavior
Characters
Motivation
CriticAgent
ClueAgent
Logic
LogicCoherence
A subtle, bitter almond scent, almost masked by the strong truffle aroma…
Image Clue Pool
Feedback
clues generation
Role-playing and Evaluation QA Data Generation Role-playing Data Text Clue Pool Image Clue Pool
RoleplayAgent
self-introductions
discussions
QaAgent
Reasoning chain
Where did you go after 10:00 last night? Are there any witnesses to your whereabouts?
Generated QA
One-hop Clues
I went to the bar, the bar owner can testify
QA
Two-hop Clues
can you provide more details about the symptoms you experienced?
Three-hop Clues
I…
Human&LVLM
Evaluation QA data Chain-Of-Thought
Linked Clues
Figure 2: The details of game scripts generated via our multi-agent framework. Verifiable Type Score Long-script QA
0.3
0.3
Format Score
Multi-Modal QA
Score
0.5
Score 0.7
Unverifiable Type Score Role-playing
Correctness Score
Score
Reward model
Match Score
0.2
Thought string
Role-playing data
Clues list
ScoreAgent
Answer option
SFT Evaluation QA data
rollout
Stage 1: Supervised Fine-tuning
Stage 2: Reinforcement Learning
update Enhanced-VLM
Figure 3: The bottom part outlines the two-stage training strategy. The top part showcases the ScoreAgent design, which applies specialized reward functions to different training data types for reward calculation during reinforcement learning. Self-Introduction
Discussion
Cai Wenhua(innocent):
Cai Wenhua(innocent):
Hello everyone, I am Cai Wenhua, 40 years old, …. I did want to kill Boss Zhen, but someone got there first. I am a low-key person and I don't pay much attention to other people's affairs. I usually listen to music with headphones. Bai Zhuangyuan(murder): Hello everyone, I am Bai Zhuangyuan, a top student who graduated from Peking University….,I really have no motive to kill, and I don’t have the time to commit the crime. I admit that I killed Boss Zhen, but I had my reasons for doing so. I hope everyone can forgive me.
{“chosen_player”: “#b#.Sa Bawang", "question": “Sa Bawang, you said you saw the ghost girl fainted on the ground at 21:15, but according to my observation, she was still. Can you describe in detail what you saw? ", "clues share": "I noticed that the ghost girl was wearing a cabbage jade pendant around her neck that was exactly the same as the one I gave to Xiaofang, which made me suspect that she might be my daughter.“} {“chosen_player”: “#a#. Cai Wenhua", "question": “Cai Wenhua, What did you eat tonight? What do you like to do?", "clues share": " I noticed that Cai Wenhua is a person who likes sports and food.“ }
Figure 4: Red denotes low-scoring and green denotes high-scoring examples. Low scores arise from off-topic, self-contradictory, or rule-violating behaviors in selfintroductions and discussions, such as irrelevance to the script or premature identity revelation.
3.3.2
Reinforcement Learning
Murder Mystery Games involve role-consistency behavior under imperfect information, requiring truthful cooperation from innocent players and strategic deception from the murderer—a setting where existing VLMs exhibit clear limitations. Since self-introductions and discussions do not have a single correct answer and are highly context-
dependent, supervised fine-tuning (SFT) alone is insufficient to meet these requirements. Instead of training an independent reward model, we adopt an LLM-as-Judge approach (Zhu et al., 2023; Li et al., 2025a; Whitehouse et al., 2025). Specifically, the ScoreAgent introduced earlier is implemented by prompting a powerful LLM to score candidate interactions—assigning higher rewards to identityconsistent self-introductions and meaningful discussions, while assigning lower rewards to irrelevant or rule-violating responses, as illustrated in Figure 4. For evaluation on unverifiable data types , i.e., generated self-introduction and discussion where no ground truth exists, we design the reward as: M(R), for Self-Introduction, S= (1) γM(R) + (1 − γ)Schoice (R), for Discussion. We define the auxiliary reward components as fol-
lows: Sformat (R) equals 1 if the response R is in valid JSON format, and 0 otherwise; Smatch (R) equals 1 if all referenced image clues are correctly matched, and 0 otherwise; Schoice (R) equals 1 if the player chooses to ask the murder suspect, 0.5 if asking another player, and 0 if asking themselves. For evaluation on verifiable data type (e.g., with standard answers for long script QA and multimodal QA), we define the reward functions as follows: 1(αScorrect > β) · Scorrect (A, Â) +(1 − α)Sformat (R) , (a), S= 1(αScorrect > β) · Scorrect (A, Â) 1 + 2 (1 − α) Sformat (R) + Smatch (R) , (b). (2) (a) for Long Script QA and (b) for Multimodal QA. Here, α balances answer correctness with format and clue matching, while β imposes a correctness threshold to discourage reward hacking based solely on format adherence. For multimodal QA, Smatch encourages correct identification and grounding in relevant visual clues, enabling the model to filter irrelevant information and improve reasoning accuracy. With reward functions defined above, we optimize the policy using GRPO (DeepSeek-AI, 2025) without the KL penalty term. For each prompt (C, Qi ) we sample G responses (actions) R1 , . . . , RG ∼ πθold (·|C, Qi ), compute their rewards ri = S(Ri ), then form standardized advantages:
4
Experiments
we conduct a series of experiments on both synthetic data and human-annotated data to assess the effectiveness of our proposed framework. 4.1
Experimental Settings
Implementation details. We evaluate our framework on both Qwen2.5-VL-3B-Instruct and Qwen2.5-VL-7B-Instruct (Bai et al., 2025), demonstrating its effectiveness across different model scales. Given the extremely long contexts and highly variable numbers of image clues in Murder Mystery Games—ranging from 5 to 82 per script—we set the context window to 65,536 tokens during training and uniformly resize all image clues to 512 × 512. Additional details of the data synthesis and training setup are provided in Appendix C.
Metrics Our evaluation metrics are organized into three categories. Reasoning & Analysis includes Multi-hop Multimodal Reasoning (MMR), which evaluates multi-hop reasoning over heterogeneous evidence, and Case Murder Detail (CMD), which measures the quality of open-ended explanations of the murderer’s actions and motives, scored by DeepSeek-R1 against reference answers on a 100-point scale. Role-playing & Decision comprises Role-Playing (RP), assessing the coherence and naturalness of role-playing dialogues on a 10point scale, and Decision-Making (DM), evaluating the accuracy of identifying the murderer in the final vote. Perception includes Long-script Understanding (LSU) for long-context comprehension, Text-rich Image Understanding (TIU) for extracts X X ri − µ ing clues from text-dense images, and Media-rich µ = G1 ri , σ = G1 (ri − µ)2 , ai = , Image Understanding (MIU) for integrating textual σ i i and visual information in complex images. (3) Baselines We compare our trained model against both proprietary and open-source models: (1) ProG prietary VLMs: GPT-4V, Gemini 1.5 Pro, and X 1 i |C,Qi ) min ππθ θ (R , Claude; (2) Open-source VLMs: Mistral-small3.1(Ri |C,Qi ) old G i=1 24B, Gemma3-27B-it, Gemma3-12B-it, LLaVA i |C,Qi ) 13B, LLaMA3.2-Vision-11B, Qwen2.5-VL-7Bclip ππθ θ (R , 1 − ϵ, 1 + ϵ a . i (Ri |C,Qi ) old (4) Instruction, Gemma3-4B-it, and Qwen2.5-VL-3Bwhere ϵ governs the clip range, preserving sta- Instruction. bility. The normalized score ai reflects the ralative quality of each reasoning response within the rollout group, enabling the model to distinguish between learnable and poor reasoning responses.
4.2
Main results
Figure 5 shows consistent reward improvements across all subtasks, validating the effectiveness of
Method
Reasoning & Analysis
Role-playing & Decision
MMR
RP
CMD
Perception
DM
LSU
TIU
MIU
24.2% 16.9% 19.2%
92.40 88.80 88.80
51.88 57.78 35.31
69.25 57.84 55.02
15.83% 33.09% 15.83% 19.50% 11.69% 25.61% 17.09% 20.14%
76.34 83.07 23.92 81.25 57.20 83.97 34.70 72.88
64.24 50.74 21.35 65.43 24.13 40.63 23.50 35.09
56.42 48.41 18.70 55.01 22.60 38.74 22.38 32.16
82.04 85.27 77.13 87.40 +14.52
69.76 61.97 71.01 74.56 +39.56
60.10 58.88 44.05 61.09 +28.93
83.27 86.41 78.66 89.15⋆ +5.18
71.02 63.85 72.48 77.28⋆ +36.65
61.42 59.73 45.23 62.53 ⋆ +23.79
Proprietary VLMs GPT-4V Gemini-1.5-Pro Claude
58.75 57.39 57.78
Gemma3-27B-it Mistral-small3.1-24B LLaVA-13B Gemma3-12B-it LLaMA3.2-Vision-11B Qwen2.5-VL-7B-Instruct (baseline) Gemma3-4B-it Qwen2.5-VL-3B-Instruct (baseline)
48.28 44.92 19.01 49.96 33.71 37.63 40.53 30.92
26.43 19.20 22.07
6.43 7.22 7.89
Open-source VLMs 26.91 40.43 20.78 33.22 26.39 30.70 18.21 23.93
7.61 7.53 2.17 7.34 3.95 7.11 7.16 4.69
Ours and Ablations (Qwen2.5-VL-3B-Instruct) w/o Supervised Fine-tuning w/o Reinforcement Learning w/o Image Clues Match Ours (Full Model) Improvement vs. 3B
48.56 45.83 48.75 55.01 +24.09
27.84 17.02 33.25 34.25 +10.32
5.32 4.76 5.15 6.35 +1.66
34.32% 24.62% 31.25% 35.00% +14.86%
Ours and Ablations (Qwen2.5-VL-7B-Instruct) w/o Supervised Fine-tuning w/o Reinforcement Learning w/o Image Clues Match Ours (Full Model) Improvement vs. 7B
50.12 47.40 50.22 58.42⋆ +20.79
29.36 18.50 34.91 36.18 +5.48
5.68 5.12 5.44 6.82 -0.29
35.48% 26.13% 32.54% 36.87% +11.26%
Table 1: Performance comparison across seven metrics grouped by capability types. Bold denotes the best result and underline indicates the second-best result across all methods. ⋆ indicates that our model surpasses all open-source VLMs. Results are reported for both Qwen2.5-VL-3B-Instruct and Qwen2.5-VL-7B-Instruct backbones. Multimodal QA
0.8 0.6 0.4 0.2 0.0
Long Scripts QA 1.0
0
Raw Smoothed 2000 4000 6000 8000 10000 Step Self-Introduction
Reward Score
Reward Score
1.0
0.2 0.1 0.0
0
250
500
Raw Smoothed 750 1000 1250 1500 Step
0.7 0.6 0
1.0 Reward Score
Reward Score
0.3
0.8
0.5
0.5 0.4
0.9
Raw Smoothed 2000 4000 6000 8000 10000 Step Discussion
0.8 0.6 0.4 0.2 0
2000
4000 Step
Raw Smoothed 6000 8000
Figure 5: Training reward curves for verifiable subtasks (Multimodal and Long Scripts QA, top row) and unverifiable role-playing tasks (Self-Introduction and Discussion, bottom row).
our RL-based optimization. Verifiable tasks converge to higher and more stable rewards, while role-playing tasks (Self-Introduction and Discussion) exhibit lower plateaus and higher variance, reflecting the inherent subjectivity of dialogue behaviors.
Table 1 further evaluates our framework across seven metrics on Murder Mystery Games. On both Qwen2.5-VL-3B-Instruct and Qwen2.5-VL7B-Instruct, our full model consistently outperforms the corresponding open-source baselines and ablations, demonstrating the robustness and scalability of the proposed framework. Compared with strong open-source VLMs such as Gemma3-27B-it and Mistral-small3.1-24B, which perform reasonably well on text-centric tasks, our model achieves substantially better results on Multi-hop Multimodal Reasoning (MMR) and Decision-Making (DM), where deep integration of multimodal evidence and role-consistent reasoning is required. Notably, scaling the backbone from 3B to 7B yields consistent gains across most metrics, with the 7B full model achieving the best overall MMR score and further improvements in perception and decision-making tasks. In comparison with proprietary VLMs, our model attains competitive or superior performance in perception tasks, underscoring the effectiveness of agent-driven data synthesis and training for complex multimodal reasoning under
long contexts and imperfect information.
Data
4.3
Human 51.03 30.72 6.04 32.21% 79.05 69.61 56.42 Synthetic 48.35 27.90 5.41 34.21% 82.02 68.45 54.52 Human+Syn 55.01 34.25 6.35 35.00% 87.40 74.56 61.09
Ablation Study
Ablation on training components We first conduct ablation studies on Qwen2.5-VL-3B-Instruct and Qwen2.5-VL-7B-Instruct to examine the contributions of different training components, as summarized in Table 1. Removing supervised finetuning (SFT) leads to consistent performance degradation across reasoning and perception metrics, highlighting its role in initializing long-context understanding and multimodal alignment. Excluding reinforcement learning (RL) results in substantial drops in multi-hop reasoning and case analysis(CMD) performance, indicating that RL is critical for refining evidence selection and decisionmaking behaviors. Besides, removing the imageclue matching reward degrades multimodal perception and reasoning, especially on MIU and MMR, confirming its importance for filtering irrelevant visual information and grounding reasoning in pertinent clues. Ablation on Training Data Sources To further analyze the impact of training data composition, we conduct additional ablation experiments by finetuning the model using only human-annotated data or only synthetic data, while keeping all other settings identical. Importantly, all evaluation datasets are human-annotated, avoiding any risk of circular evaluation. As shown in Table 2, both settings improve substantially over the base models, indicating that synthetic data alone can already enhance multimodal reasoning and decision-making. However, training with either data source alone consistently underperforms the full setting that combines humanannotated and synthetic data. This complementarity suggests that human annotations provide highquality grounding and supervision, while synthetic data generated by our multi-agent framework enriches interaction diversity and reasoning patterns. Together, they yield the strongest and most balanced performance across all evaluation metrics. 4.4
Analysis of LLM-as-Judge Evaluation with Human Judgments
We adopt the LLM-as-Judge paradigm exclusively for unverifiable evaluation settings (i.e., RP and CMD), where no single ground-truth answer exists and evaluation necessarily relies on structured but subjective criteria. Importantly, LLM-based
MMR CMD RP
DM
LSU TIU MIU
Table 2: Ablation on training data sources using Qwen2.5-VL-3B-Instruct with all evaluation sets are human-annotated.
Judge Pair
CMD r RP r
GPT-4o ↔ DeepSeek-r1 Gemini-2.5-Pro ↔ DeepSeek-r1 GPT-4o ↔ Gemini-2.5-Pro
0.83 0.79 0.87
0.68 0.64 0.72
Table 3: Pairwise Pearson correlation between different LLM judges (p < 10−2 for all cases).
judging is used only during training and evaluation, and does not introduce additional cost at inference time. To assess evaluation reliability and potential bias, we employ three independent LLM judges—DeepSeek-r1, GPT-4o, and Gemini-2.5Pro—and conduct analysis on 100 randomly sampled instances per task. Inter-judge agreement reaches moderate-to-substantial levels, with Cohen’s κ of 0.58 for CMD and 0.47 for RP. As shown in Table 3, pairwise Pearson correlations are consistently high, indicating strong agreement across models with different architectures and training distributions. This suggests that the reward signal is not dominated by any single judge model. We further evaluate alignment with human judgment. As shown in Table 4, the aggregated LLM scores (mean of three judges) achieve strong rank correlation with human evaluations. Beyond correlation, we analyze absolute score deviations in Table 5. A large proportion of predictions fall within small deviation ranges (e.g., within 1–2 points for RP and 3 points for CMD), indicating close quantitative agreement. Even when using a single judge (DeepSeek-r1), the distribution remains comparable, suggesting robustness of the evaluation signal. Overall, these results demonstrate that LLM-asJudge provides a stable, consistent, and humanaligned evaluation mechanism. While individual judges may exhibit minor variations, aggregation across multiple models effectively mitigates bias and yields reliable supervision.
Task CMD RP
Spearman ρ
References
p-value −2
< 10 < 10−2
0.71 0.62
Table 4: Correlation between human judgments and aggregated LLM-as-Judge scores. Task
-1∼0
1∼2
3∼4
>4
RP (Human vs. Avg LLM) RP (Human vs. DeepSeek)
63% 58%
23% 24%
10% 12%
4% 6%
Task
-2∼0
1∼3
4∼6
>6
CMD (Human vs. Avg LLM) CMD (Human vs. DeepSeek)
48% 46%
32% 30%
15% 18%
5% 6%
Table 5: Distribution of score differences between human judgments and LLM-based evaluations.
5
Conclusion
We present a multi-agent collaborative framework and a two-stage fine-tuning strategy to enhance VLMs for complex reasoning and role-playing tasks in Murder Mystery scenarios. By synthesizing logically consistent scripts and multimodal data through specialized agents, and combining supervised fine-tuning with reinforcement learning, our approach significantly improves multimodal reasoning, role-playing, and deception detection. Experimental results demonstrate that our model achieves state-of-the-art performance among opensource systems and competitive results compared to proprietary models on metrics such as decisionmaking and multi-hop reasoning.
6
Limitations
While the proposed framework shows strong potential, several limitations remain. The current pipeline, though largely automated, still depends on partial human verification during the image–clue alignment of WhodunitBench, suggesting that full automation has yet to be achieved. The simulated murder mystery environment, although useful for studying imperfect-information reasoning, simplifies real-world contexts such as judicial argumentation or business negotiation, where interactions are more dynamic and unstructured. Future work should expand testing to more realistic domains, and establish clearer ethical guidelines to ensure responsible development and application.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. Qwen2.5-vl technical report. Preprint, arXiv:2502.13923. Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Xing Gao, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang, and Jingren Zhou. 2024a. Socialbench: Sociality evaluation of role-playing conversational agents. Preprint, arXiv:2403.13679. Kang Chen, Zheng Lian, Haiyang Sun, Rui Liu, Jiangyan Yi, Bin Liu, and Jianhua Tao. 2024b. Can deception detection go deeper? dataset, evaluation, and benchmark for deception reasoning. Preprint, arXiv:2402.11432. Zhiwei Chen, Yupeng Hu, Zhiheng Fu, Zixu Li, Jiale Huang, Qinlei Huang, and Yinwei Wei. 2026. Intent: Invariance and discrimination-aware noise mitigation for robust composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 20463–20471. Zhiwei Chen, Yupeng Hu, Zixu Li, Zhiheng Fu, Xuemeng Song, and Liqiang Nie. 2025. Offset: Segmentation-based focus shift revision for composed image retrieval. In Proceedings of the ACM International Conference on Multimedia, page 6113–6122. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948. Google Gemini Team. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. Technical report. DeepMind Google. 2023. Introducing gemini: our largest and most capable ai model. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2025a. From generation to judgment: Opportunities and challenges of llm-as-a-judge. Preprint, arXiv:2411.16594. Zixu Li, Yupeng Hu, Zhiwei Chen, Shiqi Zhang, Qinlei Huang, Zhiheng Fu, and Yinwei Wei. 2026. Habit: Chrono-synergia robust progressive learning framework for composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 6762–6770.
Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. 2025b. A survey of state of the art large vision language models: Alignment, benchmark, evaluations and challenges. arXiv preprint arXiv:2501.02189. Zeyu Liu, Yuhang Liu, Guanghao Zhu, Congkai Xie, Zhen Li, Jianbo Yuan, Xinyao Wang, Qing Li, Shing-Chi Cheung, Shengyu Zhang, Fei Wu, and Hongxia Yang. 2025. Infi-mmr: Curriculum-based unlocking multimodal reasoning via phased reinforcement learning in multimodal small language models. Preprint, arXiv:2505.23091. Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei ge Chen, Olga Vrousgos, Corby Rosset, Fillipe Silva, Hamed Khanpour, Yash Lara, and Ahmed Awadallah. 2024. Agentinstruct: Toward generative teaching with agentic flows. Preprint, arXiv:2407.03502. OpenAI. 2023. Gpt-4v(ision) system card. System card, OpenAI. Published September 25, 2023. Davide Paglieri, Bartłomiej Cupiał, Sam Coward, Ulyana Piterbarg, Maciej Wołczyk, Akbir Khan, Eduardo Pignatelli, Łukasz Kuciński, Lerrel Pinto, Rob Fergus, Jakob Nicolaus Foerster, Jack Parker-Holder, and Tim Rocktäschel. 2024. Benchmarking agentic llm and vlm reasoning on games. arXiv preprint arXiv:2411.13543. Guozhi Qiu, Zhiwei Chen, Zixu Li, Qinlei Huang, Zhiheng Fu, Xuemeng Song, and Yupeng Hu. 2026. Melt: Improve composed image retrieval via the modification frequentation-rarity balance network. arXiv preprint arXiv:2603.29291. Yan Rong, Jinting Wang, Shan Yang, Guangzhi Lei, and Li Liu. 2025. Audiogenie: A training-free multi-agent framework for diverse multimodality-to-multiaudio generation. arXiv preprint arXiv:2505.22053. Jiajun Shi, Jian Yang, Jiaheng Liu, Xingyuan Bu, Jiangjie Chen, Junting Zhou, Kaijing Ma, Zhoufutu Wen, Bingli Wang, Yancheng He, Liang Song, Hualei Zhu, Shilong Li, Xingjian Wang, Wei Zhang, Ruibin Yuan, Yifan Yao, Wenjun Yang, Yunli Wang, and 10 others. 2025. Korgym: A dynamic game platform for llm reasoning evaluation. Preprint, arXiv:2505.14552. Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. 2025. Reason-rft: Reinforcement fine-tuning for visual reasoning. Preprint, arXiv:2503.20752. Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin, Xiaowen Dong, Yanfeng Wang, and Siheng Chen. 2025. Synthesizing post-training data for LLMs through multi-agent simulation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 23306–23335, Vienna, Austria. Association for Computational Linguistics.
Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. 2025. Multimodal chain-of-thought reasoning: A comprehensive survey. Preprint, arXiv:2503.12605. Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. 2024. Genartist: Multimodal llm as an agent for unified image generation and editing. Advances in Neural Information Processing Systems, 37:128374– 128395. Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. 2025. J1: Incentivizing thinking in llmas-a-judge via reinforcement learning. Preprint, arXiv:2505.10320. Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. 2025. Reinforcing spatial reasoning in vision-language models with interwoven thinking and visual drawing. Preprint, arXiv:2506.09965. Shuang Wu, Liwen Zhu, Tao Yang, Shiwei Xu, Qiang Fu, Yang Wei, and Haobo Fu. 2024. Enhance reasoning for large language models in the game werewolf. Preprint, arXiv:2402.02330. Junlin Xie, Ruifei Zhang, Zhihong Chen, Xiang Wan, and Guanbin Li. 2024. Whodunitbench: Evaluating large multimodal agents via murder mystery games. Advances in Neural Information Processing Systems, 37:86655–86687. Zelai Xu, Zhexuan Xu, Xiangmin Yi, Huining Yuan, Xinlei Chen, Yi Wu, Chao Yu, and Yu Wang. 2025a. Vs-bench: Evaluating vlms for strategic reasoning and decision-making in multi-agent environments. Preprint, arXiv:2506.02387. Zixiang Xu, Yanbo Wang, Yue Huang, Jiayi Ye, Haomin Zhuang, Zirui Song, Lang Gao, Chenxi Wang, Zhaorun Chen, Yujun Zhou, Sixian Li, Wang Pan, Yue Zhao, Jieyu Zhao, Xiangliang Zhang, and Xiuying Chen. 2025b. Socialmaze: A benchmark for evaluating social reasoning in large language models. Preprint, arXiv:2505.23713. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600. Mingyu Zhang, Zixu Li, Zhiwei Chen, Zhiheng Fu, Xiaowei Zhu, Jiajia Nie, Yinwei Wei, and Yupeng Hu. 2026a. Hint: Composed image retrieval with dualpath compositional contextualized network. arXiv preprint arXiv:2603.26341. Wenyuan Zhang, Xinghua Zhang, Haiyang Yu, Shuaiyi Nie, Bingli Wu, Juwei Yue, Tingwen Liu, and Yongbin Li. 2026b. Expseek: Self-triggered experience seeking for web agents. Preprint, arXiv:2601.08605.
Xiaojiang Zhang, Jinghui Wang, Zifei Cheng, Wenhao Zhuang, Zheng Lin, Minglei Zhang, Shaojie Wang, Yinghan Cui, Chao Wang, Junyi Peng, Shimiao Jiang, Shiqi Kuang, Shouyu Yin, Chaohang Wen, Haotian Zhang, Bin Chen, and Bing Yu. 2025a. Srpo: A crossdomain implementation of large-scale reinforcement learning on llm. Preprint, arXiv:2504.14286. Xinjie Zhang, Jintao Guo, Shanshan Zhao, Minghao Fu, Lunhao Duan, Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang. 2025b. Unified multimodal understanding and generation models: Advances, challenges, and opportunities. arXiv preprint arXiv:2505.02567. Yuyao Zhang, Jinghao Li, and Yu-Wing Tai. 2025c. Layercraft: Enhancing text-to-image generation with cot reasoning and layered object integration. arXiv preprint arXiv:2504.00010. Zheng Zhang, Nuoqian Xiao, Qi Chai, Deheng Ye, and Hao Wang. 2025d. Multimind: Enhancing werewolf agents with multimodal reasoning and theory of mind. arXiv preprint arXiv:2504.18039. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. Qinglin Zhu, Runcong Zhao, Bin Liang, Jinhua Du, Lin Gui, and Yulan He. 2025. Player*: Enhancing llmbased multi-agent communication and interaction in murder mystery games. Preprint, arXiv:2404.17662.
A
Appendix Agent Prompt Settings
This appendix provides the detailed prompt settings and configuration parameters for each specialized agent in the interactive murder mystery script generation pipeline. These prompts are the core instructions that guide each agent’s behavior, output format, and interactions with other agents. A.1
OutlineAgent
Outline is responsible for constructing the initial narrative framework. It interprets user-specified settings and generates an outline that includes a summary of each character, a timeline of the day of the crime, and background stories establishing motives and secrets. The system prompt used in OutlineAgent is presented in Figure 9 A.2
CharacterAgent
The CharacterAgent is designed to generate detailed character profiles, ensuring that each character has distinct traits, motivations, and secrets. The system prompt guiding the CharacterAgent is shown in Figure 10.
A.3
CriticAgent
The CriticAgent evaluates the coherence, plausibility, and narrative structure of the generated content, offering constructive feedback to refine the overall script. The system prompt used by the CriticAgent is detailed in Figure 11. A.4
ClueAgent
The ClueAgent generates a set of public multimodal clues that reflect critical but non-obvious details of the environment and storyline. These clues are designed to aid in the deduction process without explicitly revealing the murderer, ensuring meaningful contributions to the overall narrative. Figure 12 illustrates the ClueAgent’s system prompt. A.5
QaAgent
The QaAgent operates in a systematic and layered manner to construct a diverse set of questionanswer pairs, designed to evaluate and enhance the VLM’s perception and reasoning capabilities. Its workflow consists of the following key steps: 1. Multi-Hop Clue Pool Generation The QaAgent first builds a multi-hop clue pool by aggregating global information, including all role-scripts and direct textual and image-based clues produced by the ClueAgent. This clue pool serves as the foundation for generating more complex, multistep reasoning question-answer pairs. 2. Question Generation Leveraging the information from the multi-hop clue pool and other sources, the QaAgent creates a variety of questions tailored to test different aspects of the VLM’s capabilities: • Long Script QA: These questions are derived from all role-scripts, challenging the VLM to comprehend and reason across extensive narrative contexts. • Multi-Modal QA: This category includes both text-rich(This metric assesses agents’ proficiency in precisely interpreting and extracting clues from text-rich images, emphasizing their Optical Character Recognition (OCR) capabilities) and media-rich questions(This metric evaluates how effectively agents integrate textual and visual elements to interpret and understand more complex clues within images, which may include diagrams, maps or residential layouts. It aims to gauge