Eliciting Medical Reasoning with Knowledge-enhanced Data Synthesis: A Semi-Supervised Reinforcement Learning Approach Haolin Li1,2 , Shuyang Jiang1,2 , Ruipeng Zhang5 , Jiangchao Yao3,4 Ya Zhang4,2,6 , Yanfeng Wang4B , 1
College of Computer Science and Artificial Intelligence, Fudan University 2 Shanghai AI Laboratory CMIC, Shanghai Jiao Tong University 4 School of Artificial Intelligence, Shanghai Jiao Tong University 5 Department of Radiology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine 6 Institute of Artificial Intelligence for Medicine, Shanghai Jiao Tong University School of Medicine 3
Abstract
Memorization-Example:
arXiv:2604.11547v1 [cs.LG] 13 Apr 2026
Is Wilson's disease described as an iron storage disease?
While large language models hold promise for complex medical applications, their development is hindered by the scarcity of highquality reasoning data. To address this issue, existing approaches typically distill chainof-thought reasoning traces from large proprietary models via supervised fine-tuning, then conduct reinforcement learning (RL). These methods exhibit limited improvement on underrepresented domains like rare diseases while incurring substantial costs from generating complex reasoning chains. To efficiently enhance medical reasoning, we propose MedSSR, a Medical Knowledge-enhanced data Synthesis and Semi-supervised Reinforcement learning framework. Our framework first employs rare disease knowledge to synthesize distribution-controllable reasoning questions. We then utilize the policy model itself to generate high-quality pseudo-labels. This enables a two-stage, intrinsic-to-extrinsic training paradigm: self-supervised RL on the pseudo-labeled synthetic data, followed by supervised RL on the human-annotated real data. MedSSR scales model training efficiently without relying on costly trace distillation. Extensive experiments on Qwen and Llama demonstrate that our method outperforms existing methods across ten medical benchmarks, achieving up to +5.93% gain on rare-disease tasks. Our code is available at https://github.com/tdlhl/MedSSR.
1
Introduction
Large language models (LLMs) have demonstrated remarkable reasoning capabilities (Brown et al., 2020; Wei et al., 2022; Achiam et al., 2023; Chiang et al., 2023), spurring the development of diverse post-training methods (Ouyang et al., 2022; Rafailov et al., 2023). Among them, various methods have been designed to enhance the reasoning B: Corresponding author
Memorization 151767, 78%
Rare Disease 5889, 3%
Reasoning 42729, 22%
Reasoning-Example: A newborn infant comes to the attention of the neonatal care unit because he started having heavy and rapid breathing. In addition, he was found to be very irritable with pale skin and profuse sweating. Finally, he was found to have cold feet with diminished lower extremity pulses. Cardiac auscultation reveals a harsh systolic murmur along the left sternal border. Notably, the patient is not observed to have cyanosis. Which of the following treatments would most likely be effective for this patient's condition? A. Leukotriene E4 B. Prostaglandin E1 C. Prostaglandin E2 D. Prostaglandin I2 E. Thromboxane A2
Figure 1: Distribution of question types in existing medical datasets. Only 22% are reasoning-intensive, and just 3% among them concern rare diseases.
ability of LLMs in the medical field, aiming to address complex medical problems (Zhang et al., 2023, 2024). However, unlike natural domains where large amounts of reasoning-intensive data are available, downstream domains like healthcare inherently suffer from a scarcity of reasoningintensive data (Thapa et al., 2025; Chen et al., 2025d; Li et al., 2025). Most existing medical benchmarks consist primarily of memorizationoriented questions, rather than complex reasoning cases. This issue is especially acute for underrepresented subfields like rare diseases, where labeled data is extremely limited (see Figure 1). Due to privacy constraints and the specialized expertise required, acquiring complex medical reasoning data remains exceptionally challenging, yet critical for real-world clinical applications. Building on this challenge of data scarcity, existing methods for enhancing medical reasoning primarily use large-scale proprietary models to generate chain-of-thought (CoT) reasoning traces (Chen et al., 2025b). The generated traces are then used in supervised fine-tuning to initialize the reason-
SFT
Self-labeling
RL
Policy Model Avg Token/Sample > 1300
Generated Questions
Real Questions