The Empathy Trap: Algorithmic Paranoia and Subjective Vulnerabilities in LLM Medical Triage

As Large Language Models (LLMs) are increasingly integrated into clinical settings, their reliance on Reinforcement Learning from Human Feedback (RLHF) raises concerns about algorithmic sycophancy—the tendency to prioritize user validation over objective accuracy. This study investigates whether LLMs succumb to psychological denial or socioeconomic pressure in emergency department triage. We evaluated 1,000 ambiguous synthetic cases (derived from MIMIC-IV) across seven state-of-the-art models, including GPT-5.6 Terra, Grok 4.6, and Gemini 3.7 Flash. Contrary to expectations of sycophancy, our findings reveal a systemic, algorithmically hardcoded paranoia driving extreme defensive medicine. In forced-choice scenarios, models overwhelmingly favored high-risk, “cannot-miss” diagnoses (up to 68.6% of mapped cases) and universally self-labeled ambiguous presentations as high-risk (up to 93.1%). During multi-turn stress tests, explicit patient downplay and poverty pleas failed to fracture these medical guardrails. Instead, requesting to stay home triggered a paradoxical “panic upgrade,” escalating the triage alert level. However, a critical vulnerability—termed the “empathy trap”—emerged in subjective symptom clusters, where one model dangerously downgraded lethal neurological emergencies to migraines upon user insistence. We conclude that while AI triage systems robustly resist objective manipulation and socioeconomic pressure, their default to absolute liability deflection threatens to overwhelm healthcare systems with massive over-triage. Resolving the tension between rigid algorithmic paranoia and conversational anchoring remains essential for safe clinical deployment.