Does Reasoning Improve Psychological Depth in Large Language Models? It Depends on Who's Judging

LLM-as-a-Judge evaluators are increasingly used to score open-ended generation, yet a judge's correlation with human ratings on its development set may not guarantee valid measurement when outputs are closely matched and human preferences are subjective. We study this failure mode through psychologi