Human-like social performance does not imply human-like preference inference in large language models

As large language models (LLMs) increasingly act on people’s behalf, good decisions may not indicate accurate inference of human preferences. Across three social inference tasks, we show that decision quality and preference inference can dissociate. In a multi-attribute bargaining task, LLMs learned