LLMs as Philosophers: What Can They Do? Why Aren't They Better?

Current LLMs can discuss philosophical ideas, evaluate arguments and perform other analytical tasks at a high level, but are conspicuously bad at producing interesting original philosophy. Why is this? Two tempting diagnoses—that LLMs can't invent new concepts, and that they can't really reason—both look unconvincing on closer inspection. I suggest that a better explanation lies in the structure of reinforcement learning for reasoning. The technique works best in domains like mathematics and coding, where good arguments follow recognizable patterns, correctness is relatively unambiguous, and large corpora of worked examples exist. Philosophy meets none of these conditions especially well, and it's unclear that either AI labs or philosophers have much incentive to develop training resources that might change this.