[A] tendency to make errors doesn’t always indicate a lack of capability. If I make a calculation error when adding two numbers, it doesn’t mean I don’t know how to do arithmetic; if I momentarily mix up Bob Dylan and Dylan Thomas, it doesn’t mean I can’t tell folk music from poetry. . . . [W]e originally owe this distinction between ‘competence’ (what you can in theory do) and ‘performance’ (what you end up doing, including when tired or otherwise impaired) to Chomsky, who needed it to explain why people’s syntax often came out wonky, or at least in ways not predicted by his phrase structure grammar.

Just like humans, LLM performance often falls short of its competence.

Christopher Summerfield, These Strange New Minds: How AI Learned to Talk and What It Means (2024)