Further musings on AI
When there’s a hammer in your hand, everything looks like a nail. And so I see people around me, people who I’ve known to be clever and capable software engineers, start using AI for things that I frankly find concerning. Writing an MCP server for what could have been a shell script may just be wasteful — it’s like the kid who just got their driver’s license and now they’re driving everywhere, even to the corner store. But some people are outsourcing even the simplest tasks to AI now instead of thinking about it themselves. Sure, a calculator is super useful to get the square root of some large number, or the cosine of a non-well-known value. But then soon enough people can’t even subtract 7 from 13 because they’re so used to the machine calculating everything for them.
Some people argue along the lines of “it’s not intelligent, it’s just a word completion engine”. However, we know very well that trivially simple building blocks can give rise to deeply complex phenomena, see for example the ant colonies in Hofstader’s Gödel, Escher, Bach. So while it is true that large language models are token prediction engines, this does not preclude them from being intelligent.
So are they? My take is yes — insofar as they can provide the correct solution or answer to questions that we all would agree require intelligence to anwer. What they are not is consciuous.
In much of the same vein, the people who argue that today’s LLMs are quite intelligent as evidenced by their results in mathematics competitions and those who argue that they are stupid since they can’t even count the number of ‘r’s in ‘strawberry’ are both correct in their particular claims about the models’ certain capabilities and both wrong about the way they generalize these. LLMs can indeed solve very complex tasks while struggling with trivially simple ones.
The problem here is that we rely on our intuition of how intelligence works. But the rules that hold true for humans simply do not apply the same way to language models.
More generally: the LLM talks like a person, so we’re assume it works and behaves like one. This is incorrect. The way AI works subverts our intuition, and we haven’t yet internalized the proper mental models for how to think about them.