Further musings on AI

When there’s a hammer in your hand, everything looks like a nail. And so I see people around me, people who I’ve known to be clever and capable software engineers, start using AI for things that I frankly find concerning. Writing an MCP server for what could have been a shell script may just be wasteful — it’s like the kid who just got their driver’s license and now they’re driving everywhere, even to the corner store. But some people are outsourcing even the simplest tasks to AI now instead of thinking about it themselves. Sure, a calculator is super useful to get the square root of some large number, or the cosine of a non-well-known value. But then soon enough people can’t even subtract 7 from 13 because they’re so used to the machine calculating everything for them.

Some people argue along the lines of “it’s not intelligent, it’s just a word completion engine”. However, we know very well that trivially simple building blocks can give rise to deeply complex phenomena, see for example the ant colonies in Hofstader’s Gödel, Escher, Bach. So while it is true that large language models are token prediction engines, this does not preclude them from being intelligent.

So are they? My take is yes — insofar as they can provide the correct solution or answer to questions that we all would agree require intelligence to anwer. What they are not is consciuous.

In much of the same vein, the people who argue that today’s LLMs are quite intelligent as evidenced by their results in mathematics competitions and those who argue that they are stupid since they can’t even count the number of ‘r’s in ‘strawberry’ are both correct in their particular claims about the models’ certain capabilities and both wrong about the way they generalize these. LLMs can indeed solve very complex tasks while struggling with trivially simple ones.

The problem here is that we rely on our intuition of how intelligence works. But the rules that hold true for humans simply do not apply the same way to language models.

More generally: the LLM talks like a person, so we’re assume it works and behaves like one. This is incorrect. The way AI works subverts our intuition, and we haven’t yet internalized the proper mental models for how to think about them.

I see lots of people remarking that “The job was never to write. The job was to solve problems.” which is not untrue, but ‘solving problems’ is way too high-level and vague to be helpful in any way. My car mechanic solves problems. The dentist solves problems. The Wolf solves problems. They have nothing in common.

So don’t tell me the job was not to write code.
It was in the job description. It was on the interview. It was what we all were doing in a large portion of the time. Of course the job was to write code. It was lots of other things too, sure, but dismissing someone’s lament about losing the coding portion of the job by saying the job is to solve problems is like telling an amputee they have nothing to be disappointed about because the task is to get from point A to point B, not to walk there specifically.