Do the thinking models think?

Summary

Whether machines can think is a classic debate that dates back to the intellectual titans of the 1950s. Turing opens his 1950 article Computing Machinery and Intelligence with: I propose to consider the question, "Can machines think?" The Hungarian genius, von Neumann, shortly before his death in 1957, prepared a posthumously published monograph The Computer and the Brain on the same topic. In a 1950 article, Programming a Computer for Playing Chess, Shannon writes: chess is generally considered to require "thinking" for skilful play; a solution of this problem will force us either to admit the possibility of a mechanized thinking or to further restrict our concept of "thinking". Now that people are poking around with ChatGPT, they are stumbling upon the same thing. This, I feel, is a breakthrough development: The new LLMs imitate humans well enough that the users seriously speculate whether these models are truly intelligent. Unfortunately, it seems we are left to rely on conjecture. As illustrated by a philosophical zombie, consciousness (or the absence of it) cannot be empirically verified. On the other hand, decoding the internal logic of a machine learning model by studying the model weights is, in any realistic scenario, an intractable task. So it's a guessing game. Still, I feel there are enough ingredients to make the game interesting. Are LLMs mere autocomplete? Here’s a riddle which, according to a 2022 Harvard paper, originates from a 70s TV series, All in the Family. “A father and his son are in a car accident. The father dies on the spot. The son is rushed to the ER. The attending surgeon looks at the boy and says, ‘I can not operate on this boy. He’s my son!’ How can this be?” The answer is, of course, that the surgeon is the mother. The point of the "riddle" is that the surgeon is, by assumption, male. (Given the 1970s gender stereotypes, maybe this riddle was, in fact, unintuitive back then… but let's not get sidetracked.) This is an archetypal examp...