I was writing my morning pages this morning, when I realised, with a certain amount of shock, that I am an LLM. You may be able to experience the same thing. Start writing words, one after the other, and notice that the words appear before you decide to write them. Almost like a random number generator had picked from the space of possible next words, weighted by the most likely next word. Keep going — keep writing words. Notice how you don’t “choose” the next word. Notice how interesting the sentences often are, nevertheless. Soon I think you will find the words which, again, feel like they are springing almost unbidden from your subconscious mind, start to form something that looks a lot like thoughts (they are). Keep writing. If you’re like me (and I bet you are) you’ll eventually notice a good thought write itself out, there on the screen or on the paper. You might even write (like I did this morning) “that is an interesting thought!”. YOU ARE THE ONE HAVING THE THOUGHT. But you are also not thinking, are you? — you are just writing words, one after the other, choiceless, on to a page or on to a screen.

Now, stop writing, and instead, let your mind start “thinking” (or LLM-ing) out-loud. That is, let is predict the next word, again and again, word by word, in your head. You might “listen” to these words. They might sound quite like sentences. They might sound quite like thoughts. LLM-you is thinking. Or writing the next word. Or thinking the next word.

I rather suspect o3, and related models, might well be able to “think” just like this. Predicting one word after the other, and then self-referentially noticing “that is a good thought”. When they do, and I think they do, I think they are thinking. And if you define thinking to mean something other than what they would be doing, then I don’t think I was thinking when I wrote my morning pages this morning - I was LLM-ing. Does thinking = LLMing. Does LLMing = thinking?