Bag of words, have mercy on us
Look, I donât know if AI is gonna kill us or make us all rich or whatever, but I do know weâve got the wrong metaphor. We want to understand these things as people. When you type a question to ChatGPT and it types back the answer in complete sentences, it feels like there must be a little guy in there doing the typing. We get this vivid sense of âitâs alive!!â, and we activate all of the mental faculties we evolved to deal with fellow humans: theory of mind, attribution, impression management, stereotyping, cheater detection, etc. We canât help it; humans are hopeless anthropomorphizers. When it comes to perceiving personhood, weâre so trigger-happy that we can see the Virgin Mary in a grilled cheese sandwich: A human face in a slice of nematode: And an old man in a bunch of poultry and fish atop a pile of books: Apparently, this served us well in our evolutionary historyâmaybe itâs so important not to mistake people for things that we err on the side of mistaking things for people.1 This is probably why weâre so willing to explain strange occurrences by appealing to fantastical creatures with minds and intentions: everybody in town is getting sick because of WITCHES, you canât see the sun right now because A WOLF ATE IT, the volcano erupted because GOD IS MAD. People who experience sleep paralysis sometimes hallucinate a demon-like creature sitting on their chest, and one explanation is that the subconscious mind is trying to understand why the body canât move, and instead of coming up with âIâm still in REM sleep so thereâs not enough acetylcholine in my brain to activate my primary motor cortexâ, it comes up with âBIG DEMON ON TOP OF MEâ. This is why the past three years have been so confusingâthe little guy inside the AI keeps dumbfounding us by doing things that a human wouldnât do. Why does he make up citations when he does my social studies homework? How come he can beat me at Go but he canât tell me how many ârâs are in the word âstrawberryâ? Why is he telling me to put glue on my pizza?2 Trying to understand LLMs by using the rules of human psychology is like trying to understand a game of Scrabble by using the rules of Pictionary. These things donât act like people because they arenât people. I donât mean that in the deflationary way that the AI naysayers mean it. They think denying humanity to the machines is a well-deserved insult; I think itâs just an accurate description.3 As long we try to apply our person perception to artificial intelligence, weâll keep being surprised and befuddled. We are in dire need of a better metaphor. Hereâs my suggestion: instead of seeing AI as a sort of silicon homunculus, we should see it as a bag of words. An AI is a bag that contains basically all words ever written, at least the ones that could be scraped off the internet or scanned out of a book. When users send words into the bag, it sends back the most relevant words it has. There are so many words in the bag that the most relevant ones are often correct and helpful, and AI companies secretly add invisible words to your queries to make this even more likely. This is an oversimplification, of course. But itâs also surprisingly handy. For example, AIs will routinely give you outright lies or hallucinations, and when youâre like âUhh hey that was a lieâ, they will immediately respond âOh my god Iâm SO SORRY!! I promise Iâll never ever do that again!! Iâm turning over a new leaf right now, nothing but true statements from here onâ and then they will literally lie to you in the next sentence. This would be baffling and exasperating behavior coming from a human, but itâs very normal behavior coming from a bag of words. If you toss a question into the bag and the right answer happens to be in there, thatâs probably what youâll get. If itâs not in there, youâll get some related-but-inaccurate bolus of sentences. When you accuse it of lying, itâs going to produce lots of words from the âIâve been accused of lyingâ part of the bag. Calling this behavior âmaliciousâ or âerraticâ is misleading because itâs not behavior at all, just like itâs not âbehaviorâ when a calculator multiplies numbers for you. âBag of wordsâ is a also a useful heuristic for predicting where an AI will do well and where it will fail. âGive me a list of the ten worst transportation disasters in North Americaâ is an easy task for a bag of words, because disasters are well-documented. On the other hand, âWho reassigned the species Brachiosaurus brancai to its own genus, and when?â is a hard task for a bag of words, because the bag just doesnât contain that many words on the topic.4 And a question like âWhat are the most important lessons for life?â wonât give you anything outright false, but it will give you a bunch of fake-deep pablum, because most of the text humans have produced on that topic is, no offense, fake-deep pablum. When you forget that an AI is just a big bag of words, you can easily slip into acting like itâs an all-seeing glob of pure intelligence. For example, I was hanging with a group recently where one guy made everybody watch a video of some close-up magic, and after the magician made some coins disappear, he exclaimed, âI asked ChatGPT how this trick works, and even it didnât know!â as if this somehow made the magic extra magical. In this personâs model of the world, we are all like shtetl-dwelling peasants and AI is like our Rabbi Hillel, the only learned man for 100 miles. If Hillel canât understand it, then it must be truly profound! If that guy had instead seen ChatGPT as a bag of words, he would have realized that the bag probably doesnât contain lots of detailed descriptions of contemporary coin tricks. After all, magicians make money from performing and selling their tricks, not writing about them at length on the internet. Plus, magic tricks are hard to describeââHe had three quarters in his hand and then it was two pennies!ââso youâre going to have a hard time prompting the right words out of the bag. The coin trick is not literally magic, and neither is the bag of words. The âbag of wordsâ metaphor can also help us guess what these things are gonna do next. If you want to know whether AI will get better at something in the future, just ask: âcan you fill the bag with it?â For instance, people are kicking around the idea that AI will replace human scientists. Well, if you want your bag of words to do science for you, you need to stuff it with lots of science. Can we do that? When it comes to specific scientific tasks, yes, we already can. If you fill the bag with data from 170,000 proteins, for example, itâll do a pretty good job predicting how proteins will fold. Fill the bag with chemical reactions and it can tell you how to synthesize new molecules. Fill the bag with journal articles and then describe an experiment and it can tell you whether anyone has already scooped you. All of that is cool, and I expect more of it in the future. I donât think weâre far from a bag of words being able to do an entire low-quality research project from beginning to endâcoming up with a hypothesis, designing the study, running it, analyzing the results, writing them up, making the graphs, arranging it all on a poster, all at the click of a buttonâbecause weâve got loads of low-quality science to put in the bag. If you walk up and down the poster sessions at a psychology conference, you can see lots of first-year PhD students presenting studies where they seemingly pick some semi-related constructs at random, correlate them, and print out a p-value (âDoes self-efficacy moderate the relationship between social dominance orientation and system-justifying beliefs?â). A bag of words can basically do this already; you just need to give it access to an online participant pool and a big printer.5 But science is a strong-link problem; if we produced a million times more crappy science, weâd be right where we are now. If we want more of the good stuff, what should we put in the bag?âŠ
Send this story to anyone â or drop the embed into a blog post, Substack, Notion page. Every play sends rev-share back to Experimental History.