Mind reading is common among us humans. Not in the way psychologists claim, by getting a warm stream of consciousness that fills everyone’s experience, or by randomly pulling a thought out of your head, in the way psychologists claim. Everyday mind reading is more subtle: We watch people’s facial expressions and movements, listen to their words, and judge or intuit what might be going on in their minds.
In psychologists, this intuitive psychology—the ability to attribute to others mental states different from our own—is called theory of mind, and its absence or impairment is related to autism, schizophrenia and others developmental disabilities. Theory of mind helps us communicate and understand each other; it enables us to enjoy literature and movies, play games and understand our social environment. Competence is, in many ways, an essential part of being human.
What if machines could read minds?
Recently, Michal Kosinski, a psychologist at the Stanford Graduate School of Business, made that argument: Large-scale language models like OpenAI’s ChatGPT and GPT-4 — next-word prediction machines trained on vast amounts of text from the Internet — have developed a theory of mind.His studies have not yet been peer-reviewed, but they have sparked scrutiny and conversation among cognitive scientists who have recently been trying to answer the oft-asked question – can ChatGPT do it? this? – and transfer it to stronger areas of scientific inquiry. What are the powers of these models, and how might they change our understanding of our own minds?
“Psychologists won’t accept any assertions about the abilities of young children based solely on anecdotes of your interactions with them, which seems to be what’s happening with ChatGPT,” says Alison Gopnik, a psychologist at the University of California, Berkeley. The first batch in the 1980s Researchers who study theory of mind. “You have to do very careful and rigorous testing.”
Previous research by Dr. Kosinski has shown that neural networks trained to analyze facial features such as nose shape, head angle and emotional expression can predict people’s behavior. Political Views and sexual orientation with amazing accuracy (about 72% in the first case and about 80% in the second case).His recent work on large-scale language models used classical psychometric theory to measure children’s ability to attribute wrong belief to others.
A new generation of chatbots
A brave new world. A new crop of chatbots powered by artificial intelligence has sparked a debate over whether the technology can upend the internet’s economy, rendering today’s powerhouses obsolete and creating the industry’s next giants. Here’s what to know about bots:
A famous example is sally anne test, one of the girls, Anne, moves a marble from a basket to a box when the other girl, Sally, is not looking. The researchers claim that to know where Sally would look for the marbles, the audience would have to use theory of mind, reasoning about Sally’s perceptual evidence and belief formation: Sally didn’t see Annie move the marbles to the box, so she still Believe it is where she left it for the last time, in the basket.
Dr. Kosinski presents 10 large language models with 40 unique variants of these psychometric test theories – describing situations like the Sally-Anne test, where a person (Sally) forms an error belief. He then asked the models questions about these situations, prompting them to see whether they could attribute false beliefs to the characters involved and accurately predict their behavior. He found that GPT-3.5, released in November 2022, does so 90 percent of the time, and GPT-4, released in March 2023, does so 95 percent of the time.
in conclusion? Machines have a theory of mind.
But shortly after these results were published, Harvard psychologist Tomer Ullmann responded a set of own experiments, showing that even for the most complex large-scale language models, small tweaks in hints can completely change the generated answers. If a container is described as transparent, a machine will not be able to infer that someone can see it. In these cases, machines struggled to take into account people’s testimony, sometimes failing to distinguish between objects inside the container and on top of it.
Carnegie Mellon University computer scientist Maarten Sap, Conducted more than 1,000 psychological tests Entering large language models, it was found that state-of-the-art transformers such as ChatGPT and GPT-4 only pass about 70% of the time. (In other words, they were 70 percent successful in attributing false beliefs to the people described in the test situation.) The discrepancy between his data and Dr. Kosinski’s could be down to differences in the test, but Dr. 95% of the time it won’t be evidence for a real theory of mind. Machines often fail in a patterned way, unable to reason abstractly and often making “spurious associations,” he said.
Dr. Ullman noted that machine learning researchers have spent the past few decades trying to capture the flexibility of human knowledge in computer models. This difficulty, he says, has been a “shadow discovery” that lurks behind every exciting innovation.Researchers have shown that language models often give wrong or irrelevant answers if unnecessary information is entered before asking a question; some chatbots are thrown off by hypothetical discussions about talking birds so much that they end up claims birds can talk. Because their reasoning is sensitive to small changes in input, scientists refer to these machines’ knowledge as “crisp“
Dr. Gopnik compared the theory of mind of large language models to her own understanding of general relativity. “I’ve read enough to know what those words are,” she said. “But if you asked me to make new predictions or say what Einstein’s theory tells us about new phenomena, I would be very confused because I don’t really have a theory in my head.” Human theory of mind, by contrast, is linked to other mechanisms of commonsense reasoning; it stands up to scrutiny.
Overall, Dr. Kosinski’s work and responses to it fit into the debate about whether the capabilities of these machines can be compared to those of humans—a debate Split Researchers working on natural language processing. Are these machines random parrots, alien intelligence, or fraudsters? A 2022 Survey Researchers in the field found that of the 480 researchers who responded, 51 percent believed that large language models could eventually “understand natural language in some non-trivial sense,” while 49 percent believed they could not. .
Dr. Ullmann does not discount the possibility of machine understanding or a machine theory of mind, but he is wary of attributing human abilities to non-human things.He noticed a famous 1944 study Produced by Fritz Heider and Marianne Simmel, participants are shown an animated film of two triangles and a circle interacting. When the subjects were asked to write about what happened in the movie, nearly all described the shapes as people.
“Lover in the second dimension, no doubt; the second little triangle and the sweet circle,” wrote one participant. “Triangle No. 1 (hereinafter referred to as the villain) spies on the young lover. Ah!”
Explaining human behavior by talking about beliefs, desires, intentions, and thoughts is natural, and often socially required. This tendency is central to who we are—so essential that we sometimes try to read the minds of things that don’t have minds, at least not like our own.
