Artificial intelligence sheds light on how the brain processes language

With the new language models of artificial intelligence, scientists will be able to expand knowledge (Photo: Pixabay)

In recent years, AI language models have gotten very good at certain tasks. In particular, they excel in predict the next word in a text string; This technology helps search engines and texting applications predict the next word you will type.

The newer generation of predictive language models also seems to learn something about the underlying meaning of language. These models can not only predict the next word, but can also perform tasks that seem to require some genuine understanding, such as answering questions, summarizing documents, and completing the story.

These models were designed to optimize the performance of the specific function of predicting text, without trying to mimic anything about how the human brain performs this task or understands language. But a new study Neuroscientists from MIT suggests that the underlying function of these models resembles the function of language processing centers in the human brain. Computer models that work well for other types of language tasks do not show this similarity to the human brain, offering evidence that the human brain can use next-word prediction to drive language processing.

La supercomputadora D-Wave 2X en el Quantum Artificial Intelligence Laboratory (QuAIL) de la NASA Ames Research Center in Mountain View, California, es vista - REUTERS/Stephen Lam/File Photo
La supercomputadora D-Wave 2X en el Quantum Artificial Intelligence Laboratory (QuAIL) de la NASA Ames Research Center in Mountain View, California, es vista – REUTERS/Stephen Lam/File Photo

The better the model is at predicting the next word, the more it fits the human brain. It is surprising that the models fit so well, and it suggests very indirectly that perhaps what the human language system is doing is predicting what will happen next, “explained Nancy Kanwisher, Walter A. Rosenblith Professor of Cognitive Neuroscience, Institute Fellow. McGovern Research Center for Brain and Brain Center at MIT. Minds and Machines (CBMM) and author of the new study.

The new high-performance next-word prediction models belong to a class of models called deep neural networks that they contain computational “nodes” that form connections of different intensity and layers that pass information to each other in prescribed ways. During the last decade, Scientists have used deep neural networks to create vision models that can recognize objects as well as the primate brain does. Research at MIT has also shown that the underlying function of visual object recognition models matches the organization of the primate visual cortex, although those computer models were not specifically designed to mimic the brain.

Joshua Tenenbaum, professor of computational cognitive science at MIT and member of the CBMM and the MIT Artificial Intelligence Laboratory, and Evelina Fedorenko, associate professor of neuroscience Frederick A. and Carole J. Middleton Career Development and member of the McGovern Institute, are the authors. study, which appears this week in the Proceedings of the National Academy of Sciences. Martin Schrimpf, an MIT graduate student working at CBMM, is the first author of the paper.

A new study by neuroscientists at MIT suggests that the underlying function of these models resembles the function of language processing centers in the human brain.
A new study by neuroscientists at MIT suggests that the underlying function of these models resembles the function of language processing centers in the human brain.

In the new study, The MIT team used a similar approach to compare language processing centers in the human brain with models of language processing. The researchers analyzed 43 different language models, including several optimized for next word prediction. These include a model called GPT-3 (Generative Pretrained Transformer 3), which, given a prompt, can generate text similar to what a human would produce. Other models were designed to perform different language tasks, such as filling in the blank in a sentence.

As each model was presented with a series of words, the researchers measured the activity of the nodes that make up the network. They then compared these patterns to activity in the human brain, measured in subjects performing three language tasks: listen to stories, read sentences one at a time, and read sentences in which one word is revealed at a time. These human data sets included functional magnetic resonance imaging (fMRI) data and intracranial electrocorticographic measurements taken in people undergoing brain surgery for epilepsy.

They found that the best performing next word prediction models had activity patterns that closely resembled those seen in the human brain. Activity in those same models was also highly correlated with measures of human behavior, such as how quickly people could read text. “We found that models that predict neural responses well also tend to better predict human behavioral responses, in the form of read times. And then both are explained by the performance of the model in predicting the next word. This triangle really connects everything together, ”said Schrimpf.

The new findings are consistent with previously proposed hypotheses that prediction is one of the key functions in language processing.  (Getty Images)
The new findings are consistent with previously proposed hypotheses that prediction is one of the key functions in language processing. (Getty Images)

“A key conclusion of this work is that the Language processing is a very restricted problem: the best solutions that AI engineers have created end up being similar, as this paper shows, to the solutions found by the evolutionary process that created the human brain. Since the artificial intelligence network did not seek to mimic the brain directly, but ends up looking like a brain, this suggests that, in a sense, there has been a kind of convergent evolution between artificial intelligence and nature“Said Daniel Yamins, a professor of psychology and computer science at Stanford University, who was not involved in the study.

Game changer

One of the key computational features of predictive models like GPT-3 is an element known as a forward unidirectional predictive transformer. This type of transformer is capable of making predictions of what will come next, based on previous sequences. An important feature of this transformer is that it can make predictions based on a very long previous context (hundreds of words), not just the last words. Scientists have not found any brain circuitry or learning mechanism that corresponds to this type of processing, Tenenbaum says. Nevertheless, the new findings are consistent with previously proposed hypotheses that prediction is one of the key functions in language processing.

The researchers now plan to build variants of these language processing models to see how small changes to their architecture affect their performance and their ability to adapt to human neural data. “For me, this result has changed the rules of the game”Fedorenko said. “It is totally transforming my research program, because I would not have predicted that in my lifetime we would come up with these computationally explicit models that capture enough about the brain that we can harness them to understand how the brain works.”

Models that predict neural responses well also tend to better predict human behavioral responses - SeongJoon Cho / Bloomberg
Models that predict neural responses well also tend to better predict human behavioral responses – SeongJoon Cho / Bloomberg

The researchers also plan to try to combine these high-performance language models with some computer models that Tenenbaum’s lab has previously developed that can perform other types of tasks, such as building perceptual representations of the physical world.. “If we can understand what these language models do and how they can connect with models that do things that are more like perceiving and thinking, then that can give us more integrative models of how things work in the brain. This could lead us to better models of artificial intelligence, as well as providing us with better models of how more part of the brain works and how general intelligence arises, than we have had in the past, “concluded Tenenbaum.

Joshua Tenenbaum, professor of computational cognitive science at MIT and member of the CBMM and the MIT Artificial Intelligence Laboratory, and Evelina Fedorenko, associate professor of neuroscience Frederick A. and Carole J. Middleton Career Development and member of the McGovern Institute, are the authors. study, which appears this week in the Proceedings of the National Academy of Sciences. Martin Schrimpf, an MIT graduate student working at CBMM, is the first author of the paper.

KEEP READING:

Language stimulation and a challenging challenge: transforming words by rearranging letters
How stimulated is your brain? 11 compelling visual challenges to keep you going
The riddle that helps maintain and exercise attention: solving this exercise can bring us closer to our goals with greater success



Reference-www.infobae.com

Leave a Reply

Your email address will not be published. Required fields are marked *