Do LLMs Understand? AI Pioneer Yann LeCun Spars with DeepMind’s Adam Brown.

Janna Levin (Professor of Physics and Astronomy) 00:00.940

Um but I have one other question. Okay, no, I have two. I have two in the lightning around. Uh, are we on the precipice of Doomsday or a Renaissance in human creativity,

Yann LeCun (Chief AI Scientist) 00:10.320

Jan? Renaissance.

Janna Levin (Professor of Physics and Astronomy) 00:12.120

Adam.

Adam Brown (Research Scientist) 00:13.680

Most likely Renaissance.

Janna Levin (Professor of Physics and Astronomy) 00:17.600

Um, I have to throw this out. The same question to the audience, but I'm going to phrase it more colorfully, which I think they'll relate to. Will the robot overlords rise up against humanity? Yes, hands up.

Janna Levin (Professor of Physics and Astronomy) 00:30.000

Oh, interesting. Okay, no hands up. Okay, how many robots in the audience hands out? Okay, so okay, so that's interesting. See, that's cool. It was a little more nose maybe, although the light is blinding.

Janna Levin (Professor of Physics and Astronomy) 00:47.360

All right, we're going to come back and ask that again at the end. So here we are, these neural nets have been taught to execute a process we now call deep learning and there's other kinds of learning that take off. And what are the large language models specifically which is

Janna Levin (Professor of Physics and Astronomy) 01:04.520

really what has swept up the news and people's personal experience. We're We're mostly

Janna Levin (Professor of Physics and Astronomy) 01:10.160

relating to large language models. And And what are the large language models, Adam? Maybe you could take that.

Adam Brown (Research Scientist) 01:15.280

Yeah, so a large language model is uh you've probably played with some of them, ChatGPT, Gemini made by my company, various others um made by other companies. It is a special kind of neural network that's trained on particular inputs and particular outputs and trained in a

Adam Brown (Research Scientist) 01:31.960

particular way.

Adam Brown (Research Scientist) 01:33.120

So, it is at At heart, it is mainly the kind of deep neural network that was pioneered by by Yann and by others, but with a particular architecture designed for the following task. Uh it takes text in, so it'll it'll read some uh the first few words of some sentence or the first

Adam Brown (Research Scientist) 01:53.720

few paragraphs of some book, and it will try and predict what the next word is gonna be.

Adam Brown (Research Scientist) 01:59.640

And so you take a deep neural network with a particular architecture and And you have it read basically to first approximation the entire internet. And for every word that comes along on the entire internet, all of the text data and our other kind of data you can find, you then

Adam Brown (Research Scientist) 02:17.960

ask it, what do you think the next word's going to be?

Adam Brown (Research Scientist) 02:20.080

What do you think the next word's going to be? And to the extent that it gets it right, you give it a little bit of reward and strengthen those neural pathways, to the extent that it gets it wrong, you you diminish those neural pathways. And

Adam Brown (Research Scientist) 02:32.920

if you do that, Uh, it'll just start off spewing just completely random words for its prediction. But uh if you train it on a million words, it'll still be spewing random words. If you train it on a a billion words, it'll maybe have just started to learn subject, verb, object,

Adam Brown (Research Scientist) 02:48.960

and various bits of sentence structure. Uh,

Adam Brown (Research Scientist) 02:52.120

and if you train it as we do today on on a trillion words or more tens of trillions of words, uh then it'll start become the conversation partner that you you probably, I hope, uh played around with today. Mhm.

Janna Levin (Professor of Physics and Astronomy) 03:03.840

No No, um, it, it it strikes me as intriguing like it's it's it amuses me sometimes people get really outraged at their chatbot that they're engaged with when it leads them astray or lies to them. And sometimes I've said well it's it's doesn't need to be words, it it might as

Janna Levin (Professor of Physics and Astronomy) 03:22.120

well be colors or symbols, it's just playing a mathematical game and therefore doesn't have a sense of meaning. Now

Janna Levin (Professor of Physics and Astronomy) 03:29.080

I know Adam sort of objected to my summary of that. Do you think that they are extracting meaning um in the same sense that we do when we are engaging in composing sentences.

Yann LeCun (Chief AI Scientist) 03:45.560

Well, they're certainly extracting some meaning um but it's it's a lot more superficial than what most humans uh would extract from from text. Most humans uh intelligence is linked to is is grounded into an underlying reality, right? And language is a way to express phenomena or

Yann LeCun (Chief AI Scientist) 04:09.400

things in that or concepts grounded in that reality. Um,

Yann LeCun (Chief AI Scientist) 04:14.080

LLM's don't have any notion of the underlying reality. And so their understanding is is relatively superficial. Um, they don't really have common sense in the in the way that we understand it. Uh, but if you train them long enough, they they will answer correctly most questions

Yann LeCun (Chief AI Scientist) 04:33.680

that people will think about asking. That's the way

Yann LeCun (Chief AI Scientist) 04:37.080

they're trained. You You You collect all the questions that everybody has ever asked them and then you train them to produce the correct answer for this. Now, there's always going to be new questions or new prompts, new sequences of words for which the system has not really been

Yann LeCun (Chief AI Scientist) 04:53.080

trained and for which it might produce complete nonsense.

Adam Brown (Research Scientist) 04:57.160

Okay? So, in that sense, they don't have the real understanding of the underlying reality or they do have an understanding but it's It's superficial. And so, you know, the next question is, how do we fix that?