Ilya Sutskever – We're moving from the age of scaling to the age of research

Ilya Sutskever (Co-founder and Chief Scientist) 00:00.000

I think I think there is so I think there's a more general point. I think it's actually really mysterious how the brain encodes high level desires. Sorry, how evolution encodes high level desires. Like it's pretty easy to understand how evolution would would endow us with the

Ilya Sutskever (Co-founder and Chief Scientist) 00:20.000

desire for food that smells good.

Ilya Sutskever (Co-founder and Chief Scientist) 00:22.360

Because smell is a chemical and so just pursue that chemical. It's very easy to imagine such a evolution doing such a thing. But evolution also has has endowed us with all these social desires. Like we we really care about being seen positively by society. if we care about being

Ilya Sutskever (Co-founder and Chief Scientist) 00:41.800

in a good standing, we like all these social intuitions that we have, I feel strongly that they're baked in and I don't know how evolution did it because it's a high level concept that's represented in the brain. Like what people think like let's say you are like you care about

Ilya Sutskever (Co-founder and Chief Scientist) 01:03.240

some social thing. It's not like a low level signal like smell. It's not something that for which there's a sensor. Like the brain needs to do a lot of processing to piece together lots of bits of information to understand what's going on socially and somehow evolution said

Ilya Sutskever (Co-founder and Chief Scientist) 01:21.080

that's what you should care about.

Dwarkesh Patel (Host) 01:22.040

Yes.

Ilya Sutskever (Co-founder and Chief Scientist) 01:22.840

How did it do it. And he did it quickly too.

Dwarkesh Patel (Host) 01:25.180

Yeah.

Ilya Sutskever (Co-founder and Chief Scientist) 01:25.620

Because I think all these sophisticated social things that um we care about, I think they evolved pretty recently.

Dwarkesh Patel (Host) 01:32.180

Yeah.

Ilya Sutskever (Co-founder and Chief Scientist) 01:32.580

So evolution had an easy time hardcore in this high level desire. And I maintain or, you know, at least I'll say, I'm unaware of good hypothesis for how it's done. I I had some ideas I was kicking around, but none of them none of them uh are satisfying.

Dwarkesh Patel (Host) 01:52.020

Yeah. And what's especially impressive if it was a desire that you learned in your lifetime, it kind of makes sense cuz your brain is intelligent, it makes sense why you'd be able to learn

Dwarkesh Patel (Host) 02:01.660

intelligent desires. But your point is that the desire is maybe this is not your point, but one way to understand it is the desire is built into the genome and the genome is not intelligent, right? But it's able to you're somehow able to describe this feature that requires like

Dwarkesh Patel (Host) 02:16.540

it's not even clear how you define that feature and you can get it into you can build it into the genes.

Ilya Sutskever (Co-founder and Chief Scientist) 02:21.900

Yeah, essentially. Or maybe I'll put it differently. If you think about the tools that are available to the genome, it says, "Okay, here's a recipe for building a brain." And you could say, "Here is a recipe for connecting the dopamine neurons to like the smell sensor."

Dwarkesh Patel (Host) 02:36.420

Yeah.

Ilya Sutskever (Co-founder and Chief Scientist) 02:37.340

And if the smell is a certain kind of, you know, good smell, you want to eat that. I could imagine the genome doing that. I'm I'm claiming that it is harder to imagine. It's harder to imagine the genome saying, "You should care about some complicated computation that your

Ilya Sutskever (Co-founder and Chief Scientist) 02:54.340

entire brain that like a big chunk of your brain does. That's all I'm claiming. I I can tell you like a speculation I was wondering how it could be done and let me offer a speculation and I'll explain why the speculation is probably false. So the speculation is okay. So the

Ilya Sutskever (Co-founder and Chief Scientist) 03:10.140

brain it's like the brain has those regions you know the brain regions we have our cortex right it has all those brain regions and the cortex is uniform but the brain regions And and and the neurons in the cortex, they kind of speak to their neighbors mostly. And that explains

Ilya Sutskever (Co-founder and Chief Scientist) 03:27.940

why you get brain regions. Because if you want to do some kind of speech processing, all the neurons that do speech need to talk to each other. And they can and because neurons can only speak to their nearby neighbors, for the most part, it has to be a region. All the regions

Ilya Sutskever (Co-founder and Chief Scientist) 03:40.420

are mostly located in the same place from person to person. So maybe evolution hardcoded literally a location on the brain. So it says, "Oh, like when when like, you know, the GP GPS of the brain, GPS coordinates, such and such. When that fires, that's what you should care

Ilya Sutskever (Co-founder and Chief Scientist) 03:56.860

about. Like maybe that's what evolution did, because that would be within the toolkit of evolution.

Dwarkesh Patel (Host) 04:01.420

Yeah, although there are examples where, for example, people who are born blind have that area of their cortex adopted by another sense. And I have no idea, but I'd be surprised if

Dwarkesh Patel (Host) 04:16.900

the desires or the reward functions which require visual signal no longer worked, you know, people who have their different areas of their cortex co-opted. For example, if you no longer have vision, can you still feel the sense that I want people around me to like me and so

Dwarkesh Patel (Host) 04:35.060

forth, which usually there's also visual cues for.

Ilya Sutskever (Co-founder and Chief Scientist) 04:38.020

So, I actually fully agree with that. I I think there's an even stronger counterargument to this theory which is like if you think about people, so there are people who get half of their brains removed in uh childhood.

Dwarkesh Patel (Host) 04:50.700

Yeah.

Ilya Sutskever (Co-founder and Chief Scientist) 04:51.580

And they still have all their brain regions but they all somehow move to just one hemisphere, which suggests that the brain regions the their location is not fixed and so that theory is not true. It would have been cool if it was true, but it's not. And so I think that's a

Ilya Sutskever (Co-founder and Chief Scientist) 05:05.580

mystery, but it's an interesting mystery. Like the fact is somehow evolution was able to endow us to care about social stuff very, very reliably. And even people who have like all kinds of strange mental conditions and deficiencies and emotional problems tend to care about this

Ilya Sutskever (Co-founder and Chief Scientist) 05:21.660

also.