Ilya Sutskever – We're moving from the age of scaling to the age of research

Ilya Sutskever (Co-founder and Chief Scientist) 00:00.040

So, the it, we don't know how to build. And I think that a lot a lot hinges on understanding reliable generalization. And I'll say another thing, which is like, you know, one of the things that you could say is what what that cause alignment to be difficult is that human value

Ilya Sutskever (Co-founder and Chief Scientist) 00:20.080

that it's it's um your ability to learn human values is fragile, then your ability to optimize them is fragile, you will you actually learn to optimize them. And then can't you say, "Are these not all instances of unreliable generalization?" Why is it that human beings appear to

Ilya Sutskever (Co-founder and Chief Scientist) 00:38.320

generalize so much better? What if generalization was much better? What would happen in this case? What would be the effect? But those we can't we we can't like those questions are right now still unanswerable.

Dwarkesh Patel (Host) 00:48.480

Um, how does one think about what AI going well looks like? Because I think you've scoped out how AI might evolve, we'll have these sort of continual learning agents. AI will be very powerful. Maybe there will be many different AIs. How do you think about lots of continent

Dwarkesh Patel (Host) 01:06.280

compute size intelligences going around? How dangerous is that? How do we make that less dangerous. And how do we do that in a way that protects a equilibrium where there might be misaligned AI's out there and bad actors out there. So

Ilya Sutskever (Co-founder and Chief Scientist) 01:26.380

one reason why I like the AI that cares for sentient life, you know, and we can debate on whether it's good or bad. But if the first end of these dramatic systems actually do care for, you know, Love humanity or something, you know, care for sentient life. Obviously, this also

Ilya Sutskever (Co-founder and Chief Scientist) 01:48.260

needs to be achieved. This needs to be achieved. So, if this is achieved by the first

Ilya Sutskever (Co-founder and Chief Scientist) 01:54.340

N of those systems, then there then I can see it go well. At least for quite some time. And then there is the question of what happens in the long run. What happens in the long run? How do you achieve a long run equilibrium? And I think that there There is an answer as well. And

Ilya Sutskever (Co-founder and Chief Scientist) 02:14.980

I don't like this answer. But it needs to be considered. In the long run, you might say okay, so if you have a world where powerful eyes exist. In the short term, you could say okay, you have universal high income. You have universal high income. And we all doing well. But we

Ilya Sutskever (Co-founder and Chief Scientist) 02:35.060

know that what do the Buddhist say? Change is the only constant. And so things change. And there is some kind of government political structure thing, and it changes. Because these things have a shelf life. You know, some new new government thing comes up and it functions and

Ilya Sutskever (Co-founder and Chief Scientist) 02:51.260

then after some time, it stops functioning. That's something that you see happening all the time. And so I think that for the long run equilibrium, one approach, you could say, "Okay, so maybe every person will have an AI that will do their bidding." And that's good. And if that

Ilya Sutskever (Co-founder and Chief Scientist) 03:10.660

could be maintained indefinitely, that That's true. But the downside with that is okay, so then the AI goes and like earns earn you know earns money for for the person and you know advocates for their needs in like the political sphere and maybe then writes a little report

Ilya Sutskever (Co-founder and Chief Scientist) 03:27.140

saying okay here's what I've done here's the situation and the person says great keep it up. But the person is no longer a participant. And then you can say that's a precarious place to be in. But so I'm going to preface by saying I don't like this solution, but it is a

Ilya Sutskever (Co-founder and Chief Scientist) 03:46.460

solution. And the solution is if people become part AI with some kind of neuralink plus plus. Because what will happen as a result is that now the AI understands something and we understand it too. Like because now the understanding is transmitted wholesale. So now if the AI is

Ilya Sutskever (Co-founder and Chief Scientist) 04:04.980

in some situation, now it's like you are involved in that situation yourself fully. And I think this is the answer to the equilibrium. I

Dwarkesh Patel (Host) 04:14.860

wonder if uh the fact that emotions which were developed millions or in many cases billions of years ago in a totally different environment are still guiding our actions so strongly is an example of alignment success. To maybe spell out what I mean, the brain stem has these I

Dwarkesh Patel (Host) 04:40.300

don't know if it's more accurate to call it a value function or reward function. but the brain stem has a directive of it saying mate with somebody who's more successful. The cortex is the part that understands what does success mean in the modern context. But the brain stem is

Dwarkesh Patel (Host) 04:53.140

able to align the cortex and say however you recognize success to be and I I'm not smarter than to understand what that is. You're still going to pursue this directive.