Ilya Sutskever – We're moving from the age of scaling to the age of research

Dwarkesh Patel (Host) 00:00.840

There's been public estimates that you know companies like OpenAI spend on the order of 5 6 billion dollars a year even just so so far on

Dwarkesh Patel (Host) 00:09.000

experiments. This is separate from the amount of money they're sending on inference and so forth. So, seems like they're spending more a year running like research experiments than you guys have in total

Ilya Sutskever (Co-founder and Chief Scientist) 00:20.040

funding. I think it's a question of what you do with it. It's a question of what you do with it. Like they have a like is the more I think in in in their case and the case of others, I think there is a lot more demand on the training compute. There's a lot more different work

Ilya Sutskever (Co-founder and Chief Scientist) 00:34.280

streams. There is there are different modalities there is just more stuff. And so it becomes fragmented.

Dwarkesh Patel (Host) 00:42.440

How will SSI

Ilya Sutskever (Co-founder and Chief Scientist) 00:43.080

make money? You know, my answer to this question is something like we just right now we just focus on the research and then the answer to this question will reveal itself. I think there will be lots of possible answers.

Dwarkesh Patel (Host) 00:58.480

Hm. Is SSI's plan still to straight shot

Ilya Sutskever (Co-founder and Chief Scientist) 01:01.200

super intelligence? Maybe. I think that there is merit to it. I think there's a lot of merit because I think that it's very nice to not be affected by the day-to-day market competition. But I think there are two reasons that may cause us to change the plan. One is pragmatic if

Ilya Sutskever (Co-founder and Chief Scientist) 01:24.760

timelines turn out to be long, which they might. And second, I think there is a lot of value in the best and most powerful AI being out there impacting the world. Yeah. I think this is a meaningfully

Dwarkesh Patel (Host) 01:43.000

valuable thing. But then so why is your default plan to straight out super intelligence? Because it sounds like, you know, open AI and therapeutic all these other companies, their explicit thinking is look we have weaker and weaker intelligences that the public can get used to

Dwarkesh Patel (Host) 01:56.920

and prepare for and why is it potentially better to build the super intelligence

Ilya Sutskever (Co-founder and Chief Scientist) 02:03.080

directly? So I'll make the case for and against. Yeah. The case for is that you are So one of the challenges that people face when they're in the market is that they have to participate in the rat race. And the rat race is quite difficult in that it exposes you to to to

Ilya Sutskever (Co-founder and Chief Scientist) 02:22.240

difficult trade-offs which you need to make. And there is it is it is nice to say we'll insulate ourselves from all this and just focus on the research and come out only when we are ready and not before. But the counterpoint is valid too. And those are those are opposing forces.

Ilya Sutskever (Co-founder and Chief Scientist) 02:41.520

The counterpoint is hey, it is useful for the world to see powerful AI. It is useful for the world to see powerful AI because that's the only way

Dwarkesh Patel (Host) 02:52.040

you can communicate it. Well, I guess not even just that you can communicate

Ilya Sutskever (Co-founder and Chief Scientist) 02:55.160

the idea, but Communicate the AI. Not the idea. Communicate the AI.

Dwarkesh Patel (Host) 03:00.760

What do you

Ilya Sutskever (Co-founder and Chief Scientist) 03:01.000

mean communicate the AI? So okay, so let's suppose you read an essay about AI. And the essay says AI is going to be this and AI is going to gonna be that and it's gonna be this. And you read it and you say, "Okay, this is

Dwarkesh Patel (Host) 03:12.040

an interesting essay."

Ilya Sutskever (Co-founder and Chief Scientist) 03:13.040

Right. Now suppose you see an AI doing this and AI doing that, it is incomparable. Like basically, I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight

Dwarkesh Patel (Host) 03:33.040

shot. Yeah. Well, I guess it's not even that which I but I do think that is an important part of it. The other big thing is, I can't think of another discipline in human engineering and research where the end artifact was made safer mostly through just thinking about how to make

Dwarkesh Patel (Host) 03:51.600

it safe as opposed to why are airplane crashes per mile so much lower today than they were decades ago? Why is it so much harder to find a bug in Linux than it would have been decades ago? And I think it's mostly because these systems were deployed to the world. You noticed

Dwarkesh Patel (Host) 04:07.120

failures Those failures were corrected and the systems became more robust. Now I'm not sure why AGI and superhuman intelligence would be any different, especially given and I hope we can we're going to get to this. It seems like the the harms of super intelligence are not just

Dwarkesh Patel (Host) 04:24.560

about like having some malevolent uh paper clipper out there, but it's just like this is a really powerful thing and we don't even know how to conceptualize how people will interact with it, what people will do with it. And having gradual access to it seems like a um better way

Dwarkesh Patel (Host) 04:39.240

to maybe spread out the impact of it and

Ilya Sutskever (Co-founder and Chief Scientist) 04:42.120

to help people prepare for it. Well, I think I think on this point, even in the straight shot scenario, you would still do a gradual release of it. It's how I would imagine it. The The gradualism would be an inherent inherent component of any plan. It's just a question of what

Ilya Sutskever (Co-founder and Chief Scientist) 05:01.940

is the first thing that you get out of the door. That's number one. Number two, I also think, you know, I believe you have advocated for continual learning more than other people. And I actually think that this is an important and correct thing, and here is why. So, one of the

Ilya Sutskever (Co-founder and Chief Scientist) 05:20.540

things. So, I'll give you another example of how thinking how language affects thinking. And in this case, this will be two words, two words that have shaped everyone's thinking I maintain. First word, AGI. Second word, pre-training. Let me explain. So, the word the term AGI,

Ilya Sutskever (Co-founder and Chief Scientist) 05:45.700

why does this term exist? It's a very particular term. Why does it exist? There's a reason. The reason that the term AGI exists is in my opinion not so much because it's like a very important essential descriptor of of some end state of intelligence, but because it is a reaction

Ilya Sutskever (Co-founder and Chief Scientist) 06:10.940

to a different term that existed and the term is narrow AI. If you go back to ancient history of game playing AI, of checkers AI, check AI, computer games AI. Everyone would say, "Look at this narrow intelligence." Sure the chess AI can beat Kasparov, but it can't do anything

Ilya Sutskever (Co-founder and Chief Scientist) 06:28.380

else. It is so narrow, artificial narrow intelligence. So in response as a reaction to this, some people said, "Well, this is not good. It is so narrow. What we need is general AI." General AI, an AI that can just do all the things. The second And and that term just got a lot of

Ilya Sutskever (Co-founder and Chief Scientist) 06:52.900

traction. Yeah. The second thing that got a lot of traction is pre-training. Specifically, the recipe of pre-training. I think the current the way people do RL now is maybe um un is undoing the conceptual imprint of pre-training. But pre-training had the property. You do more

Ilya Sutskever (Co-founder and Chief Scientist) 07:13.460

pre-training and the model gets better at everything more or less uniformly. Yeah. General AI. pre-training gives AGI. But the thing that happened with AGI and pre-training is that in some sense they overshot the target. Because by the kind if you think about the term AGI, you

Ilya Sutskever (Co-founder and Chief Scientist) 07:38.260

will realize and especially in the context of pre-training, you will realize that a human being is not an AGI. Because a human being, yes, there is definitely a foundation of skills. A human being A human being lacks a huge amount of knowledge. Instead, we rely on continual

Ilya Sutskever (Co-founder and Chief Scientist) 07:59.860

learning. We rely on continual learning. And so then when you think about, okay, so let's suppose that we achieve success and we produce a safe super some kind of safe super intelligence. The question is, but how do you define it? Where on the curve of continual learning is it

Ilya Sutskever (Co-founder and Chief Scientist) 08:14.780

going to be? I will produce like um a super intelligent 15-year-old that's very eager to go you say, "Okay, I'm going to" They don't know very much at all. The great student, very eager. You go and be a programmer. You go and be a doctor. Go and learn. So you could imagine that

Ilya Sutskever (Co-founder and Chief Scientist) 08:31.980

the deployment itself will involve some kind of a learning trial and error period. It's a process as opposed to you drop