Ilya Sutskever – We're moving from the age of scaling to the age of research

Ilya Sutskever (Co-founder and Chief Scientist) 00:00.280

You know it's crazy,

Ilya Sutskever (Co-founder and Chief Scientist) 00:01.800

that all of this is

Dwarkesh Patel (Host) 00:03.000

real.

Ilya Sutskever (Co-founder and Chief Scientist) 00:04.200

Yeah, meaning what?

Dwarkesh Patel (Host) 00:05.120

Don't you think so?

Ilya Sutskever (Co-founder and Chief Scientist) 00:06.120

Meaning what? Like all the AI stuff and all the area, yeah, that it's happened like Isn't it straight out of science

Dwarkesh Patel (Host) 00:13.080

fiction? Yeah. Another thing that's crazy is like how normal this slow take off feels. The idea that we'd be investing 1% of GDP in AI, like I feel like it would felt like a bigger deal, you know? Where right now it just feels like

Ilya Sutskever (Co-founder and Chief Scientist) 00:27.240

We get used to things free fast turns out, yeah. My But also it's kind of like it's abstract, like, what does it mean? What it means that you see it in the news Yeah. that such and such company announced such and such dollar amount. Right. That's that's all you see. Right. It's

Ilya Sutskever (Co-founder and Chief Scientist) 00:43.040

not really felt in any other way so far.

Dwarkesh Patel (Host) 00:45.360

No. Should we actually begin here? I think this is an interesting discussion. Sure.

Dwarkesh Patel (Host) 00:48.760

I think your point about well, from the average person's point of view, nothing is that different. We'll continue being true even into the singularity.

Ilya Sutskever (Co-founder and Chief Scientist) 00:57.280

No, I don't

Dwarkesh Patel (Host) 00:58.000

think so. Okay, interesting.

Ilya Sutskever (Co-founder and Chief Scientist) 01:00.000

So, the thing which I was referring to, not feeling different is okay, so such and such company announced some difficult to comprehend dollar amount of investment. Right. I don't think anyone knows what to do with that. Yeah. But I think that the impact of AI is going to be

Ilya Sutskever (Co-founder and Chief Scientist) 01:20.240

felt. AI is going to be diffused through the economy. There are very strong economic forces for this. And I think the impact is going to be felt very strongly.

Dwarkesh Patel (Host) 01:30.640

When do you expect that impact? I think the models seem smarter than their economic impact would imply.

Ilya Sutskever (Co-founder and Chief Scientist) 01:38.320

Yeah, this is one of the very confusing things about the models right now. How to reconcile the fact that they are doing so well on e-vals. And you look at the e-vals and you go, those are pretty hard e-vals. Right. They're doing so well. But The economic impact seems to be

Ilya Sutskever (Co-founder and Chief Scientist) 02:02.680

dramatically be hot. And it's almost like It's It's very difficult to make sense of how can the model on the one hand do these amazing things and then on the other hand like repeat itself twice in some situation in a kind of a an example would be let's say you use vibe coding to

Ilya Sutskever (Co-founder and Chief Scientist) 02:22.880

do something and you go to some place and then you get a bug. And then you tell the model, can you please fix the bug? Yeah. And the model says, "Oh my god, you're so right. I have a bug. Let me go fix that." And it introduces a second bug. Yeah. And then you tell it you have

Ilya Sutskever (Co-founder and Chief Scientist) 02:37.680

this you have this new second bug. And it tells you, "Oh my god, how could I've done it? You're so right again." And brings back the first bug. Yeah. And you can alternate it between those Yeah. And it's like, how is that possible? Yeah. It's like I'm not sure. But it does

Ilya Sutskever (Co-founder and Chief Scientist) 02:52.280

suggest that the something strange is going on. I have two possible explanations. So here this is the more kind of a whimsical explanation is that maybe our RL training makes the models a little bit too single-minded and narrowly focused, a little bit too I don't know, unaware,

Ilya Sutskever (Co-founder and Chief Scientist) 03:14.080

even though it also makes them aware in some other ways. And because of this, they can't do basic things, but there is another explanation which is back when people were doing pre-training, the question of what data to train on was answered. Because the that answer was

Ilya Sutskever (Co-founder and Chief Scientist) 03:34.400

everything. Yeah. When you do pre-training, you need all the data. So you don't have to think it's going to be this data or that data. Yeah. But when people do RL training, they do need to think. They say okay we want to have this kind of RL training for this thing and that kind

Ilya Sutskever (Co-founder and Chief Scientist) 03:52.600

of RL training for that thing. And from what I hear, all the companies have teams that just produce new RL environment And then the question is, well, what are those? There are so many degrees of freedom. There is such a huge variety of all environments you could produce. And

Ilya Sutskever (Co-founder and Chief Scientist) 04:10.280

one of the one thing you could do, and I think that's something that is done inadvertently is that people take inspiration from the e-vows. You say, "Hey, I would love our model to do really well when we release it. I want the e-vows to look great. What would be RL training that

Ilya Sutskever (Co-founder and Chief Scientist) 04:31.360

would help on this task, right? I think that is something that happens and I think it could explain a lot of what's going on. If you combine this with generalization of the models actually being inadequate that has the potential to explain a lot of what we are seeing, this

Ilya Sutskever (Co-founder and Chief Scientist) 04:47.760

disconnect between eval performance and actual real real world performance, which is something that we don't today exactly even understand what what we mean by that.