EP20: Yann LeCun - part 9/11

Yann LeCun (Chief AI Scientist) 00:00.120

back in the nineteen eighties japan started a huge project called the fifth generation computer project which was like computers with CPU 's that were going to run lists and you know inference engines and stuff right and the hottest job in the late eighties was going to be

Yann LeCun (Chief AI Scientist) 00:17.360

knowledge engineer you were going to sit next to an expert and then the knowledge of the expert into rules and facts right and then the computer would be able to basically do what the expert was this was manual behavior clothing OK uh and it kind of worked but only for a few

Yann LeCun (Chief AI Scientist) 00:33.830

domains where where economically it made sense and it was doable at the level of reliability that was good enough but it was not a pass towards kind of human level intelligence the idea somehow like the delusion that people today have that the current AI mainstream you know

Yann LeCun (Chief AI Scientist) 00:57.190

fashion is going to take us to human intelligence has happened already three times during my career and probably five or six times before right you should you should see what people are saying about the perceptron right the new york times article people were saying oh we're

Yann LeCun (Chief AI Scientist) 01:10.910

going to have like super intelligent machines within ten years marvin

? (?) 01:14.310

minsky in the sixties says oh within ten years the best chess player in the world would be a computer it took a bit longer than that and you know and you know this you know this happened over and over again in nineteen fifty six or something when newell and simon produced the

? (?) 01:32.720

the general problem solver very modestly called the general

Yann LeCun (Chief AI Scientist) 01:37.280

problem solver OK what they what they thought was really cool they say OK the way we think is very simple we pose a problem there is a number of different solutions to that problem different proposals for solution a space of potential solutions like you know you do like

Yann LeCun (Chief AI Scientist) 01:55.030

traveling salesman right there was a number of you know factorial you know N factorial paths possible paths you just have to look for the one that is the best right and they say like every problem can be formulated this way essentially for a search for the best solution if you

Yann LeCun (Chief AI Scientist) 02:12.200

can formulate the problem as

? (?) 02:15.790

an objective by writing a program that checks whether it's a good solution or not or gives a rating to it and then you have a search algorithm that search through the space of possible solution for one that optimizes that score that you saw the AI OK now what they didn't know at

? (?) 02:34.130

the time is order complexity theory that basically every problem that is interesting is exponential or AP complete or whatever right and

Yann LeCun (Chief AI Scientist) 02:42.110

so oh we have to use heuristic programming you know kind of come up with heuristics for every new problem and basically you know there are general problem solver was not that general so like this idea somehow that the latest idea is going to take you to you know AGI or whatever

Yann LeCun (Chief AI Scientist) 02:57.510

you want to call it is very dangerous and a lot of very smart people fell into that trap many times over the last seven decades do you think that the field will ever figure out continual or incremental learning sure yeah that's sort of a technical problem well well i thought i

Yann LeCun (Chief AI Scientist) 03:16.790

thought catastrophic forgetting right because your weights that you trained so much money on get overwritten sure so you train just a little bit of it i mean we don't already do this with SSL right we train a foundation model like for video or something like vijay patu you know

Yann LeCun (Chief AI Scientist) 03:31.910

producers really good representations of video and then if you want to train the system for a particular task you train a small head on top of it and that head can be you know along continuously and even your word model can be trained continuously that's not an issue i don't see

Yann LeCun (Chief AI Scientist) 03:44.550

this as like a big a huge challenge frankly in fact raya heads to LPR simon and i and a few of our colleagues back in two thousand five two thousand six build a learning based navigation system for mobile robots that had this kind of idea so it was it was commercial net that was

Yann LeCun (Chief AI Scientist) 04:02.230

doing semantic segmentation from camera images and on the fly the top layers of that network would be adapted to the current environment so you do a good job and the labels came from short wrench uh traversability that were indicated by stereo vision essentially so yeah i mean

Yann LeCun (Chief AI Scientist) 04:27.670

you can do this it's particularly if you have multimodal yeah i don't see this as a big challenge it's been a pleasure to have you real pleasure to thank you so much thank you thank you