EP20: Yann LeCun - part 5/15

Ravid Shwartz-Ziv (Assistant Professor) 00:00.430

mini hack they had to dumb it down just for for AI so some of my you know colleagues have been working with is actually one of my master students so and and you know michael enough who i mentioned earlier has been also doing some work there now what's interesting there is that

Ravid Shwartz-Ziv (Assistant Professor) 00:19.400

there is a type of situations like this where you need to plan OK but you need to plan in the process of uncertainty the problem with you know all games and adventure games in particular is that you don't have complete visibility of the state of the system you don't have the map

Ravid Shwartz-Ziv (Assistant Professor) 00:31.910

in advance you need to explore and blah blah blah you can get killed every time you do this and you know but the actions are essentially discreet yes correct possible actions is turn based and so in that sense it's like chess acceptance is not you know chess is fully observable

Ravid Shwartz-Ziv (Assistant Professor) 00:50.070

go also it's really observable stratego isn't that stratego isn't you know poker is not and so it makes it more difficult if you have uncertainty of course but those are games where the number of actions you can take is discrete and basically you know what you need to do is do

Ravid Shwartz-Ziv (Assistant Professor) 01:15.070

tree exploration OK and to do of course the tree of possible states you know goes exponentially with the number of moves and so you have to have some way of generating only the moves that are likely to be good and basically never generate the other ones or select them down and

Ravid Shwartz-Ziv (Assistant Professor) 01:33.550

you need to have a value function which is something that tells you OK i can't plan to the end of the game but even though i'm planning sort of nine moves ahead i have some way of estimating whether evaluating whether a position is good or bad it's going to lead me to you know

Ravid Shwartz-Ziv (Assistant Professor) 01:48.190

victory or solution right so you need those two components basically something that guesses what the good moves are and then something that you know essentially evaluates ends and if you have those both of those things you can train those functions using something like

Ravid Shwartz-Ziv (Assistant Professor) 02:04.350

reinforcement learning or behavioral cloning if you have data i mean the basic idea for this goes back to samuel 's checker players from nineteen sixty four it's not recent but but of course was you know the power

Ravid Shwartz-Ziv (Assistant Professor) 02:19.030

of it was demonstrated with you know alphago and

Yann LeCun (Chief AI Scientist) 02:22.390

alpha zero and things like that so that's good but that's a domain where humans suck humans are terrible at playing chess right they're playing go like machines are much better than we are because of the speed of tree exploration and because of the memory that's required for for

Yann LeCun (Chief AI Scientist) 02:39.670

tree exploration we just don't have enough memory capacity to do breadth first tree exploration so we suck at it like you know when alphago came out you know people before that thought that the best human players were maybe two or three stones handicapped like below an ideal

Yann LeCun (Chief AI Scientist) 02:56.390

player that they call god no like you know humans are terrible like we you know the best players in the world need like eight or nine stories well i i can't believe i i get the pleasure to talk about game AI with with yan i just have a few follow up questions on this the first

Yann LeCun (Chief AI Scientist) 03:14.390

one is this example that you talk about around humans being terrible at chess and i'm familiar a bit with the development of chess AI over the years you know i've heard this referred to as more of X paradox and explained as you know humans have evolved over billions or millions

Yann LeCun (Chief AI Scientist) 03:32.480

sorry it's large N number of years to physical locomotion and that's why babies and humans are very good at this but we have not evolved at all to play chess so that's one question and then a second question that's related is

Ravid Shwartz-Ziv (Assistant Professor) 03:46.350

a lot of people today who play video games and i'm one of them have observed that it feels like AI at least in terms of like enemy AI has not improved really in twenty years right that some of the best examples are still like halo one and fear from the early two thousands so

Ravid Shwartz-Ziv (Assistant Professor) 04:02.470

when do you think that you know advancements that we've been doing in the lab are going to actually have real impact on like gamers you know and and in a non like generative AI sense yeah i used to be a giver never

Yann LeCun (Chief AI Scientist) 04:15.600

addictive one but but my family is in it because i have three sons in their thirties and they have a video game design studio between them so so i was sort of you know embedded in that culture but yeah no you're right and you know it's it's also it's also true that the you know

Yann LeCun (Chief AI Scientist) 04:36.880

despite the accuracy of physical simulators a lot of the a lot of those simulations are not used by studios who make animated movies because they want control they don't necessarily want accuracy they want control and in games it's only the same thing it's a creative act and

Yann LeCun (Chief AI Scientist) 04:53.390

what you want is some control about the course of the story or the way the you know NPC kind of behave and all that stuff right and the key AI kind of you know is difficult to maintain control at the moment so i mean it will come but you know there's there's some resistance from

Yann LeCun (Chief AI Scientist) 05:13.040

the creators but i think OK marvik paradox is very much still in force so moravec i think formulated it in nineteen eighty eight if i don't know correctly and he said like yeah you know things that we think of as uniquely human intellectual tasks like PHS we can do with

Yann LeCun (Chief AI Scientist) 05:36.190

computers or you know computing integrals or whatever but the thing that we take for granted we don't even think is an intelligent task like what a cat can do we still can do with robots and even now forty seven years later we still can't do them well i mean of course we can you

Yann LeCun (Chief AI Scientist) 06:00.030

know train robots

Ravid Shwartz-Ziv (Assistant Professor) 06:01.110

you know by imitation and a bit of reinforcement learning and you know by training through simulation to kind of locomote and you know what obstacles and do various things but then not nearly as inventive and creative and you know agile

Yann LeCun (Chief AI Scientist) 06:19.230

as a cat it's not because we can't build a robot we certainly can it's just we can't we can't make them smart enough to do all the stuff that a cat or unit mouse can do let alone the dog or a monkey right so so you have all those people bloviating about like you know AGI in in a

Yann LeCun (Chief AI Scientist) 06:37.670

year or two