EP20: Yann LeCun

Ravid Shwartz-Ziv (Assistant Professor) 00:13.690

hi anne and welcome to the information bottleneck and i have to say this is a bit weird for me like i've known you for almost five years and we've worked closely together but this is the first time that i'm interviewing you for a podcast right usually our conversations are more

Ravid Shwartz-Ziv (Assistant Professor) 00:33.610

like yan it doesn't work what they should do OK so even though i'm sure all of our audience knows you i will say yana kun is a turing award winner one of the godfathers of deep learning the inventors of convolutional neural networks founder of metals fundamental AI research lab

Ravid Shwartz-Ziv (Assistant Professor) 00:59.790

and still their chief AI scientist and the professor at NYU so welcome

Yann LeCun (Chief AI Scientist) 01:08.520

pleasure to be here yeah

Ravid Shwartz-Ziv (Assistant Professor) 01:11.680

and it's a pleasure for me to be anywhere near you i have been you know in this industry for a lot less time than either one of you doing research for a lot less time so the fact that i'm able to publish papers somewhat regularly with ravid has been an honor and to be able to

Ravid Shwartz-Ziv (Assistant Professor) 01:33.270

start hosting this podcast has been even more of one so it's really a pleasure to sit down with you

Yann LeCun (Chief AI Scientist) 01:39.960

yeah

Ravid Shwartz-Ziv (Assistant Professor) 01:40.280

so we'll try like congratulations on the new startup you recently announced that after twelve years at meta you're starting a new startup advanced machine intelligence that you focus on world model and so first of all how does it feel to be in the in the other side going from a

Ravid Shwartz-Ziv (Assistant Professor) 02:02.630

big company to starting something from

Yann LeCun (Chief AI Scientist) 02:06.070

scratch well i co founded companies before i was you know involved more peripherally than than this new one but you know i know i know how this works what's unique about this one is a new phenomenon where there is enough hope from the part of investors that you know AI will have

Yann LeCun (Chief AI Scientist) 02:28.990

a big impact that they are ready to invest a lot of money essentially which means now you can create a startup where you know the first couple of years are essentially focused on research that just was not possible before like you know the only place to do research in industry

Yann LeCun (Chief AI Scientist) 02:45.830

before was in a large company that was you know not fighting for its survival and basically had a dominant position market and had a you know long enough view that they were willing to to fund long term projects so from you know history the the big labs that we remember like

Yann LeCun (Chief AI Scientist) 03:09.190

bell labs belong to AT and T which basically had a monopoly yield telecommunication in the US you know IBM had a monopoly on big computers essentially right and they had a good research lab xerox as a monopoly on photocopiers and that enabled them to park did not enable them to

Yann LeCun (Chief AI Scientist) 03:26.430

profit from the research going on there but that profited apple microsoft research google research and fair at meta and the industry is shifting again fair had a big influence on AI the AI research ecosystem by essentially being very open right publishing everything open

Yann LeCun (Chief AI Scientist) 03:54.190

sourcing everything with and with tools like pythorch but also like research prototypes that a lot of people have been using in industry so we caused other labs like google to become more open and other labs to also kind of publish much more systematically than before but what's

Yann LeCun (Chief AI Scientist) 04:12.030

been happening over the last couple of years is that rosso 's labs have been kind of climbing up and becoming more secretive and that's certainly the case i mean that was the case for a penny i several years ago and and now google is becoming more closed and possibly even meta

Yann LeCun (Chief AI Scientist) 04:35.200

so yeah i mean it was it was time for the type of stuff that i'm interested in to kind of do it outside so

Ravid Shwartz-Ziv (Assistant Professor) 04:46.510

to be clear then does ami advanced machine intelligence plan to do their research in the open

Yann LeCun (Chief AI Scientist) 04:56.080

yeah upstream research i mean in my opinion you cannot really call it research unless you publish what you do because otherwise you can get easily fooled by yourself you know you you come up with something you think it's the best thing since sliced bread OK if you don't actually

Yann LeCun (Chief AI Scientist) 05:14.510

submit it to the rest of the community you might just be delusional and i've seen that phenomenon many times you know in lots of industry research lab where there's sort of internal hype about you know some internal projects without kind of realizing that other people are doing

Yann LeCun (Chief AI Scientist) 05:33.360

things that actually are better right so if you if you tell the scientists like you know publish your your work first of all that is an incentive for them to do better work that is more you know whether methodology is kind of more thorough and the results are kind of more

Yann LeCun (Chief AI Scientist) 05:51.710

reliable the research is more reliable it's good for them because very often when you work on a research project the impact you may have on product could be months years or decades down the line and you cannot tell people like you know come work for us don't say what you're

Yann LeCun (Chief AI Scientist) 06:11.070

working on and maybe there is a product you will have an impact on five years from now like in the in the meantime like they can't be motivated to really do something useful so if you tell them that they they tend to work on things that have a short term impact right so if you

Yann LeCun (Chief AI Scientist) 06:26.430

really want breakthroughs you need to let people publish you can't do it any other way and this is something that a lot of the industry is forgetting at the moment

Ravid Shwartz-Ziv (Assistant Professor) 06:35.310

does AMI like what products if any does AMI plan to to produce or make is it research or more than that

Yann LeCun (Chief AI Scientist) 06:44.350

no it's more than that it's actual products OK but you know what things i have to do with with you know what models and you know planning and and basically we have the ambition of becoming kind of one of the main suppliers of intelligent systems down the line we think the the

Yann LeCun (Chief AI Scientist) 07:03.990

current architectures that are employed you know at adams or you know agent systems that are based on NLM's umm work OK for language even agenting systems really don't work very well they require a lot of data to basically clone the behavior of humans and they're not that

Yann LeCun (Chief AI Scientist) 07:21.750

reliable so we think the proper way to handle this and i've been saying this for almost ten years now is have role models that are capable of predicting what would be the consequence or the consequences of an action or a sequence of actions that an AI system might take and then

Yann LeCun (Chief AI Scientist) 07:42.510

the system arrives at a sequence of actions or an output by optimization by figuring out what sequence of actions will optimally accomplish a test you know setting for myself that's planning OK so i think in the central part of intelligence is being able to predict the

Yann LeCun (Chief AI Scientist) 08:02.040

consequences of your actions and then use them for planning and that's what we're that's what i've been working on for many years we've been making fast progress with you know a combination of projects here at NYU and also at meta and now it's time to basically make it make it

Yann LeCun (Chief AI Scientist) 08:21.990

real

Ravid Shwartz-Ziv (Assistant Professor) 08:24.160

and what do you think are the missing parts like and why you think it's taking so long because you're talking about it as you said like for many years already but it's still not better than LLM right it's

Yann LeCun (Chief AI Scientist) 08:38.150

not the same thing as LLM right it's designed to handle modalities that are high dimensional continuous and noisy and LNM's completely suck at this like they really do not work right if you try to train an LLM to kind of learn good representations of images or video they're

Yann LeCun (Chief AI Scientist) 08:57.040

really not that great you know generally vision capabilities for for AI AI systems right are trained separately they're not part of the whole LLM thing so yeah if you want to handle data that is high dimensional continuous and noisy you cannot use generative models you can

Yann LeCun (Chief AI Scientist) 09:21.360

certainly not use generative models that tokenize your data into kind of discrete symbols OK it's just no way and we have a lot of empirical evidence that this simply doesn't work very well what does work is learning an abstract representation space that illuminates a lot of

Yann LeCun (Chief AI Scientist) 09:37.670

details about the input essentially all these details that are not predictable which includes noise and make predictions in that representation space and this is the idea of jetpack johnson then in predictive architectures which you know you are as familiar to yeah

Ravid Shwartz-Ziv (Assistant Professor) 09:54.430

with sorry because you worked on this yeah so also randall was a hosted in the past in in the podcast i probably talked about this at length

Yann LeCun (Chief AI Scientist) 10:06.310

so so there's a lot of ideas around this and let me tell you my history around this OK i have been convinced for a long time probably the better part of twenty years that the the proper way to building intelligent systems was through some form of unsupervised learning i started

Yann LeCun (Chief AI Scientist) 10:26.710

working on unsupervised learning as the basis for you know making progress in the early two thousands mid two thousands before that i wasn't so convinced this was the way to go and and basically this was the idea of you know training auto encoders to learn representations right

Yann LeCun (Chief AI Scientist) 10:47.430

so you have an input you run into an encoder it finds a representation of it and then you decode so you guarantee that the representation contains all the information about the input does that that iteration is long like insisting that the representation contains all the

Yann LeCun (Chief AI Scientist) 11:01.910

information about the input is a bad idea OK i didn't know this at the time so what we worked on was you have several ways of doing this you know jeff hinton at the time was working on restricted boss machines joshua benji was working on the noisy autoeurs which actually became

Yann LeCun (Chief AI Scientist) 11:18.480

quite successful in different contexts right for NLP among others and i was working on sparse auto encoders so basically you know if you train on auto encoder you need to recognize the the representation so that the autoencoder does not trivially learn an identity function and

Yann LeCun (Chief AI Scientist) 11:33.550

this is the information bottleneck podcast listens about information bottleneck right you need to create an information bottleneck to limit the information content of the representation and i thought high dimensional sparse representations was actually a good way to go so so a

Yann LeCun (Chief AI Scientist) 11:50.190

bunch of my students did their PH D on this corelu who is not a chief AI architect at alphabet and also the CTO actually did this finished here on this with me and you know a few a few other a few other false macro on zetto and eden bro and a few others so this was kind of the

Yann LeCun (Chief AI Scientist) 12:12.080

idea and then as it turned out and the idea the reason why we worked on this was because we wanted to pre train very deep neural nets by pre training in those things that auto encoders we thought that was the way to go what happened though was that we started like you know

Yann LeCun (Chief AI Scientist) 12:30.070

experimenting with things like normalization rectification instead of hyperbole tangential to sigmoid like radius that ended up you know basically allowing us to train fairly deep network completely supervised so self supervised learning and this was at the same time that data

Yann LeCun (Chief AI Scientist) 12:51.790

set started to get bigger and so it turned out like you know supervisor it worked fine so the whole idea of self supervised or unsupervised learning was put put aside and then came resnet and you know that's a completely solved the problem of training very deep architecture

Yann LeCun (Chief AI Scientist) 13:07.400

that's right in twenty fifteen but then in twenty fifteen i started you know thinking again about like how do we push towards like human level AI which really was the original objective of fair really and my objective my life 's you know mission and realized that you know all

Yann LeCun (Chief AI Scientist) 13:26.110

the approaches of reinforcement learning and things of that type were basically not scaling you know reinforcement learning is incredibly inefficient in terms of samples and so this is not the way to go and and so the idea of world models right the system that can predict the

Yann LeCun (Chief AI Scientist) 13:44.400

consequence consequences of its action they can plan i started researching playing with this around twenty fifteen sixteen my keynote at what was still called nips at the time in twenty sixteen was on role model i was arguing for it those basically the centerpiece of my talk was

Yann LeCun (Chief AI Scientist) 14:03.590

like this is what we should be working on like you know grandma was action condition and if your residents are working on this on video prediction and things like that we had some papers on video prediction in twenty sixteen and i made a the same mistake as before and the same

Yann LeCun (Chief AI Scientist) 14:22.230

mistake that everybody is doing at the moment which is training a video prediction system to predict at the pixel level which is really impossible and you can't really represent useful probability distributions on the space of video frames those things don't work i knew for a

Yann LeCun (Chief AI Scientist) 14:42.990

fact that because the prediction was nondeterministic we had to have a model with latent variables to represent all the stuff you don't know about the variable you're supposed to predict and so we experimented with this for years i had a student here who is now a scientist at

Yann LeCun (Chief AI Scientist) 15:00.190

fair michael enough who developed a video prediction system with latent variables and he kind of solved this problems we're facing slightly i mean today the solution that a lot of people are employing is diffusion models which is a way to train in non deterministic function

Yann LeCun (Chief AI Scientist) 15:19.710

essentially or energy based models which have been advocating for decades now which also is another way of training non deterministic functions but in the end i discovered that this was all about the idea that the really the way to get around the fact that you can't predict that

Yann LeCun (Chief AI Scientist) 15:38.240

the pixel level is to just not predict the pixel level is to another representation and predict that the representation level eliminating all the details you cannot predict and i wasn't really thinking about those

Yann LeCun (Chief AI Scientist) 15:54.000

methods early on because i thought there was a huge problem of preventing collapse so i'm sure randall talked about this but you know when you train let's say you have an observed variable X and you're trying to predict a variable Y but you don't want to predict all the details

Yann LeCun (Chief AI Scientist) 16:11.800

right to run X and Y to encoders so now you have both a representation for's for X SX and representation for Y S Y you can train a predictor to produce you know predict the representation of Y from the representation of X but if you want to train this whole thing end to end

Yann LeCun (Chief AI Scientist) 16:28.870

simultaneously this is a trivial solution where the system ignores the input and produces constant representations and the predictors probably know is trivial right so if you're on the criterion to train the system is minimized dictionary it's not going to work it's going to

Yann LeCun (Chief AI Scientist) 16:45.670

collapse i knew about this problem for a very long time because i worked on joint embedding architectures we used to call them siamese networks back in the nineties

Ravid Shwartz-Ziv (Assistant Professor) 16:54.110

those are the same because people have been using that term sign means networks even recently that's right

Yann LeCun (Chief AI Scientist) 17:00.270

i mean the concept is still you know up to date right so you have you have an X and Y then think of the X as some sort of degraded transformed or corrupted version of Y OK you run both X and Y two encoders and you tell the system look X and Y really are two views of the same

Yann LeCun (Chief AI Scientist) 17:18.470

thing presentation you compute should be the same right so if you just train a neural net you know two neural nets with shared weights right to produce the same representation for slightly different versions of the same object view whatever it is it collapses it doesn't produce

Yann LeCun (Chief AI Scientist) 17:39.030

anything useful so you have to find a way to make sure that the system you know extract as much information from the input as possible and the original idea that we had you know it was a newspaper from nineteen ninety three with simmons net was to have a contrastive term right

Yann LeCun (Chief AI Scientist) 17:55.190

so you have other pairs of samples that you know are different and you train the system to produce different representations so you have a cost function that attracts the two representations when you show two examples that are identified or similar and you repel them when you

Yann LeCun (Chief AI Scientist) 18:10.390

show it two examples that are dissimilar and we came up with this idea because someone came to us and said like can you encode signatures of someone you know drawing a signature on the tablet can you encode this on less than eighty bytes because if you can encode it in less than

Yann LeCun (Chief AI Scientist) 18:27.790

eighty bytes we can write it on the magnetic tape of a credit card so we can do signature application for credit cards right and so we give up this idea i came up with this idea of training a neural net to produce ad variables that will quantize one by each and then training

Yann LeCun (Chief AI Scientist) 18:48.600

training it to kind of do this thing

Ravid Shwartz-Ziv (Assistant Professor) 18:51.110

and did they use it

Yann LeCun (Chief AI Scientist) 18:52.630

so it worked really well and they showed it to their you know business people who said oh we're just going to ask people to type pink codes we have every

Ravid Shwartz-Ziv (Assistant Professor) 19:03.080

lesson of like that like how you can integrate the technology right

Yann LeCun (Chief AI Scientist) 19:08.280

and you know i knew this thing was kind of fishy in the first place because like you know there were countries in europe that were using smart cards right and it was much better but they just didn't want to use smart crops for some reason anyway so so we had this technology in

Yann LeCun (Chief AI Scientist) 19:24.550

the mid two thousand i worked with two of my students on to revise this idea we came up with a new objective functions to train those so these are where people now call contractive methods it's a special case of contrastive methods we have like positive examples negative

Yann LeCun (Chief AI Scientist) 19:37.790

examples and you train you know on positive examples you train the system to have low energy and for negative samples you train them to have higher energy where energy is the distance between the representations so we had two papers at CDPR in two thousand five two thousand six

Yann LeCun (Chief AI Scientist) 19:52.790

by raya hadsel who is now the head of deep mind foundation the sort of fair like division of deep mind if you want and summit chopra who is actually a faculty here at NYU now working on medical imaging and so this gathered a bit of interest in the community and sort of revived a

Yann LeCun (Chief AI Scientist) 20:16.150

little bit of work on those ideas but it still wasn't working very well those contrasting methods really were producing representations of images for example that were kind of relatively low dimensional if we measured like those the eigenvalue spectrum of the coherence matrix of

Yann LeCun (Chief AI Scientist) 20:32.590

the representations that came out of those things it would fill up maybe two hundred dimensions never more like even training on imagenet and things like that even with data augmentation and so that was kind of disappointing and it did work OK there was a bunch of papers on this

Yann LeCun (Chief AI Scientist) 20:47.120

and it worked OK there was there was white paper from deeper it seemed clear that demonstrated you could get decent performance with contrastive training applied to siamese nets but then about five years ago one of my postdocs stephane denis at meta tried an idea that at first i

Yann LeCun (Chief AI Scientist) 21:13.390

didn't think it would work which was to essentially have some measure of information quantity that comes out of the encoder and then trying to maximize that OK and the reason i didn't think it would work is because i'd seen a lot of experiments along those lines that jeff hinton

Yann LeCun (Chief AI Scientist) 21:32.350

was doing in the nineteen eighties are trying to maximize information and you can never maximize information because you never have appropriate measures of information content that is at orbound if you want to maximize something you want to either be able to compute it or you

Yann LeCun (Chief AI Scientist) 21:50.890

want to a lower bound on it so you can push it up right and for information content we only have upper balance so i always thought this was completely hopeless and then you know stefan came up with a technique which was was called battle twins baru is a famous theoretical

Yann LeCun (Chief AI Scientist) 22:10.800

neuroscientist who came up with the idea information maximization and and it kind of worked it was wow so there i said like we have to push this right so we came up with another method with a student of mine adrian bard co advised with jean-ponse who's affiliated with NYU two

Yann LeCun (Chief AI Scientist) 22:32.720

technique called vikrag variance invariance covariance regularization and that's a network to be simpler and work even better and since then we made progress and randall recently you know discussed an idea with him that can be pushed and made practical it's called sigreg the

Yann LeCun (Chief AI Scientist) 22:51.350

whole system is called he's responsible for the name at all latent euclidean japa right yeah and sigreg has to do with sort of making sure that there are distribution of vectors that come out of the encoder is an isotropic gaussian that's the i in the G so i mean there's a lot

Yann LeCun (Chief AI Scientist) 23:21.110

of things happening in this domain which are really cool i think there's going to be some more progress over the next year or two and we'll get a lot of experience with this and and i think that's kind of a really good promising set of techniques to train models that learn

Yann LeCun (Chief AI Scientist) 23:37.710

abstract representations which i think is key and what do you think are the missing party like do you think like more compute will help or like we need better algorithms like it's kind of like do you believe in the bitter lessons right like do you think well and and furthermore

Yann LeCun (Chief AI Scientist) 23:55.550

what do you think about you know the data quality problems with the internet post twenty twenty two right i've heard people compare it to low background steal now to refer to all that data before LLM's came out like low background tokens i mean OK yeah i think i'm totally

Yann LeCun (Chief AI Scientist) 24:10.510

escaping that problem OK here is the thing and i've i've been you know using this argument publicly over the last couple of years training an LLM if you wanted to have any kind of you know decent performance requires training on basically all the available freely available text

Yann LeCun (Chief AI Scientist) 24:26.670

on the internet plus some you know synthetic data plus licensed data et cetera so a typical LLM like you know number three you know going back a year or two is trained on thirty trillion tokens the token is typically three bytes so that's ten to the fourteen bytes for pre

Yann LeCun (Chief AI Scientist) 24:43.790

training OK we're not talking about fine tuning ten to the fourteen bytes and for the LLM's to be able to really kind of exploit this they need to have a lot of memory storage because basically those are isolated facts there is a little bit of redundancy in text but a lot of it

Yann LeCun (Chief AI Scientist) 25:05.620

is just isolated facts right and so you need a lot of you need very big networks because you need a lot of memory to store all those facts if we go to them OK now compare this with video ten to the fourteen bytes if you count two megabytes per second for video for you know

Yann LeCun (Chief AI Scientist) 25:28.960

relatively compressed video not how you compress but a bit that would represent fifteen thousand hours of video ten to the fourteen bytes if fifteen thousand hours of video you have the same amount of data as the charity of all the texts available on the internet now fifteen

Yann LeCun (Chief AI Scientist) 25:46.870

thousand hours of video is absolutely nothing it's thirty minutes of youtube uploads OK it's the amount of visual information that a four year old has seen in his or her life the entire life waking time is about sixteen thousand hours in four years so a lot of information we

Yann LeCun (Chief AI Scientist) 26:06.990

have video models now vijay vijaypat two actually that just came out last summer that was trained on the equivalent of a century of video data and it's all public data OK much more data but much less than the biggest LNM actually because even though it's it's more bytes it's

Yann LeCun (Chief AI Scientist) 26:29.790

more redundant to say OK it's more redundant so it's less useful actually when you use self supervised learning you do need redundancy you cannot learn anything in self supervise or anything by the way if it's completely random redundancy is what you can learn and so so this

Yann LeCun (Chief AI Scientist) 26:46.790

just much richer structure in you know real world data like video than there is in text which kind of led me to claim that we absolutely never ever going to get to human level AI by just training on text it's just never going to happen right it's a big debate in philosophy of

Yann LeCun (Chief AI Scientist) 27:05.750

whether AI should be grounded in reality or whether it could be just you know in the realm of symbolic manipulation and things like this we talk about for world models and grounding i think you know there's still a lot of people who don't even understand what the idealized world

Yann LeCun (Chief AI Scientist) 27:22.350

model is in a sense right so for example i'm influenced by having watched star trek which i would hope you've seen a little bit of and thinking of the holodecks right i always thought that the holodeck was like an idealized perfect world model right even so many episodes of it

Yann LeCun (Chief AI Scientist) 27:37.950

going too far right and people walking out of it right but you know it even simulates things like smell and physical touch so do you you think that something like that is like the idealized world model or do you think like a different model or like way of defining it would be OK

Yann LeCun (Chief AI Scientist) 27:53.390

this is an excellent question and there is an excellent is because it goes to the core of really what you know what i think we should be doing which i'm doing and how wrong a secret video is OK so so people think you know think that a world model is something that reproduces all

Yann LeCun (Chief AI Scientist) 28:14.910

details of what the world does they think of it as a simulator yeah right and of course because you know deep learning is the thing you're going to use some deep learning system as a simulator a lot of people also are focused on video generation which is kind of a cool thing

Yann LeCun (Chief AI Scientist) 28:29.070

right you you produce those cool videos and they're wow you know people are sort of really impressed by them now there's no guarantee whatsoever then when you train a video generation system it actually has an accurate model of the underlying dynamics of the world and it's

Yann LeCun (Chief AI Scientist) 28:44.070

learned anything you know particularly abstract about it and so so the idea that somehow a model needs to reproduce every detail of the reality is wrong and hurtful and i'm going to tell you why OK a good example of simulation is CFD computational fluid dynamics it's used all

Yann LeCun (Chief AI Scientist) 29:10.550

the time people use supercomputers for that right so you want to simulate the flow of air around an airplane you cut up the space into little cubes and within each cube you have a small vector that represents the state of that cube which is you know velocity density or mass and

Yann LeCun (Chief AI Scientist) 29:32.670

temperature and maybe a couple of other things right so and then you solve navier stokes equations which are which is a differential partial differential equation and you can see related flow of air now the thing is this does not actually necessarily solve the equations very

Yann LeCun (Chief AI Scientist) 29:51.590

accurately if you have chaotic behavior like turbulences and stuff like that simulation is only you know approximately correct but in fact that's already an abstract representation of the underlying phenomenon the underlying phenomenon is molecules of air that bump into each

Yann LeCun (Chief AI Scientist) 30:07.590

other and bump on the wing and on the airplane right ever goes to that level to do the simulation that would be crazy right it would require an amount of computation that's just insane and it would depend on the initial condition i mean there's all kinds of reasons we don't do

Yann LeCun (Chief AI Scientist) 30:24.990

this and maybe it's not molecules maybe it's you know at a lower level we should simulate particles and like you know do the feynman diagrams and simulate you know all the different paths that those particles are employing because they don't take one path right it's not

Yann LeCun (Chief AI Scientist) 30:38.190

classical it's quantum so at the bottom it's like quantum field theory and probably already that that is an abstract representation so so you know everything that takes place between us at the moment in principle can be described through quantum field theory OK we just have to

Yann LeCun (Chief AI Scientist) 30:58.200

measure the wave function of the universe in a cube that contains all of us and even that would not be sufficient because they're entering all particles on the other side of the universe that you know we have so it wouldn't be sufficient but let's imagine OK for the sake of of

Yann LeCun (Chief AI Scientist) 31:14.790

the argument first of all we would not be able to measure this wave function and second of all the amount of competition we would need to devote to this is absolutely gigantic it was released on gigantic quantum computer that you know is the size of euros or something so no way

Yann LeCun (Chief AI Scientist) 31:35.150

we can describe anything at that level and it's very likely that our simulation would be accurate for maybe a few nanoseconds you know beyond that we'll diverge from reality so what do we do we invent abstractions we invent abstractions like particles atoms molecules in the

Yann LeCun (Chief AI Scientist) 31:53.550

living world its proteins organelles sales organs organisms societies ecosystems etc right and basically every level in this hierarchy ignores a lot of details about the level below and what that allows us to do is make longer term more reliable longer term predictions OK so we

Yann LeCun (Chief AI Scientist) 32:19.830

can describe the dynamics between us now in terms of the underlying science and in terms of psychology OK that's a much much higher level abstraction than particle physics right and in fact you know every level in the hierarchy i just i just mentioned is a different field of

Yann LeCun (Chief AI Scientist) 32:36.990

science a field of science is essentially defined by the level of abstraction at which you start making predictions right that you allow yourself to use to make predictions in fact physicists have this down to an art in the sense that you know if i give you a box full of gas you

Yann LeCun (Chief AI Scientist) 32:59.190

could in principle simulate all the molecules of the gas but nobody ever does this but at a very abstract level we can say you know PV equals NRT right you know pressure times value equals the number of particle times you know temperature blah blah blah and so you know that you

Yann LeCun (Chief AI Scientist) 33:20.870

know global emergent phenomenological level if you increase the pressure the temperature will go up or if you increase the temperature the pressure will go up right or if you let some particles out then the pressure will go down and blah blah blah right so so we all the time we

Yann LeCun (Chief AI Scientist) 33:38.710

build phenomenological models

Ravid Shwartz-Ziv (Assistant Professor) 33:41.150

of something complicated by ignoring all kinds of details that physicists call entropy but but it's really systematic that's the way we understand the world we

Ravid Shwartz-Ziv (Assistant Professor) 33:54.200

do not memorize every detail of we certainly not reconstruct it of what we perceive so world models don't have to be simulators at all well there are simulators but in abstract representation space and

Yann LeCun (Chief AI Scientist) 34:08.080

what they simulate is only the relevant part of reality OK if i ask you where is jupiter going to be one hundred years from now i mean we have an enormous amount of information about jupiter right but within this whole information that we have about jupiter to be able to make

Yann LeCun (Chief AI Scientist) 34:24.390

that prediction where jupyter is going to be one hundred years from now you need exactly six numbers three positions and three velocities and the rest doesn't matter so you don't believe in a synthetic datasets i do no it's useful you know data from games i mean there's

Yann LeCun (Chief AI Scientist) 34:40.160

certainly a lot of things that you learn from synthetic data from you know from games and things like that i mean you know children learn a huge amount from from play which basically are kind of simulations you know the world a little bit right but but in conditions where they

Yann LeCun (Chief AI Scientist) 34:58.630

can't kill themselves but i worry at least for video games that for example the green screen like actors doing the animations they're doing extremely it's designed to look good you know for like an often badass i guess for an action game but these often don't correspond very

Yann LeCun (Chief AI Scientist) 35:16.990

well to reality and so i i worry that like a physical system that's you know been trained or through or with the assistance of world models might get similar quirks at least in the very short term is this something that worries you no it depends on what level you train them so

Yann LeCun (Chief AI Scientist) 35:31.350

for example i mean sure if you use a very accurate robotic simulator for example right it's going to accurately simulate the dynamics of an arm you know when you apply torques to it it's going to move in a particular way dynamics no problem now simulating the friction that

Yann LeCun (Chief AI Scientist) 35:47.510

happens you know when you grab an object and manipulate it that's super hard to do it accurately friction is very hard to simulate OK and so those simulators are not particularly accurate for manipulation they're good enough that you know you can train a system to do it and then

Yann LeCun (Chief AI Scientist) 36:02.470

you can do you know seem to real with a little bit of adaptation so that can work but it does not i mean the point is much more important like for example there's a lot of completely basic things about the world that we completely take for granted which we can learn at a very

Yann LeCun (Chief AI Scientist) 36:18.150

abstract level but it's not language related OK so the fact for example and i've used this example before and people made fun of me for it but it's really true OK i have those objects on the table and the fact that when i push the table the object moves with it like this is

Yann LeCun (Chief AI Scientist) 36:33.670

something we learned it's not something that you're born with OK the fact that most objects will fall when you let them go right with gravity maybe it's run this around the edge of and the reason people make fun of me with this is because i said you know LLM's don't understand

Yann LeCun (Chief AI Scientist) 36:50.310

this kind of stuff right and and they absolutely do not even today but but you can train them to give the right answer when you ask them a question you know if i put an object on the table then i push the table what will happen to the object it will answer the object moves with

Yann LeCun (Chief AI Scientist) 37:07.670

it but because it's been fine tuned to do that OK so it's more like regurgitation that sort of real understanding of the underlying dynamics but if you look on i don't know sura like nano nano banana they they have a good physics of the world right they are not perfect they have

Yann LeCun (Chief AI Scientist) 37:23.150

some physics yeah they have some physics right so do you think like we can't push it or do you think like it's a one way to learn physics actually make predictions in our presentation space they use diffusion transformers and that prediction that the computation of the video

Yann LeCun (Chief AI Scientist) 37:45.470

snippet at an abstract level is done in representation space OK not always auto regressively by the way sometimes it's just in parallel and then there's a second diffusion model that turns this abstract representations into a nice looking video and that might be more collapse we

Yann LeCun (Chief AI Scientist) 38:03.190

don't know right because we can't really measure like the coverage of such systems with reality but but like you know the to the the previous point i can train like here is another completely obvious concept to us that we don't even imagine that we learn but we do learn it a

Yann LeCun (Chief AI Scientist) 38:27.390

person cannot be in two places at the same time OK we run this because very early on we learn object permanence the fact that when an object disappears still exists OK and reappears it's the same object that you saw before how can we train an AI system to learn this concept so

Yann LeCun (Chief AI Scientist) 38:47.080

object permanence you know you just show it a lot of videos where objects you know go behind the screen and then reappear on the other side or what they go behind the screen and the screen goes away and the object is still there and when you show four months old babies scenarios

Yann LeCun (Chief AI Scientist) 39:01.320

where things like this are violated their eyes open like super big and they're like super surprised because reality just you know violated their internal model the same thing when you show a scenario of like a little car on the platform you push it off the platform and it

Yann LeCun (Chief AI Scientist) 39:18.270

appears to float in the air they also look at it you know nine months ten months old babies look at it like really surprised six months old baby barely pay attention because they haven't run over gravity yet so they haven't been able to like you know incorporate the notion every

Yann LeCun (Chief AI Scientist) 39:33.720

object is supposed to fall so this kind of learning is really what's what's important and you do this you can learn this from very abstract things you know the same way babies learn about like you know social interactions by you know being told stories with like simple pictures

Yann LeCun (Chief AI Scientist) 39:51.590

it's a simulation an abstract simulation of the world but it sort of launched them you know particular behavior so you could imagine like training a system from let's say an adventure game like a top down to the adventure game where you know you you tell your character like you

Yann LeCun (Chief AI Scientist) 40:07.430

know move north and he goes to the other room and it's not in the first room anymore because it moved to the other room right now of course in adventure games you have gandalf that you can call and it just appears right so that's not physical but but like when you pick up a key

Yann LeCun (Chief AI Scientist) 40:21.470

from a you know from a treasure chest you have the key no one else can have it and you can use it to open a door like there's a lot of things that you learn that are very basic you know even in sort of abstract environments yeah and i just want to observe that some of those

Yann LeCun (Chief AI Scientist) 40:38.110

adventure games that they try to train models and one of them you might know about is nethack right sure and nethack is fascinating because it is an extraordinarily hard game like ever ascending in that game without cheats is like twenty years without you know going to the wiki

Yann LeCun (Chief AI Scientist) 40:55.510

people still don't do it from playing and my understanding is that AI agents the very best agent models we have or even world models are pathetic absolutely yeah yeah so to the point people have come up with sort of you know dumbed down version of net house mini hack mini hack

Yann LeCun (Chief AI Scientist) 41:13.790

exactly

Ravid Shwartz-Ziv (Assistant Professor) 41:14.430

mini hack they had to dumb it down just for for AI so some of my you know colleagues have been working with is actually one of my master students so and and you know michael enough who i mentioned earlier has been also doing some work there now what's interesting there is that

Ravid Shwartz-Ziv (Assistant Professor) 41:33.400

there is a type of situations like this where you need to plan OK but you need to plan in the process of uncertainty the problem with you know all games and adventure games in particular is that you don't have complete visibility of the state of the system you don't have the map

Ravid Shwartz-Ziv (Assistant Professor) 41:45.910

in advance you need to explore and blah blah blah you can get killed every time you do this and you know but the actions are essentially discreet yes correct possible actions is turn based and so in that sense it's like chess acceptance is not you know chess is fully observable

Ravid Shwartz-Ziv (Assistant Professor) 42:04.070

go also it's really observable stratego isn't that stratego isn't you know poker is not and so it makes it more difficult if you have uncertainty of course but those are games where the number of actions you can take is discrete and basically you know what you need to do is do

Ravid Shwartz-Ziv (Assistant Professor) 42:29.070

tree exploration OK and to do of course the tree of possible states you know goes exponentially with the number of moves and so you have to have some way of generating only the moves that are likely to be good and basically never generate the other ones or select them down and

Ravid Shwartz-Ziv (Assistant Professor) 42:47.550

you need to have a value function which is something that tells you OK i can't plan to the end of the game but even though i'm planning sort of nine moves ahead i have some way of estimating whether evaluating whether a position is good or bad it's going to lead me to you know

Ravid Shwartz-Ziv (Assistant Professor) 43:02.190

victory or solution right so you need those two components basically something that guesses what the good moves are and then something that you know essentially evaluates ends and if you have those both of those things you can train those functions using something like

Ravid Shwartz-Ziv (Assistant Professor) 43:18.350

reinforcement learning or behavioral cloning if you have data i mean the basic idea for this goes back to samuel 's checker players from nineteen sixty four it's not recent but but of course was you know the power of it was demonstrated with you know alphago and alpha zero and

Ravid Shwartz-Ziv (Assistant Professor) 43:37.110

things like that so that's good but that's a domain where humans suck humans are terrible at playing chess right they're playing go like machines are much better than we are because of the speed of tree exploration and because of the memory that's required for for tree

Ravid Shwartz-Ziv (Assistant Professor) 43:53.870

exploration we just don't have enough memory capacity to do breadth first tree exploration so we suck at it like you know when alphago came out you know people before that thought that the best human players were maybe two or three stones handicapped like below an ideal player

Ravid Shwartz-Ziv (Assistant Professor) 44:10.950

that they call god no like you know humans are terrible like we you know the best players in the world need like eight or nine stories well i i can't believe i i get the pleasure to talk about game AI with with yan i just have a few follow up questions on this the first one is

Ravid Shwartz-Ziv (Assistant Professor) 44:29.030

this example that you talk about around humans being terrible at chess and i'm familiar a bit with the development of chess AI over the years you know i've heard this referred to as more of X paradox and explained as you know humans have evolved over billions or millions sorry

Ravid Shwartz-Ziv (Assistant Professor) 44:46.760

it's large N number of years to physical locomotion and that's why babies and humans are very good at this but we have not evolved at all to play chess so that's one question and then a second question that's related is a lot of people today who play video games and i'm one of

Ravid Shwartz-Ziv (Assistant Professor) 45:02.870

them have observed that it feels like AI at least in terms of like enemy AI has not improved really in twenty years right that some of the best examples are still like halo one and fear from the early two thousands so when do you think that you know advancements that we've been

Ravid Shwartz-Ziv (Assistant Professor) 45:19.230

doing in the lab are going to actually have real impact on like gamers you know and and in a non like generative AI sense yeah i used to be a giver never addictive one but but my family is in it because i have three sons in their thirties and they have a video game design studio

Ravid Shwartz-Ziv (Assistant Professor) 45:37.070

between them so so i was sort of you know embedded in that culture but yeah no you're right and you know it's it's also it's also true that the you know despite the accuracy of physical simulators a lot of the a lot of those simulations are not used by studios who make animated

Ravid Shwartz-Ziv (Assistant Professor) 45:58.720

movies because they want control they don't necessarily want accuracy they want control and in games it's only the same thing it's a creative act and what you want is some control about the course of the story or the way the you know NPC kind of behave and all that stuff right

Ravid Shwartz-Ziv (Assistant Professor) 46:16.710

and the key AI kind of you know is difficult to maintain control at the moment so i mean it will come but you know there's there's some resistance from the creators but i think OK marvik paradox is very much still in force so moravec i think formulated it in nineteen eighty

Ravid Shwartz-Ziv (Assistant Professor) 46:38.030

eight if i don't know correctly and he said like yeah you know things that we think of as uniquely human intellectual tasks like PHS we can do with computers or you know computing integrals or whatever but the thing that we take for granted we don't even think is an intelligent

Ravid Shwartz-Ziv (Assistant Professor) 47:00.770

task like what a cat can do we still can do with robots and even now forty seven years later we still can't do them well i mean of course we can you know train robots you know by imitation and a bit of reinforcement learning and you know by training through simulation to kind of

Ravid Shwartz-Ziv (Assistant Professor) 47:21.870

locomote and you know what obstacles and do various things but then not nearly as inventive and creative and you know agile as a cat it's not because we can't build a robot we certainly can it's just we can't we can't make them smart enough to do all the stuff that a cat or unit

Ravid Shwartz-Ziv (Assistant Professor) 47:42.190

mouse can do let alone the dog or a monkey right so so you have all those people bloviating about like you know AGI in in a year or two

Yann LeCun (Chief AI Scientist) 47:53.800

is completely deluded just completely delusion because the real world is way more complicated are you not going to get it you're not going to get anywhere by tokenizing the world and using NMS it's just not going to happen

? (?) 48:08.000

so so what is your timelines when we will see like intel AGI whatever it means

? (?) 48:15.200

or like and also where are you and where are you on the optimist pessimist sorry because you know there's some doomers among or doomerism amongst like gary marcus and and i think well

Yann LeCun (Chief AI Scientist) 48:26.160

i know gary marcus no

? (?) 48:27.280

he's he's a critique he's critiques it sorry the dumer would be

? (?) 48:30.670

yahshua yeah yeah there you go like where do you fall on all

Yann LeCun (Chief AI Scientist) 48:34.030

these things OK i'll answer the first question first OK so first of all there is no such thing as general intelligence this concept makes absolutely no sense because it's it's really designed to designate human level intelligence the human intelligence is super specialized OK we

Yann LeCun (Chief AI Scientist) 48:57.910

can handle the real world really well like navigate and blah blah blah we can handle other humans really well because we evolved to do this and chest we suck OK so and there's a lot of tests that we suck at that where a lot of other animals are much better than we are OK so what

Yann LeCun (Chief AI Scientist) 49:13.190

that means is that we are specialized we think of ourselves as being general but it's simply an illusion because all of the problems that we can apprehend are the ones that we can think of right and vice versa and so we're in general in all the problems that we can imagine but

Yann LeCun (Chief AI Scientist) 49:32.470

these are a lot of problems that we cannot imagine and there's some mathematical arguments for this which i may not go into unless you ask me but so there is so this this this concept of general intelligence is complete BS we can talk about human level intelligence right so are

Yann LeCun (Chief AI Scientist) 49:53.360

we going to have machines that are as good as humans in all the domains where humans are good or better than humans and the answer is you know we already have machines that are better than humans in some domains like you know we have machines that can translate you know fifteen

Yann LeCun (Chief AI Scientist) 50:06.360

hundred languages into fifteen hundred other languages in any direction no humans can do this right and and you know there's a lot of examples and you know which hasn't go in various other things umm but while we have machines that are as good as humans in all domains the answer

Yann LeCun (Chief AI Scientist) 50:26.340

is absolutely yes there's no question at some point we'll have machines that are as good as humans in all domains

? (?) 50:32.500

OK and that leads but it's not going to be an event it's going to be very progressive we're going to make some conceptual

Yann LeCun (Chief AI Scientist) 50:40.630

advances maybe based on you know jetpack one models planning things like that uh over the next few years and if we're lucky if you don't hit an obstacle that we see uh perhaps this will lead to kind of good paths to human level AI but but perhaps we're still missing a lot of

Yann LeCun (Chief AI Scientist) 50:59.670

basic concepts and so the most optimistic view is that perhaps you know the you know learning goodwill models and and you know being able to do panning and you know understanding complex signals that are continuous high dimensional noise significant progress in that direction

Yann LeCun (Chief AI Scientist) 51:20.110

over the next two years the most optimistic view is that we'll have something that is close to human intelligence or maybe dog intelligence within you know five to ten years OK but that's the most optimistic it's very likely that as what happened you know

? (?) 51:38.310

multiple times in the history of AI in the past there's some obstacle we're not seeing yet which will you know actually kind of require us to invent some new conceptual new things to go beyond in which case that may take twenty years maybe maybe more OK but no question it will

? (?) 51:57.790

happen no question do you think it will be easier to get from the current level to a dog level intelligence compared to a dog to humans levels no i think i think the hardest part is to get to dog level once you get to dog level you basically have most of the ingredients right

? (?) 52:14.190

and then you know what's missing from OK what's missing from like primates to humans beyond just size of the brain is language maybe OK but language is basically handled

Yann LeCun (Chief AI Scientist) 52:25.270

by

? (?) 52:26.510

the vernicki area which is a tiny little piece of brain dust

Yann LeCun (Chief AI Scientist) 52:30.110

right here and the brochure area which is a tiny piece of brain right here both of those evolved in the last you know less than a million years maybe two and it can be that complicated and we already have it at ends that do a pretty good job at you know you know encoding

Yann LeCun (Chief AI Scientist) 52:46.310

language into abstract representations and then decoding thoughts into into text so maybe we'll use LLM for that so LLM will be like the wernicke and brochure areas in our brain what we're working on right now is the prefrontal cortex which is where one model resides well well

Yann LeCun (Chief AI Scientist) 53:03.830

this this gets me into you know a few questions about safety and the destabilizing potential impact so i'll start this with something a little bit funny which is to say if we really get dog level intelligence then the AI of tomorrow has gotten profoundly better than any human at

Yann LeCun (Chief AI Scientist) 53:18.870

smell and and something like that is just you know tip of the iceberg for the the destabilizing impacts of AI tomorrow let alone today i mean we have sam altman talking about super persuasion because AI docs is you so it it figures out who you are through the multi turn so it

Yann LeCun (Chief AI Scientist) 53:35.270

gets really good at kind of customizing its arguments towards you we've had AI psychosis right like people who have done horrible things as a result of kind of believing in a syncophantic AI that that is telling them to do things they shouldn't do happened to be by the way whoa

Yann LeCun (Chief AI Scientist) 53:53.190

you've got to tell us about that too what wendy a few months ago it was it was i didn't want you and i walked down to get lunch and there's a dude who's surrounded by a whole bunch of police officers and and security guys and i work past and the guy recognizes me and says oh

Yann LeCun (Chief AI Scientist) 54:11.070

mister lekun and the police officer kind of whisked me away outside and tells me like you don't want to talk to him turns out the guy had come from you know the midwest by bus here and he he's a kind of emotionally disturbed he you know he had gone through prison blah blah blah

Yann LeCun (Chief AI Scientist) 54:36.950

for kind of various things and he was carrying a bag with you know like a huge wrench and and pepper spray and a knife and so the the security guards got alarmed and basically called the police and then the police realized OK you know this guy is kind of weird so they you know

Yann LeCun (Chief AI Scientist) 54:54.790

took him away and had him examined and eventually he went back to the midwest but i mean he didn't feel threatening to me but the police wasn't so sure so so yeah it happens i had you know high school students writing emails to me saying i read all of those you know piece by

Yann LeCun (Chief AI Scientist) 55:16.880

doomers who said like you know AI is going to take over the world and either kill us all or take our jobs so what i'm totally depressed i don't go to school anymore and so i you know i answer to them said like you know don't i don't believe all that stuff you know but humanity

Yann LeCun (Chief AI Scientist) 55:33.150

is still going to be in control of of all of this now there's there's no question that you know every powerful technology has you know good consequences and bad side effects that sometimes are predicted and corrected sufficiently in advance and sometimes not so much right and

Yann LeCun (Chief AI Scientist) 55:54.040

it's always a trade off that's a history of technological progress right so they stay cars as an example OK cars crash sometimes and initially you know brakes were not that reliable and cars would like flip over and there was no you know seatbelts and blah blah blah right and

Yann LeCun (Chief AI Scientist) 56:12.670

eventually kind of the industry made

? (?) 56:14.110

progress and you know started putting sealbelts and and crumple zone and and and you know automatic kind of controlling systems so that you know the car doesn't go sway and doesn't flip or whatever so cars now are much safer than they used to be there's one thing that is now

? (?) 56:37.180

mandatory in every car sold in the EU and it's actually an AI system that looks out the window it's called it's called AEDS automatic emergency braking system it's basically a common shot

Yann LeCun (Chief AI Scientist) 56:50.380

there right and it looks at the windshield and it detects you know all objects and if it detects on an object is too close it just automatically breaks and if you detect that it's going to be a collision that the driver is not going to be able to avoid it just stops the car or

Yann LeCun (Chief AI Scientist) 57:12.550

sways right and that when statistics i read is that this reduces frontal collisions by forty percent and so it became mandatory equipment in every car sold in the EU even low end because it saves rise so this is AI not killing people saving lives right i mean also same thing for

Yann LeCun (Chief AI Scientist) 57:34.070

like medical imaging and everything there's a lot of live being saved by AI at the moment and like so but do you think so you jeff and joseph right like both of you won the the touring award together and like and you have different opinions about it right and jeff says like he's

Yann LeCun (Chief AI Scientist) 57:57.590

regrets and joshua works on safety and you trying to push it forward and do you think you will get to some some level of intelligent you will say oh this become too dangerous we need to work more on the safety side i mean you have to do it right i'm going to use another example

Yann LeCun (Chief AI Scientist) 58:22.510

jet engines OK i find this astonishing that you can fly halfway around the world on a two engine airplane in complete safety and i really really say halfway around the world like it's a seventeen hour flight OK from new york to singapore right on the airbus three fifty it's

Yann LeCun (Chief AI Scientist) 58:46.790

astonishing and when you look at a jet engine the turbofan it should not work right i mean there is no metal that can stand the type of temperature that takes place there in the kind of like efforts when you have like a huge turbine like you know rotating a two thousand or i

Yann LeCun (Chief AI Scientist) 59:06.230

don't know what speed like the the forest that was on it is just insane you know it's hundreds of tons so it should not be possible yet those things are incredibly reliable so what i'm saying is you can't you know build

? (?) 59:25.910

something like a turbojet the first time you build it it's not going to be safe it's going to run for ten minutes and then blow up OK and it's not going to be fuel efficient and it's you know etc it's not going to be reliable but you know as you make progress in engineering and

? (?) 59:44.030

materials et cetera there's so much you know economic motivation

Yann LeCun (Chief AI Scientist) 59:48.270

to make this good that you know eventually it's going to be the type of reliability we see today the same going to be true for AI we're going to start making systems that you know have agency can plan can reason have role models blah blah blah but we you know they're going to

Yann LeCun (Chief AI Scientist) 60:07.070

have the power of maybe a cat brain right which is about one hundred times smaller than a human brain put guardrails in them to prevent them from doing you know taking actions that are obviously uh dangerous or something you can do this at a very low level like if you have

? (?) 60:27.030

i don't know a domestic robot right that oh so so one example that stuart russell for example have used is is to say well you know if you have a robot the domestic robot and you ask you to fetch you coffee and someone is standing in front of the coffee machine if the system

? (?) 60:44.670

wants to fulfill its goal it's going to have to you know either assassinate or smash the person in front of the coffee machine to get access

Yann LeCun (Chief AI Scientist) 60:53.110

to the coffee machine and obviously you don't want that to happen now it's like the pay per click maximization it's kind of a ridiculous example because it's super easy to fix right you put some guardrail that say well you know you're a domestic robot you should stay away from

Yann LeCun (Chief AI Scientist) 61:08.030

people and maybe ask them to move if they are in a way but not actually kind of you know hurt them in any way or whatever you can do like you know you can put a whole bunch of low level conditions like this like if you have domestic robot and it's you know it's a cooking robot

Yann LeCun (Chief AI Scientist) 61:22.190

right so it has a big knife in its hand and it's you know cutting the cucumber you know don't flare your arms if there are if you're a big knife in your hand and people around OK it can be kind of a low level constraint that the system has to satisfy now some people say oh but

Yann LeCun (Chief AI Scientist) 61:39.590

you know with NLM 's we can fine tune them to not do things that are dangerous but there is always you can you can generate them you can always find prompts where they're going to kind of escape their condition you know the all the things that we stop them from from doing i

Yann LeCun (Chief AI Scientist) 61:56.870

agree that's why i'm saying we shouldn't use LMS we should use those objective driven AI architectures i was talking about earlier where you have a system that has a word model can predict the consequences of its action and can figure out a sequence of actions to accomplish a

Yann LeCun (Chief AI Scientist) 62:12.070

task but also is subject to a bunch of constraints that guarantee that whatever action is being pursued and whatever state of the world is being predicted does not endanger anybody or does not have you know when they get negative side effects right so there is it's by

Yann LeCun (Chief AI Scientist) 62:30.230

construction the system is intrinsically safe because it has all those guardrails and because it obtains its output by optimization by minimizing the objective of the task and satisfying the constraints of the guardrails it cannot escape that it's not a fine tuning right it's by

Yann LeCun (Chief AI Scientist) 62:50.310

construction yeah and and i'll there's a technique you know that that for LLM's for constraining the output space where you say that you ban all outputs except whatever you want like maybe zero to ten and everything else and they have that even for diffusion models sure do you

Yann LeCun (Chief AI Scientist) 63:08.790

think that tactics like that as they exist today significantly improve the utility of those kinds of models well they do but they're ridiculously expensive because the the way they work is that you have to have the system generate lots of proposals for an output and then have a

Yann LeCun (Chief AI Scientist) 63:23.560

shelter that says well this one is good this one is terrible etc i'll rank them and then just put out the one that has the the less toxic rating essentially so it's it's insanely expensive right so unless you have you know some sort of objective driven value function

? (?) 63:41.390

that kind of drives the system towards producing those high you know high score outputs low toxicity outputs it's going to be it's going

Yann LeCun (Chief AI Scientist) 63:51.150

to be expensive yeah and

? (?) 63:53.270

i want to change the topic just a tiny bit off we've been very technical for a moment but we you know i think our audience in the world

? (?) 64:01.310

has a few questions that are maybe a little bit more more social related you know the person who appears to be trying to fill your shoes in

Yann LeCun (Chief AI Scientist) 64:10.030

at meta alex wang where i'm curious as to do you have any thoughts or

Yann LeCun (Chief AI Scientist) 64:15.070

anything about you know kind of how how that will play out for for meta is not is not in my shoes at all he's he's he's in charge of all the R and D and product that are AI related at beta so it's not a researcher or scientist or or anything like that it's more kind of you know

Yann LeCun (Chief AI Scientist) 64:37.110

overseeing the entire operation so within meta superintelligence lab which is his organization kind of divisions if you want so one of them is fair which is long term research one of them is TBD lab which is basically building frontier models which is mostly entirely LLM focused

Yann LeCun (Chief AI Scientist) 65:04.820

a fourth organization is AI infrastructure software infrastructure hardware is some other organization one is products OK so people who take the frontier models and then turn them into actual chat bots that people can use and you know disseminate them and you know plug them into

Yann LeCun (Chief AI Scientist) 65:22.710

whatsapp and everything else right so so those are four divisions he overseas all of that so and there are several AI scientists there is AI scientists are fair that's me and i really have a long term view and basically you know i'm going to be at meta for another you know three

Yann LeCun (Chief AI Scientist) 65:43.030

weeks OK so and and fair is led by or NYU colleague rob fergus right now after joel pino left several months ago fair is being pushed towards kind of working on slightly you know shorter term projects that it has done in the in the traditionally with less emphasis on publication

Yann LeCun (Chief AI Scientist) 66:14.230

more focused on sort of helping TBD lab with the LLM's and frontier models and and you know last publication which means you know meta is becoming a little more close closed and TBD lab has achieved scientists also but which is really focused on LLM and other organizations are

Yann LeCun (Chief AI Scientist) 66:40.590

more like infrastructure and products so you know there's some appropriate research there so for example the group that works on

? (?) 66:46.310

sam segment eighteen yeah yeah that's actually part of the product division of missile there used to be at fair but because they worked on kind of relatively you know kind of outside facing kind of practical things that were kind of moved to department and and do you have any

? (?) 67:04.160

opinions on like some of the other companies that are trying to move into world models like thinking machines or even i've heard jeff bezos and and some of his it's not clear at all what seeking machine is doing maybe you have more information than me but maybe not sorry maybe

? (?) 67:21.080

i'm mixing it up here physical

Yann LeCun (Chief AI Scientist) 67:23.440

intelligence physical sorry yeah sorry and then i mix them up with like SSI as well they're all kind of like so nobody knows what they're doing including their own investors OK and he said the rumors that's a rumor i heard i don't know if it's true it's become a bit of a joke

Yann LeCun (Chief AI Scientist) 67:44.230

but but yeah physical intelligence company is is focused on you know basically producing geometrically correct videos OK where you know there is persistent geometry and you know when you look at something and you turn around and you come back it's the same object you had before

Yann LeCun (Chief AI Scientist) 68:11.030

like it doesn't change behind your back right so it's it's generative right i mean the whole idea is to generate pixels which i just spent you know a long time arguing against that was a bad idea there are other companies that are have role model a good one is wave wave W A Y V

Yann LeCun (Chief AI Scientist) 68:34.430

E so it's a company based in oxford and they i'm an advisor for full disclosure and they have they have a role model for autonomous driving and the way they're training it is that they're training a representation space by basically training a VAE or VQVAE and then training a

Yann LeCun (Chief AI Scientist) 68:52.670

predictor to do temporal prediction in that abstract representation space so they have half of it right and half of it wrong the piece they have right is that you make predictions in representation space the pieces are wrong is that they haven't figured out how to train their

Yann LeCun (Chief AI Scientist) 69:07.390

representation space in any other way than by reconstruction and i think that's bad OK but the MRL is great like it works really well i mean among all the people who kind of work in this kind of stuff they're pretty far advanced there are people who talk about similar things and

Yann LeCun (Chief AI Scientist) 69:24.790

nvidia a company called sandbox AQ the CEO of it jack hilary talks about qualitative models you know large quantitative models as opposed to large language models predictive models that can deal with continuous high dimensional noisy data right which is what also i've been kind

Yann LeCun (Chief AI Scientist) 69:44.150

of talking about and google of course has been working on you know on word models mostly using generative approaches there was an interesting effort at google by dani jar after so he built models called dreamer dreamer V one two three four yeah that was on a good path except he

Yann LeCun (Chief AI Scientist) 70:05.600

just left google to create his own startup and do you have so i'm interested so you were really criticized about the silicon valley culture that they are focusing on NLM and this is like one of the reasons that now you started with the new company is starting in paris right so

Yann LeCun (Chief AI Scientist) 70:33.360

this is something do you think that we will see more and more or do we see this is something will be very unique

? (?) 70:41.430

that only a few companies will be in europe running is global OK it has an office in paris but it's a global company has office in new york too a couple other places so OK there is an interesting phenomenon in industry which is that everybody has to do the same thing as

? (?) 71:03.510

everybody else because it's so competitive that if you start

Yann LeCun (Chief AI Scientist) 71:07.110

taking attention you're taking a good risk of falling behind because you're using a different technology than everybody else right so basically everyone is trying to catch up with the others and so that creates this herd effect and a kind of monoculture which is really specific

Yann LeCun (Chief AI Scientist) 71:23.470

to silicon valley where you know open AI beta google into our pick everybody is basically working on the same thing and you know sometimes like what happened a while back another group you know like deepseek in china comes up with kind of a new way of doing things and everybody

Yann LeCun (Chief AI Scientist) 71:43.710

is like what right you mean like other people in silicon valley are not stupid and can come up with original ideas i mean there's a bit of a you know superiority complex right but you're basically in your trench and you are you have to move as fast as possible because you can't

Yann LeCun (Chief AI Scientist) 72:00.080

afford board to kind of you know fall behind the other guys who you think are your competitors but you run the risk of being surprised by something that's completely out of the left field that uses a different set of technologies and or maybe addresses a different problem so you

Yann LeCun (Chief AI Scientist) 72:17.560

know what i've been interested in is completely orthogonal because the the whole japan idea word model is really to handle data that is not easily handled by LLM so the type of applications we're envisioning that have tons of applications in industry

? (?) 72:33.550

where the data comes to you in the form of continuous high dimensional noisy data

Yann LeCun (Chief AI Scientist) 72:38.350

including video are domains where LMS basically are not present where people are trying to use them and totally failed essentially right so if you don't want to be OK so the expression in silicon

? (?) 72:53.310

valley is that you are LLM pilled you you think that the path to superintelligence you just get up and adams you train on more synthetic data you license on more data you hire thousands of people to kind of find you know to basically school your

Yann LeCun (Chief AI Scientist) 73:09.040

system in post training you invent a new tweaks on RL

? (?) 73:13.680

and you're going to get to super intelligence and this i think is complete bullshit like it's

Yann LeCun (Chief AI Scientist) 73:18.160

just never going to work and then you add a few you know kind of reasoning techniques which basically consist in you know doing like super long chain of thoughts and then having the system generate lots and lots of different token outputs you know from which you can select good

Yann LeCun (Chief AI Scientist) 73:32.830

ones using some sort of valuation function the second LLM basically you hit i mean that's the word all those things work this is not going to take us it's just not so so yeah i mean you need to escape that culture and there are people within all the companies in silicon valley

Yann LeCun (Chief AI Scientist) 73:49.400

who think like this is never going to work i want to i want to do an event in japan blah blah blah i'm hiring them so yeah so escaping the moodle culture of silicon valley i think is important yeah this is a this part of the the story and what do you think about the competition

Yann LeCun (Chief AI Scientist) 74:17.030

between like the US china and the and europe like now that you are starting a company like do you see more i know that some there are some places are more attractive than others we're in this very paradoxical situation where all the american companies until now not meta but all

Yann LeCun (Chief AI Scientist) 74:39.840

american companies have been kind of becoming really secretive and to preserve their competitive what they think is a competitive advantage and by contrast the chinese players companies and others have been completely open so the best open source systems at the moment are

Yann LeCun (Chief AI Scientist) 75:00.630

chinese and that causes a lot of the industry to use them because they want to use a pencil system and they hold their nose a little bit because they know those models are kind of fine tuned to not answer questions about politics and stuff like that right but they don't really

Yann LeCun (Chief AI Scientist) 75:18.560

have a choice and certainly a lot of academic research now you know uses the best chinese models certainly everything that has to do with like reasoning and things like that right so so it's really paradoxical and a lot of people in the US in industry are really unhappy about

Yann LeCun (Chief AI Scientist) 75:37.430

this they really want a serious non chinese open source model there could have been was a disappointment for various reasons maybe that will get fixed with you know the the new efforts at meta or maybe meta will decide to go close as well it's still clear mistral just had a

Yann LeCun (Chief AI Scientist) 75:57.160

model early yes just be cool for cogen yeah yeah yeah that's right no it's it's it's cool so yeah they they maintain openness yeah no it's really really interesting what they're what they're doing yeah wow OK let's go to more personal questions yeah yeah so like you are sixty

Yann LeCun (Chief AI Scientist) 76:23.510

five right years old you want a turing award you just got a queen elizabeth prize basically you could retire right yeah i could that's what my wife wants me to do

? (?) 76:36.320

so why why to start a new company now like what keep you happen because i have a mission you know i mean i always thought that

Yann LeCun (Chief AI Scientist) 76:49.390

either making people smarter or more knowledgeable or making this model with the help of machines so basically increasing the amount of intelligence in the world was an intrinsically good thing OK intelligence is really kind of the commodity that is the most in demand certainly

Yann LeCun (Chief AI Scientist) 77:09.950

in like government OK so but but like in you

? (?) 77:17.080

know every aspect of of life we are limited as you know as as a species as a planet by the limited supply of intelligence

Yann LeCun (Chief AI Scientist) 77:29.480

right which is why we we we spend enormous resources educating people and and and things like that so you know increasing the amount of intelligence at the service of humanity or the planet more globally not just humans is intrinsically a good thing despite all the what the

Yann LeCun (Chief AI Scientist) 77:51.570

doomers are saying OK of course you are dangerous and you have to protect against that in the same way you have to make sure your jet engine is safe and reliable and your car doesn't kill you with a you know small crash right but that's OK that's an engineering problem there's

Yann LeCun (Chief AI Scientist) 78:06.920

no there's no like fundamental issue with that also with political department but not it's not like insurmountable so that's an interestingly good thing and if i can contribute to this i will and basically all research projects i've done in my entire career even those that were

Yann LeCun (Chief AI Scientist) 78:25.350

not related to machine learning in my professional activities were all focused on either making people smarter that's what that's why i'm

? (?) 78:34.190

a professor and

Yann LeCun (Chief AI Scientist) 78:39.640

that's why also i'm communicating publicly a lot about AI and science and things like that and a big presence on social networks and stuff like that right because i think people should know stuff right but also on machine intelligence because i think machines will assist humans

Yann LeCun (Chief AI Scientist) 78:59.470

and make them smarter OK people think there is a fundamental difference between trying to make you know machines that are intelligent and autonomous and blah blah blah and and it's a different set of technologies so i'm trying to make machines that are assistive to humans it's

Yann LeCun (Chief AI Scientist) 79:17.360

not it's the same technology it's exactly the same and it's not because the system is intelligent or even a human is intelligent that it wants to dominate or take over it's not even true of humans like it's not the humans who are the smartest that want to dominate

Yann LeCun (Chief AI Scientist) 79:32.030

others we see this on the international political scene every day it's not the smartest among among us who want to be the chief and probably many of the smartest people that we've ever met are people who basically want nothing to do with the rest of humanity right they just want

Yann LeCun (Chief AI Scientist) 79:50.480

to work on their problems you know kind of stereotyping that's what

? (?) 79:58.960

hannah wren talks about the vita contemplativa right versus like the active life or the contemplative life right and her like philosophical analysis and like making a choice kind of early on on what you work on

Yann LeCun (Chief AI Scientist) 80:11.230

right but you can be you know simultaneously kind of you know a dreamer or contemplative but have a big impact on the world right by you know your scientific production like think of einstein or something

Yann LeCun (Chief AI Scientist) 80:25.110

yeah or even newton like newton basically didn't want to meet anybody

? (?) 80:31.070

famously

Yann LeCun (Chief AI Scientist) 80:33.520

or pull the rack pull the rack was kind of you know practically autistic well well

? (?) 80:42.680

is there like a paper or idea you haven't written or or something else that you you're you know nagging that you want to get to or maybe that you don't have time or any regret

Yann LeCun (Chief AI Scientist) 80:52.590

oh yeah a lot oh my entire career has been a succession of me not devoting enough time to express my ideas and writing them down and mostly getting scooped

? (?) 81:08.920

what is the most significant one i

Yann LeCun (Chief AI Scientist) 81:13.200

don't want to go through that

Yann LeCun (Chief AI Scientist) 81:16.160

the backprop is a good one

? (?) 81:18.510

OK

Yann LeCun (Chief AI Scientist) 81:19.350

i published some sort of early version of some algorithm to trade multilayer nets which today we would call target prop and i had the back prop team figured out except i didn't write it before you know there were more heart engine they were nice enough to cite my earlier paper

Yann LeCun (Chief AI Scientist) 81:39.840

in their in theirs but so there's been a few of those yeah recurrent nets you know various other things but and things are more perhaps more recent but you know i have no regrets about this like you know this is life like you know i'm not going to say oh you know i invented this

Yann LeCun (Chief AI Scientist) 81:58.830

in nineteen ninety one and i should have

? (?) 82:01.230

like somewhere

? (?) 82:04.800

i i don't know i should say the name we

Yann LeCun (Chief AI Scientist) 82:08.120

all know well i mean if you know you know the way ideas pop up you know is is relatively complex it's rare that someone comes up with an idea in complete information and that you know nobody else comes up with similar ideas at the same time most of the time appear simultaneously

Yann LeCun (Chief AI Scientist) 82:25.110

but then there is various ways to is having the idea and then there is kind of writing it down but there is also writing it down in a sort of convincing way in a clear way and then there is kind of making it work on toy problems maybe OK and then there is making the theory that

Yann LeCun (Chief AI Scientist) 82:40.670

shows that it can work and then there is making it work on a real application right and then there is making a product out of it OK so this whole chain and you know some people are really extreme think that the only person who should get all the credit is the very first person

Yann LeCun (Chief AI Scientist) 82:55.110

who got the

? (?) 82:56.030

idea

Yann LeCun (Chief AI Scientist) 82:57.030

i think that's wrong there's there's a lot of really difficult steps to get this idea to the state what actually works so this idea of role model i mean goes back to the nineteen sixties you know people in optimal control had water models to do planning that's the way NASA you

Yann LeCun (Chief AI Scientist) 83:11.870

know planned the trajectory of the rockets to go to to orbit basically simulating the rocket and sort of bio optimization figuring out the the controller to get the rocket to where it needs to be so that's another idea very old idea the the fact that you could do some level of

Yann LeCun (Chief AI Scientist) 83:32.990

training or adaptation in this is called system identification in article control very old idea too goes back to the seventies something called yeah system identification or even MPC where you adapt the model as it goes like while you're running the system then go back to the

Yann LeCun (Chief AI Scientist) 83:48.390

seventies to some obscure paper in france and then the fact that you can just learn a model from data people have been working on this with neural net since the nineteen eighties right and and not just

Yann LeCun (Chief AI Scientist) 84:04.110

yoga it's like a whole bunch of people who have been working people who came from optimal control and realized they could use neural nets as kind of a universal function approximator and use it for direct control or feedback control or role models for planning blah blah blah and

Yann LeCun (Chief AI Scientist) 84:21.230

like a lot of things in neural nets in the nineteen eighties and nineties it kind of worked but not like to the point where it took over the industry so it's the same for you know computer vision speech recognition there were attempts at using neural nets for that back in those

Yann LeCun (Chief AI Scientist) 84:38.910

days but it started really really working well in the late two thousands where it totally took over right and then early two thousand ten S for vision mid two thousand ten's for NLP and for robotics it's starting but it's not

? (?) 84:55.030

why why i think it's only like in the this time started to get over the well

Yann LeCun (Chief AI Scientist) 85:02.150

it's combination of like having the right state of mind about it and the right mindset having the right architectures the right machine learning techniques like you know residual connections real use whatever then having powerful enough computers and having access to data and

Yann LeCun (Chief AI Scientist) 85:21.000

it's only when those planets are aligned that you get a breakthrough right which appears like a conceptual breakthrough but it's actually just a practical one like OK let's talk about confidential nets OK lots of people during the seventies had the idea or even during the

Yann LeCun (Chief AI Scientist) 85:40.190

sixties actually had the idea of using local connections like building a neural net with local connections for extracting local features and the idea that local features is like convolution like in image processing is like you know goes back to the sixties so these are not new

Yann LeCun (Chief AI Scientist) 85:56.270

concepts the fact that you can learn adaptive filters of this type using data goes back to the perceptron and adaline which is early sixties OK but that's only for one layer now the concept that you can train a system with multiple layers everybody was looking for this in the

Yann LeCun (Chief AI Scientist) 86:11.990

sixties nobody found a lot of people made proposals which kind of have worked but like none of them was convincing enough for people to say OK this is a good technique technique that was adopted is what's called polynomial classifiers we turn this into kernel methods but it's

Yann LeCun (Chief AI Scientist) 86:29.590

you know basically you sort of have a handcrafted feature extractor and then you train basically what amounts to a linear classifier on top of it that was going to come on practice in the seventies and certainly eighties but the idea that you could train a nonlinear system

Yann LeCun (Chief AI Scientist) 86:45.590

composed of multiple nonlinear steps using gradient descent the basic concept for this goes back to the kelly bison algorithm which is optimal control was mostly linear from nineteen sixty two and people in optimal control kind of word thinks about this you know in the sixties

Yann LeCun (Chief AI Scientist) 87:03.470

but nobody realized you could use this for machine learning to do pattern recognition or to do you know natural language processing that really only happened after you know the ramelight hinton williams paper in nineteen eighty five even though people had proposed the very same

Yann LeCun (Chief AI Scientist) 87:19.390

algorithm a few years before like pope werbus you don't propose you know what he called order derivatives which turns out to be backdrop but it's the same thing as the adjoint state method in optimal control so like those are ideas i mean the fact that an idea or a technique is

Yann LeCun (Chief AI Scientist) 87:34.190

reinvented multiple times in different fields and then only after the fact people say oh right it's actually the same thing and we knew about this before we didn't realize we could use this for it for this particular stuff right so all those like claims of plagiarism it's just

Yann LeCun (Chief AI Scientist) 87:49.960

it's just a complete misunderstanding of ideas

? (?) 87:56.440

OK what do you do when you're not thinking about AI

Yann LeCun (Chief AI Scientist) 88:03.840

have a whole bunch of hobbies that have very little time to actually partake in alexei aline so i go selling in the summer i like selling multi hall boats like tremor and catamarans i have a bunch of boats i like building flying contraptions

? (?) 88:31.440

so a modern da vinci i wouldn't i

Yann LeCun (Chief AI Scientist) 88:35.160

wouldn't call it airplanes because a lot of them don't look like airplanes but they don't fry OK i like the the sort of you know concrete creative act of that my dad was aerospace engineer and he mechanical engineer working in the aerospace industry and he was you know building

Yann LeCun (Chief AI Scientist) 88:54.670

airplanes as a hobby and like you know building his own radio control system and stuff like that and he got me and my brother into it my brother who works at google at google research in france in paris and and and that became kind of a family activity if you want so my brother

Yann LeCun (Chief AI Scientist) 89:12.990

and i still still do this but and then in the covid years i picked up astrophotography so i have a bunch of telescopes and pictures of the sky and i built electronics so since i was a teenager i was interested in music i was playing renaissance and baroque music and also some

Yann LeCun (Chief AI Scientist) 89:33.510

type of folk music playing wind instruments wind winds and but i was also into electronic music and my cousin who is such older than me was inspiring electronic musician so we had like analog synthesizers because i knew electronics i would like you know modify them for him and i

Yann LeCun (Chief AI Scientist) 89:54.430

was still in high school at the time and and now in my home i have a whole bunch of synthesizers and i build electronic musical instruments so so these are wind instruments you blow into them you know there's fingering and stuff but what they produce is control signals for a

Yann LeCun (Chief AI Scientist) 90:16.190

synthesizer

? (?) 90:17.950

it's cool very cool i've heard

? (?) 90:21.040

a lot of people in tech are into sailing like in yeah gotten that answer surprising amount i'm going to start trying to sail now OK

Yann LeCun (Chief AI Scientist) 90:29.040

so i tell you something about sailing it's very much like the war model story to be able to you know kind of control this elbow properly to make it go as fast as possible and everything you have to anticipate a lot of things you have to anticipate the motion of the waves like

Yann LeCun (Chief AI Scientist) 90:45.070

how the waves are going to affect your boat you know whether a gust of wind is going to come and have to you know start you know the woods going to start healing and things like that and you basically have to run CFD in your head because you have to figure out like the you know

Yann LeCun (Chief AI Scientist) 91:01.550

through dynamics you have to figure out like what is the flow of air around the around the around the sails and you know that if the angle of attack is too high it's going to be turbulent on the back and the the lift is going to be much lower so blah blah blah so like you know

Yann LeCun (Chief AI Scientist) 91:19.350

tuning sales is basically requires running cod in your head but at an abstract level you're not solving the you know navier stokes right we have really good intuitive so that's what i like about it like the whole thing that you have to build this mental you know predictive model

Yann LeCun (Chief AI Scientist) 91:36.550

of the world to be able to do a good

? (?) 91:39.390

job how many samples you need

Yann LeCun (Chief AI Scientist) 91:43.560

yeah probably a lot but but you know you get to run it on ian you know if you a few years of practice and yeah yeah

? (?) 91:55.190

OK your friends and you lived in the US for many decades already do you still feel french derek does that perspective shape your view of the of the wall of the american tech culture

Yann LeCun (Chief AI Scientist) 92:12.160

well inevitably yeah i mean you you can't completely escape your your upbringing and your culture so i mean i feel both french and american in the sense that you know i mean in the US for thirty seven years and in north america for thirty eight because i was in canada before or

Yann LeCun (Chief AI Scientist) 92:33.550

children grew up in the US and so from that point of view i'm american but i have a view certainly on you know on various aspects of science and society that probably are you know a consequence of growing up in france yeah absolutely and if you're french i mean france

? (?) 92:54.680

i'm curious i did not actually realize that you had a brother that also worked in tech i'm fast fascinated by this because yahshua bengio 's brother also works in tech and i always thought that he was the only serena venus williams situation in AI but you you too also have a

? (?) 93:12.230

brother so how many more AI research like like is it that common that it just runs in families

Yann LeCun (Chief AI Scientist) 93:17.430

i have no idea that's what i have a sister who you know is not in tech but she's also a professor my brother was a professor before he moved to google he he doesn't work on AI machine learning he's very careful not to he's a younger brother six years younger than me and he works

Yann LeCun (Chief AI Scientist) 93:39.880

on operations research and optimization essentially which now is actually also being invaded by machine learning

? (?) 93:52.400

yeah OK one more question so like if the world models work in twenty years from now what is the what is the dream like how what how does it look like how does i know like our lives will be

Yann LeCun (Chief AI Scientist) 94:10.470

total world domination OK it's a joke the the i i said this status because this is what linda starvalds used to say you say like what's your goal with linux and you say total i thought it was super funny and it actually succeeded i mean basically you know to first approximation

Yann LeCun (Chief AI Scientist) 94:37.630

every computer in the world runs linux there's only a few desktops that don't and a few iphones but you know everything else with linux so really like you know having you know pushing towards like a recipe for training and building intelligence systems perhaps all the way to

Yann LeCun (Chief AI Scientist) 94:56.910

human intelligence or more and and basically building AI systems that would you know help people and humanity more generally in their daily lives at all times amplifying human intelligence will be their boss right it's not like those things are going to dominate us because again

Yann LeCun (Chief AI Scientist) 95:16.350

it's not because something is intelligent that it wants to dominate those are two different things in humanity you know we are hardwired having to influence other people and sometimes it's through domination sometimes it's through prestige but the hardwired evolution to do this

Yann LeCun (Chief AI Scientist) 95:37.070

because we are a social species there's no reason we would build those kind of drive into into our intelligent systems and it's not like they're going to develop those kinds of drives by themselves so so yeah i'm quite optimistic

? (?) 95:54.280

me too so am i all right OK so we have final questions from the audience and so yeah let's start and if you were starting your AI career today what skills and research directions would you focus on

Yann LeCun (Chief AI Scientist) 96:12.400

i get a lot this question a lot from young students or parents of future students i mean i think you should learn things that have a long shelf life and you should learn things that help you learn to learn because technology is you know evolving so quickly that you all kind of

Yann LeCun (Chief AI Scientist) 96:33.270

you know the ability to learn really quickly and basically that can that is done by you know learning very busy so in the context of stem right science and technology engineering mathematics and i'm talking about humanities here this is although you should learn philosophy this

Yann LeCun (Chief AI Scientist) 96:58.310

is done by learning things i have a long shared life so the joke i say is that if you first of all you the things i have a long shelf life tend to not be computer science OK so here's a computer science professor you know are you against studying computer science don't count

? (?) 97:15.440

don't count to study

Yann LeCun (Chief AI Scientist) 97:19.520

and have a terrible coefficient to make which is i studied electrical engineering as an undergrad so i'm not a real computer scientist OK but what what we should do is learn kind of basic things in mathematics in modeling mathematics that can be connected with reality you tend

Yann LeCun (Chief AI Scientist) 97:37.190

to learn this kind of stuff in engineering in some schools that's linked with computer science but sort of you know electrical engineering mechanical engineering et cetera engineering disciplines you're all you're running in the US calculus one two three that gives you a good

Yann LeCun (Chief AI Scientist) 97:52.110

basis right computer science you don't you know you can get away with just calculus one that's not enough right you know we're learning you know probability theory and linear algebra you know all the stuff that are really kind of basic and then if you do network engineering

Yann LeCun (Chief AI Scientist) 98:09.680

things like i don't know control tiers or signal processing like all of those methods optimization you know all of those methods are really useful for things like AI and then you can you can basically learn similar things in physics because physics is all about like what should

Yann LeCun (Chief AI Scientist) 98:27.240

i represent about reality to be able to make predictive models right and that's really what intelligence is about so so i think you can learn most of what you need to learn also if you go through a physics a curriculum but obviously you need to learn enough computer science to

Yann LeCun (Chief AI Scientist) 98:45.190

kind of program and use computers and even though AI is going to help you be more efficient at programming you still need to know like how how to do this what

? (?) 98:56.150

do you think about vivec coding

Yann LeCun (Chief AI Scientist) 98:59.310

i mean it's cool it makes is going to cause a funny kind of thing where a lot of the code that will be written will be used only once because it's going to become so cheap to write code right you're going to ask your kind of AI assistant like you know produced this graph or like

Yann LeCun (Chief AI Scientist) 99:20.190

you know this research blah blah blah he's going to write a little piece of code to do this or maybe it's an applet that you need to play with for you know a little simulator and you can use it once and throw it away because so cheap to produce right so the idea that we're not

Yann LeCun (Chief AI Scientist) 99:34.710

going to need programmers anymore is false we're going to the cost of generating software you know has been going down continuously for decades and it's that's just the next step of the cars going down but it doesn't mean computers will be less useful they're going to be more

Yann LeCun (Chief AI Scientist) 99:53.030

useful

? (?) 99:56.930

OK one more question so what do you think about the connection between neuroscience and machine learning there are some ideas a lot of time that AI borrows from neuroscience and the other way right predictive coding for example do you think that it's useful to use ideas from but

Yann LeCun (Chief AI Scientist) 100:18.030

there's a lot of inspiration you can get from neuroscience from biology in general but neuroscience in particular i certainly was very influenced by classic work in neuroscience like you know hubble and riesel 's work on the architecture of the visual cortex is basically what

Yann LeCun (Chief AI Scientist) 100:34.190

led to convolutional nets right and you know i wasn't the first one to use those ideas in in artificial neural nets right there were people in the sixties trying to do this there were you know people in the eighties building locally connected networks with multiple layers they

Yann LeCun (Chief AI Scientist) 100:48.470

didn't have ways to train them with backdrop you know there was the the cognitron the neo cognitron from fukushima which had a lot of the ingredients just not proper learning algorithm and there was another kind of aspect of the cognitron which is that it was really meant to be

Yann LeCun (Chief AI Scientist) 101:06.350

a model of the visual cortex so it tried to reproduce every quirks of biology for example the fact that in the brain you don't have positive and negative weights you have positive and negative neurons so all the neurons all the synapses coming out of an inhibitor neurons neurons

Yann LeCun (Chief AI Scientist) 101:28.270

have negative weights OK and all the synapses coming out of non inhibitory neurons have positive weights so fukushima implemented this in his model right he implemented the fact that you know what neurons spike so we didn't have a spiking neuron model but but you cannot have a

Yann LeCun (Chief AI Scientist) 101:49.830

negative number of spikes and so so his function was basically rectification like a rail you except it had a saturation and then he knew from you know various works that there was some sort of normalization and he had to use this because otherwise there was no back props of the

Yann LeCun (Chief AI Scientist) 102:08.080

activation in this network would go haywire so we had to do like division normalization that turns out to actually be correspond to some theoretical models of the visual cortex that some of our colleagues at the center for neuroscience said NYU have been pushing like david heger

Yann LeCun (Chief AI Scientist) 102:25.230

and people like that so yeah i mean i think neuroscience is very a source of inspiration you know more more recently the sort of macro architecture of the brain in terms of you know perhaps the world model and planning and things like this like how how is that reproduced like

Yann LeCun (Chief AI Scientist) 102:45.550

why do we have a separate module in the brain for for for factual memory the hippocampus right and we see this now in certain neural net architectures that there is like a separate memory module right maybe that's a good idea i think we're going to have i think what's going to

Yann LeCun (Chief AI Scientist) 103:05.310

happen is that we're going to come up with sort of new AI neural net architectures deep learning architectures and a posterior you will discover that the characteristic that we implemented in them actually exist in the brain and in fact that's a lot of what's happening now in

Yann LeCun (Chief AI Scientist) 103:22.870

neuroscience which is that there is a lot of feedback now from AI to neuroscience where the best models of human perception are basically conventional nets today

? (?) 103:34.600

yeah OK

? (?) 103:37.440

do you have anything else that you want to add to say to the audience whatever

Yann LeCun (Chief AI Scientist) 103:43.200

you want i think we covered a lot of grounds i think you you know you want to be careful who you listen to so don't listen to AI scientists talking about economics OK so when some AI person or even a business person tells you AI is going to put everybody out of work talk to an

Yann LeCun (Chief AI Scientist) 104:07.030

economist basically none of them is saying anything anywhere close to this OK and you know the effect of technological revolutions on the labor market is something that a few people have devoted their career on doing none of them is predicting massive unemployment none of them

Yann LeCun (Chief AI Scientist) 104:30.650

is predicting that radiologists are going to be all unemployed you know et cetera right also realize that actually fielding practical applications of AI so that there are sufficiently reliable and everything is super difficult and it's very expensive and in previous waves of

Yann LeCun (Chief AI Scientist) 104:49.400

interest AI the techniques that people have put a big hope in turned out to be overly unwieldy and expensive except for a few applications so there was a big wave of interest in expert systems back in the nineteen eighties japan started a huge project called the fifth generation

Yann LeCun (Chief AI Scientist) 105:14.760

computer project which was like computers with CPU 's that were going to run lists and you know inference engines and stuff right and the hottest job in the late eighties was going to be knowledge engineer you were going to sit next to an expert and then the knowledge of the

Yann LeCun (Chief AI Scientist) 105:31.790

expert into rules and facts right and then the computer would be able to basically do what the expert was this was manual behavior clothing OK uh and it kind of worked but only for a few domains where where economically it made sense and it was doable at the level of reliability

Yann LeCun (Chief AI Scientist) 105:49.110

that was good enough but it was not a pass towards kind of human level intelligence the idea somehow like the delusion that people today have that the current AI mainstream you know fashion is going to take us to human intelligence has happened already three times during my

Yann LeCun (Chief AI Scientist) 106:12.990

career and probably five or six times before right you should you should see what people are saying about the perceptron right the new york times article people were saying oh we're going to have like super intelligent machines within ten years marvin minsky in the sixties says

Yann LeCun (Chief AI Scientist) 106:25.470

oh within ten years the best chess player in the world would be a computer it took a bit longer than that and you know and you know this you know this happened over and over again in nineteen fifty six or something when newell and simon produced the the general problem solver

Yann LeCun (Chief AI Scientist) 106:45.360

very modestly called the general problem solver OK what they what they thought was really cool they say OK the way we think is very simple we pose a problem there is a number of different solutions to that problem different proposals for solution a space of potential solutions

Yann LeCun (Chief AI Scientist) 107:03.880

like you know you do like traveling salesman right there was a number of you know factorial you know N factorial paths possible paths you just have to look for the one that is the best right and they say like every problem can be formulated this way essentially for a search for

Yann LeCun (Chief AI Scientist) 107:20.360

the best solution if you can formulate the problem as an objective by writing a program that checks whether it's a good solution or not or gives a rating to it and then you have a search algorithm that search through the space of possible solution for one that optimizes that

Yann LeCun (Chief AI Scientist) 107:39.070

score that you saw the AI OK now what they didn't know at the time is order complexity theory that basically every problem that is interesting is exponential or AP complete or whatever right and so oh we have to use heuristic programming you know kind of come up with heuristics

Yann LeCun (Chief AI Scientist) 107:55.510

for every new problem and basically you know there are general problem solver was not that general so like this idea somehow that the latest idea is going to take you to you know AGI or whatever you want to call it is very dangerous and a lot of very smart people fell into that

Yann LeCun (Chief AI Scientist) 108:14.110

trap many times over the last seven decades do

? (?) 108:17.270

you think that the field will ever figure out continual or incremental learning

Yann LeCun (Chief AI Scientist) 108:22.310

sure yeah that's sort of a technical problem well

? (?) 108:26.270

well i thought i thought catastrophic forgetting right because your weights that you trained so much money on get overwritten sure

Yann LeCun (Chief AI Scientist) 108:32.750

so you train just a little bit of it i mean we don't already do this with SSL right we train a foundation model like for video or something like vijay patu you know producers really good representations of video and then if you want to train the system for a particular task you

Yann LeCun (Chief AI Scientist) 108:46.990

train a small head on top of it and that head can be you know along continuously and even your word model can be trained continuously that's not an issue i don't see this as like a big a huge challenge frankly in fact raya heads to LPR simon and i and a few of our colleagues

Yann LeCun (Chief AI Scientist) 109:02.270

back in two thousand five two thousand six build a learning based navigation system for mobile robots that had this kind of idea so it was it was commercial net that was doing semantic segmentation from camera images and on the fly the top layers of that network would be adapted

Yann LeCun (Chief AI Scientist) 109:22.280

to the current environment so you do a good job and the labels came from short wrench uh traversability that were indicated by stereo vision essentially so yeah i mean you can do this it's particularly if you have multimodal yeah i don't see this as a big challenge

? (?) 109:46.180

it's been a pleasure to have you

Yann LeCun (Chief AI Scientist) 109:48.350

real pleasure to thank you so much thank you thank you