Do LLMs Understand? AI Pioneer Yann LeCun Spars with DeepMind’s Adam Brown. - part 9/15
2025-12-12_17-05 • 1h 15m 39s
Yann LeCun (Chief AI Scientist)
00:00.660
that's
it.
We're
going
to
write
a
program
that
does
this,
and
we're
going
to
call
it
the
general
problem
solver,
GPS,
1950
seven,
I
think.
Uh,
they
want
a
Turing
award
for
for
things
like
that,
and
It
was
It
was
great,
Yann LeCun (Chief AI Scientist)
00:13.420
but
then
they
didn't
realize
that
all
the
interesting
problems
actually
have
a
complexity
that
grows
exponentially
with
the
size
of
the
problem.
So,
in
fact,
you
can't
really
use
this
uh
technique
to
build
uh
intelligent
machines.
Yann LeCun (Chief AI Scientist)
00:26.000
It
can
be
a
component
of
it,
but
it's
really
not
not
the
thing.
So
Matthewsly,
for
it
was
on
black
came
up
with
a
perception
on
a
machine
that
could
learn.
And
he
said,
"If
we
can
train
a
machine,
then
it
can
become
infinitely
Yann LeCun (Chief AI Scientist)
00:36.240
smart."
And
so
within
10
years,
we'll
have
we
just
need
to
big,
you
know,
to
build
bigger
perceptrons,
right?
Not
realizing
that
you
need
to
train
multiple
layers
and
that
turned
out
to
be
uh
difficult
to
find
a
solution
for
this.
Yann LeCun (Chief AI Scientist)
00:49.360
Um,
then
in
the
1980s,
there
was
um
expert
systems.
Okay.
reasoning
is
is
fine.
Just
write
a
bunch
of
facts
and
a
bunch
of
rules
and
then
just
deduce
all
the
facts
from
the
original
facts
and
the
and
the
rules.
And
uh
Yann LeCun (Chief AI Scientist)
01:05.920
now
we
can
reduce
all
the
human
knowledge
into
into
this.
The
The
coolest
job
is
going
to
be
knowledge
engineer.
You're
going
to
sit
down
next
to
an
expert
and
then
write
down
all
the
rules
and
the
facts
and
turn
this
into
an
expert
system.
Yann LeCun (Chief AI Scientist)
01:19.200
And
you
know
everybody
was
excited
about
this
and
there
were
you
know
billions
that
were
invested
the
Japan
started
the
fifth
generation
computer
program
pro
uh
project
which
was
can
which
was
going
to
revolutionize
computer
science,
complete
Yann LeCun (Chief AI Scientist)
01:34.000
failure.
Okay?
It
created
an
industry.
It
was
useful
for
a
few
things,
but
basically
the
cost
of
reducing
human
knowledge
to
to
rules
uh
was
just
too
high
for
most
problems
and
so
the
whole
thing
Yann LeCun (Chief AI Scientist)
01:47.000
collapsed.
Then
there
was
neural
nets,
the
the
first
the
second
wave
of
neural
nets
1980s,
deep
you
know,
which
we
now
call
deep
learning.
a
lot
of
interest,
but
then
it
was
before
the
internet,
we
didn't
have
enough
data,
we
didn't
have
powerful
computers.
Yann LeCun (Chief AI Scientist)
02:01.280
And
now
we're
we're
going
through
the
same
cycle
again
and
we're
getting
fooled
again.
Janna Levin (Professor of Physics and Astronomy)
02:05.160
So
just
to
be
oh
Adam
please.
Adam Brown (Research Scientist)
02:07.240
In
in
technologies
every
dawn
has
before
it
false
dawns,
that
doesn't
mean
we'll
never
we'll
never
hit
the
dawn.
I
I
guess
I
would
like
um
Jan,
if
you
think
that
LLM's
are
going
to
saturate,
what
is
a
concrete
task
that
they
will
never
be
able
to
do?
Adam Brown (Research Scientist)
02:24.160
That
that
an
LLM
them
augmented
by,
you
know,
the
the
tools
we
give
it
today,
will
never
be
able
to
perform.
Yann LeCun (Chief AI Scientist)
02:32.360
Uh,
clear
up
the
dinner
table,
fill
up
the
dishwasher.
Yann LeCun (Chief AI Scientist)
02:37.200
Okay.
And
that's
easy
I
compared
to
I'm
skeptical.
That's
super
easy
compared
to
fixing
your
toilets.
Yeah.
Okay,
We
build
a
plumber,
right?
You're
never
going
to
have
a
plumber
with
LLM's.
You're
never
going
to
have
a
robot
driven
by
LLM's.
It
just
cannot
understand
the
real
Yann LeCun (Chief AI Scientist)
02:49.480
world.
It
just
Janna Levin (Professor of Physics and Astronomy)
02:50.080
can't.
So,
I
want
to
clarify
for
the
audience
that
you're
not
saying
that
machines
or
robots
won't
be
able
to
do
this.
That's
not
your
position.
You
Yann LeCun (Chief AI Scientist)
02:57.040
think
they
will.
They
will.
They
absolutely
But
just
will
not
by
this
algorithmic
approach
or
for
this
particular
approach
of
the
deep
learning
on
the
normal
Yann LeCun (Chief AI Scientist)
03:03.040
If
the
program
I'm
working
on
succeeds,
which
may
take
a
while
This
is
Jipa.
Am
I
Jipa
Jipa
Jipa
Jipa
and
and
you
know
all
the
things
world
models
and
things
that
go
with
it.
Yann LeCun (Chief AI Scientist)
03:13.240
If
it
succeeds
which
may
take
you
know
several
years
then
we
we
may
have
you
know
AI
system
there's
no
question
at
some
point
in
the
future
we
will
have
machines
that
are
smarter
than
humans
in
all
domains
that
you
know,
where
humans
have
abilities.
Yann LeCun (Chief AI Scientist)
03:27.480
There's
no
question
about
that.
They
will
happen,
okay?
You
probably
take
longer
than
you
know
some
of
the
people
in
Silicon
Valley
at
the
moment
are
saying
it
it
it
will.
Uh
Yann LeCun (Chief AI Scientist)
03:37.000
and
uh
and
it
it
will
not
be
LLMs,
it
will
not
be
generative
models
that
predict
discrete
tokens.
It
will
be
models
that
learn
abstract
representations
and
make
predictions
in
the
abstract
representations
and
can
reason
about
what
is
going
to
be
the
effect
of
me
taking
Yann LeCun (Chief AI Scientist)
03:53.080
this
action,
can
I
plan
a
sequence
of
actions
to
arrive
at
a
particular
goal.
Janna Levin (Professor of Physics and Astronomy)
03:57.200
You
call
the
self-supervised
learning.
Yann LeCun (Chief AI Scientist)
03:59.120
No,
so
self-supervised
learning
is
used
also
by
LLM's.
Self-supervised
learning
is
the
idea
that
you
train
a
system
not
for
a
particular
task
other
than
capturing
the
sort
of
underlying
structure
of
the
of
the
data
you
Yann LeCun (Chief AI Scientist)
04:14.120
you
show
it.
And
one
way
to
do
this
is
to
give
it
a
piece
of
of
data,
corrupt
it
in
some
way
by
removing
a
piece
of
it,
for
example,
masking
a
piece
of
it,
and
then
training
a
bit
more
on
that
to
predict
the
piece
that
is
missing.
So,
Yann LeCun (Chief AI Scientist)
04:30.760
LLMs
do
this,
right?
You
take
a
text,
you
remove
the
last
word,
and
you
train
the
LLMs
to
predict
the
the
word
that
is
missing.
You
have
other
types
of
language
models
that
actually
fill
up
multiple
words.
They
Yann LeCun (Chief AI Scientist)
04:42.080
turn
out
to
not
work
as
well
as
the
ones
that
just
predict
the
last
one.
Um,
at
least
for
certain
tasks.
Um,
you
can
do
this
with
video.
If
you
try
to
predict
at
the
pixel
level,
it
doesn't
work,
or
it
doesn't
work
very
well.
Um,
Yann LeCun (Chief AI Scientist)
04:55.000
My
colleagues
at
Meta
probably
boiled
a
couple
small
lakes
in
the
West
Coast
to,
you
know,
trying
to
make
this
work.
Um,
to
cool
the
GPUs.
Uh,
so
it
simply
doesn't
Autoscroll