It really makes me cringe listening to all that.
Thank you for that kind introduction, but I hate hearing about myself.
And just as a show, well, maybe if you could just applaud.
How many of you know who NVIDIA is?
And how many of you know what a GPU is?
Okay, good, I don't have to change my speech.
Ladies and gentlemen, President Rosenbaum, esteemed faculty members, distinguished guests, proud parents, and above all, the 2024 graduating class of Caltech.
This is a really happy day for you guys.
You got to look more excited. You know you're graduating from Caltech.
This is the school of the great Richard Feynman, Linus Pauling, and someone who's very influential to me and our industry, Carver Mead.
Yeah, this is a very big deal.
Today is a day of immense pride and joy.
It is a dream come true for all of you, but not just for you because your parents and families have made countless sacrifices to see you reach this milestone.
So let's take this moment and congratulate them, thank them, and let them know you love them.
You want to be super grateful today.
Your journey here is a testament of your character, determination, willingness to make sacrifices for your dreams, and you should be proud.
The ability to make sacrifices, endure pain and suffering, you will need these qualities in life.
First, both chief scientists of NVIDIA were from Caltech.
And one of the reasons why I'm giving this speech today is because I'm recruiting.
And so I want to tell you that NVIDIA is a really great company,
I'm a very nice boss,
universally loved,
come work at NVIDIA.
You and I share a passion for science and engineering,
and although we're separated by about 40 years,
we are both at the peaks of our career.
For all of you who have been paying attention to NVIDIA and myself,
you know what I mean.
It's just that in your case,
you'll have many, many more peaks to go.
I just hope that today is not my peak.
Not the peak.
And so I'm working as hard as ever
to make sure that I have many, many more peaks ahead.
Last year, I was honored to give the commencement address at Taiwan University,
and I shared several stories about NVIDIA's journey
and the lessons that we learned that might be valuable for graduates.
I have to admit that I don't love giving advice,
especially to other people's children.
And so my advice today will largely be disguised in some stories
that I've enjoyed,
and some life experiences that I've enjoyed.
I'm the longest running tech CEO in the world today, I believe.
Over the course of 31 years, I managed not to go out of business,
not get bored, and not get fired.
And so I have the great privilege of enjoying a lot of life's experiences,
starting from creating NVIDIA, from nothing to what it is today.
And so I spoke about the long road of creating CUDA,
a programming model.
The programming model that we dedicated over 20 years to invent,
and that is revolutionizing computing today.
I spoke about a very, quite public, canceled
Sega game console project we worked on,
and where intellectual honesty,
something that I know Richard Feynman cares
very deeply about and spoke quite often about,
where intellectual honesty and humility saved our company.
And how a retreat, a strategic retreat, was one of our best strategies.
All of these are counterintuitive lessons that I spoke about at the commencement.
But I encouraged the graduates to engage with AI,
the most consequential technology of our time.
And I'll speak a little bit more about that later,
but all of you know about AI.
It's hard not to be immersed in it and surrounded by it,
and a great deal of discussion about it.
And of course, I hope that all of you are using it and playing with it,
with surprising results and some magical,
some disappointing, and some surprising.
But you have to enjoy it, you have to engage it,
because it's advancing so quickly.
It is the only technology that I've known
that is advancing on multiple exponentials at the same time.
And so the technology is changing very, very quickly.
So I advised the students at the Taiwan University to run, don't walk,
and engage the AI revolution.
And yet, one year later, it's incredible how much it's changed.
And so today, what I wanted to do is share with you my perspective,
from my vantage point, of some of the important things that are happening
that you're graduating into.
And these are extraordinary things that are happening
that you should have an intuitive understanding for,
because it's going to matter to you,
it's going to matter to the industry.
And hopefully, you take advantage of the opportunity ahead of you.
The computer industry is transforming from its foundations,
literally from studs.
Everything is changing from studs on up.
And across every layer, and soon, every industry will also be transformed.
And the reason for that is quite obvious,
because computers today
are the single most important instrument of knowledge.
And it's foundational to every single industry and every field of science.
If we are transforming the computer so profoundly,
it will, of course, have implications in every industry.
And I'll talk about that in just a little bit.
And as you enter industry,
it's important you know what's happening.
Modern computing traces back to the IBM System 360.
That was the architecture manual that I learned from.
It's an architecture manual that you don't need to learn from.
A lot better documentation
and better descriptions of computers and architecture
has been presented since.
But the System 360 was incredibly important at its time,
and in fact, the basic ideas of the System 360, the architecture of it,
the principal ideas and architecture and strategy of the System 360
are still governing the computer industry today.
And it was introduced a year after my birth.
In the 80s, I was among the first generation of VLSI engineers
who learned to design chips
from Mead and Conway's landmark textbook.
And I'm not sure if it's still being taught here.
It should be in the introduction of VLSI systems.
Based on Carver Mead's pioneering work here at Caltech
on chip design methodologies and textbook that revolutionized IC design.
And it enabled our generation to design supergiant chips and ultimately the CPU.
The CPU led to exponential growth in computing.
The performance, the incredible technology advances,
that's called Moore's Law,
fueled the information technology revolution.
The industrial revolution that we are part of,
that my generation was part of,
saw the mass production of something the world had never seen before.
The mass production of something that was invisible, easy to copy,
the mass production of software.
And it led to a $3 trillion industry.
When I sat where you sat, the IT industry was minuscule.
And the concept that you could make money selling software was a fantasy.
And yet today, it's one of the most important commodities,
most important technologies
and product creations that our industry produces.
However, the limits of the NARD scaling, of transistor scaling,
and instruction level parallelism have slowed CPU performance.
And the slowed CPU performance gains is happening
at a time when computing demand continues to grow exponentially.
This exponentially growing gap
between demand of computing and the capabilities of computers,
if not addressed, computing energy consumption and cost, inflation,
would eventually stifle every industry.
We see very clear signs of computing inflation as we speak.
And after two decades of advancing NVIDIA's CUDA,
NVIDIA's accelerated computing offers a path forward.
That's the reason why I'm here.
Because finally, the industry realized
of the incredible effectiveness of accelerated computing at precisely the time
that we're witnessing computing inflation after several decades.
By offloading time-consuming algorithms to a GPU
that specializes in parallel processing,
we routinely achieve 10, 100, sometimes 1,000-fold speedups,
saving money, cost, and energy.
We now accelerate application domains from computer graphics,
ray tracing, of course, to gene sequencing, scientific computing,
astronomy, quantum circuit simulations,
SQL data processing, and even pandas, data science.
Accelerated computing has reached a tipping point.
That is our first great contribution to the computer industry,
our first great contribution to society, accelerated computing.
It now gives us a path forward for sustainable computing
where cost will continue to decline
as computing requirement continues to grow.
A hundred-fold, a hundred-fold of anything in time or cost or energy savings
that accelerated computing opened surely would trigger
a new development somewhere else.
We just didn't know what it was
until deep learning came to our consciousness.
A whole new world of computing emerged.
Jeff Hinton, Alex Krzyzewski, and Ilya Sutskever
used NVIDIA CUDA GPUs to train AlexNet
and shocked the computer vision community
by winning the 2012 ImageNet Challenge.
This was the big moment,
the big bang of deep learning,
a pivotal moment that marked the beginning of the AI revolution.
Our decisions after AlexNet transformed our company
is something that's worth taking note of.
Our decisions after AlexNet transformed our company and likely everything else.
We saw the potential of deep learning and believed,
just believed through principle thinking,
believed through our own analysis of the scalability of deep learning.
We believed the approach could learn other valuable functions.
That maybe deep learning is a universal function learner
and how many problems are difficult
or impossible to express using fundamental first principles.
And so when we saw this,
when we saw this, we thought
this is a technology we really have to pay attention to
because its limits are potentially only limited by model and data scale.
However, there were challenges at the time.
This is 2012, shortly after 2012.
How could we explore the limits of deep learning
without having to build these massive GPU clusters?
At the time we were a rather small company,
building these massive GPU clusters could cost
hundreds and hundreds of millions of dollars.
And if we didn't though,
there was no assurance that it would be effective if we scaled.
However, no one knew how far deep learning could scale.
And if we didn't build it, we'd never know.
This is one of those, if you build it, will they come?
Our logic is if we don't build it, they can't come.
And so we dedicated ourselves
based on our first principled beliefs and our analysis.
And we got ourselves to the point
where we believed this was going to be so effective
and when the company believes something, we should go act on it.
So we dove deep into deep learning,
and over the next decade, systematically reinvented everything.
We reinvented every computing layer, starting with the GPU itself.
The invention of the modern GPU,
which is very different than the GPU of the past
that we invented in the first place,
and we went on to invent just about every other aspect of computing,
the interconnects, the systems,
the networking, and of course, software.
We invested billions.
We invested billions into the unknown.
Thousands of engineers for a decade
worked on deep learning and advancing and scaling deep learning
without really knowing how far we could really take the technology.
We invested billions.
And we designed and built supercomputers
to explore the limits of deep learning and AI.
And in 2016, we announced DGX-1,
our first AI supercomputer,
and I delivered the first one to a startup in San Francisco,
a startup nobody knew anything about,
a group of friends of mine who were working on artificial intelligence,
a company called OpenAI.
In 2022, 10 years after AlexNet,
and about a million-fold increase in computing later,
a million-fold.
If you could just imagine, what would it be like if your laptop
was a million-fold more capable?
A million-fold later, OpenAI launched ChatGPT,
and AI went mainstream.
During this decade, NVIDIA transformed ourselves from a graphics company
that many of you probably first knew us as, that builds GPUs,
to now an AI company that builds massive data center
scale supercomputers.
We transformed our company completely.
We also transformed computing completely.
The fundamental way of doing computing today has been radically changed.
The computing stack now uses GPU
to process large language models
that are trained on supercomputers
rather than CPUs
that are processing instructions written by programmers.
We are now creating software that no humans can write.
We are now creating software that does things that no humans can
imagine, even just 10 years ago.
Computers are now intention-driven rather than instruction-driven.
Tell a computer what you want, and it will figure out how.
And like humans, AI applications will understand the mission,
reason, plan,
and orchestrate a team of large language models to perform tasks.
Future applications will do and perform very similar to the way we do things,
assemble teams of experts, use tools,
reason and plan, and execute our mission.
Software and what software can do has been completely changed.
Even our industry, as it's being changed and transformed,
created yet another industry,
an industry the world's never seen before.
An industry is forming right in front of our eyes.
AI's input and output are tokens.
For all the engineers in the room, you know what I mean.
These are floating-point numbers that embed intelligence.
Companies are now building
a new type of data center that didn't exist before that specialize
in producing intelligence tokens.
Essentially, AI factories.
Like AC generators that Nikola Tesla invented
of the past industrial revolution,
we now have AI token generators,
and they will be the factories of a new industrial revolution.
There's large industries producing energy, electricity.
We now have a large industry producing something invisible called software.
In the future, in the very near future,
we'll have industries that are producing, manufacturing
intelligence tokens, AI generators.
A new computing model has emerged,
and a new industry has emerged,
all because we reasoned from first principles,
formed our belief about the future, and we acted on them.
The next wave of AI is robotics,
where AI, in addition to a language model,
also has a physical world model.
We work with hundreds of companies building robots,
robotic vehicles, pick-and-place arms, humanoid robots,
and even entire gigantic warehouses that are robotic.
But unlike our AI factory strategy and our experience there,
which was really formed through reasoning and deliberate action,
our robotics journey resulted from a series of setbacks.
As you know, NVIDIA invented the GPU.
This was before we invented AI factories.
Our first great contribution to the computer industry was
reinventing computer graphics through programmable shaders.
We invented the GPU and programmable shading in 2000.
We wanted to integrate GPUs into every computer,
and so we started to combine our GPUs with motherboard chips,
and we launched a fabulous integrated graphics chip
at the time for AMD CPUs.
Our chipset business was an instant success.
I think it went from zero to a billion dollars practically overnight.
But then all of a sudden, AMD wanted to
control all of the technology in the PC,
and we wanted to stay independent,
so they purchased ATI and no longer needed us.
We turned to Intel.
That probably wasn't a great idea,
but we turned to Intel
and negotiated a license to connect to Intel CPUs.
Apple was excited by what we were building
and asked us to work on a new computer with them,
which became the first MacBook Air.
Well, Intel saw what happened and decided
they didn't want us to do that anymore,
and so they terminated our agreement.
Well, we pivoted again, and this time we went and licensed ARM,
and we built a low-power SoC, a mobile SoC,
the world's first SoC that was essentially a computer,
a full operating computer, and it was incredible.
Our chip excited Google,
and they asked us to work on a new device,
which turned out to have been the Android mobile device.
Well, Qualcomm decided they didn't want us to do that,
and so they didn't want us to connect to their modems,
and it's hard to build a mobile device without being connected to a modem.
And there were no other LTE modem companies,
so we had to exit the mobile device market.
Well, this happened practically on a year rhythm,
and we would build something,
it would be incredibly successful,
generate enormous amounts of excitement,
and then one year later,
we were kicked out of those markets.
Well, with no more markets to turn to,
we decided to build something
where we are sure there are no customers,
because one of the things you can definitely guarantee
is where there are no customers,
there are also no competitors,
and nobody cares about you.
And so we chose a market with no customers, a $0 billion market,
and it was robotics.
We built the world's first robotics computer
processing an algorithm nobody understood
at the time called deep learning.
This is over 10 years ago now.
Ten years later, I can't be happier
with what we've built
and the opportunity to create the next wave of AI.
More importantly,
we developed agility and a culture of resilience.
One setback after another,
we shook it off and skated to the next opportunity.
Each time, we gained skills and strengthened our character.
We strengthened our corporate character.
Our company is really hard to distract and really hard to discourage,
and no setback
that comes our way doesn't look like an opportunity these days.
Ironically, the robotics computer that we built today
doesn't even need graphics,
which is why our journey started in the first place.
So where we are today
tells us something and teaches us something.
The world is uncertain, as Richard Feynman would say,
and the world can be unfair and deal you with tough cards.
Swiftly, shake it off.
You've apparently been paying too much attention to your books.
Swiftly, shake it off.
Come on, that's pretty clever.
I made myself laugh.
There's another opportunity out there, or create one.
Let me tell you one more story.
I used to work from one of our international sites
for one month each summer.
When our kids were in their teens, we spent a summer in Japan.
For a weekend, we visited Kyoto and the Silver Temple.
If you haven't had a chance to go, you must.
It's renowned for its exquisite moss garden.
The day we visited was quintessential Kyoto summer day,
suffocatingly hot and humid, sticky.
Heat is radiating from the ground.
The air was thick, still.
Along with the other tourists, we wandered through the meticulously groomed moss garden.
And I noticed the lone gardener.
Now, remember, the moss garden,
this is the Silver Temple,
the moss garden is gigantic.
It's about the size of this courtyard.
And it has the collection, the largest collection of
just about apparently every species of moss in the world.
And it's just exquisitely maintained.
I noticed the lone gardener squatting,
carefully picking at the moss with a bamboo tweezer
and putting it in the bamboo basket.
And you have to, it's a bamboo tweezer,
you know, and it's just this one gardener.
And the basket looked empty.
Well, for a moment there, I thought he was picking imaginary moss
into a pile of imaginary dead moss.
And so I walked up to him
and I said, what are you doing?
And in his English, he said, I'm picking dead moss.
I'm taking care of my garden.
And I said, but your garden is so big.
And he responded, I have cared for my garden for 25 years.
I have plenty of time.
Well, that was one of the most profound learnings in my life.
And it really taught me something.
This gardener has dedicated himself to his craft and doing his life's work.
And when you do that, you have plenty of time.
I begin each morning,
I do every single morning exactly the same way,
I begin each morning
by doing my highest priority work first.
I have a very clear priority list
and I start from the highest priority work first.
Before I even get to work, my day is already a success.
I've already completed my most important work
and can dedicate my day to helping others.
And when people apologize for interrupting me,
I always say I have plenty of time and I do.
Graduates of the class of 2024,
I can hardly imagine anyone more prepared for the future than you.
You dedicated yourself,
you worked hard,
you earned a world-class education
from one of the most prestigious schools in the world.
And as you commence into the next stage,
take my learnings
and hopefully they'll help you along the way.
I hope you believe in something,
something unconventional,
something unexplored,
but let it be informed and let it be reasoned.
Then dedicate yourself to making it happen.
You may find your GPU,
you may find your CUDA,
you may find your generative AI,
you may find your NVIDIA.
I hope you will see setbacks as new opportunities.
Your pain and suffering will strengthen your character,
your resilience and agility,
and they are the ultimate superpowers.
Of all of the things that I value most about my abilities,
intelligence is not top of that list.
My ability to endure pain and suffering,
my ability to work on something for a very, very long period of time,
my ability to handle setbacks and see the opportunity just around the corner
I consider to be my superpowers,
and I hope they're yours.
And I hope you find a craft.
I hope you find a craft.
It's not important to decide on day one,
it's not even important to decide any time soon,
but I hope you do find a craft,
that you want to dedicate your lifetime to perfecting,
to hone the skills of, and let it be your life's work.
And then lastly, prioritize your life.
There's so many things going on,
there's so many things to do,
but prioritize your life
and you will have plenty of time to do the important things.
Congratulations, class of 2024, go get them.
