Hardware Architect Answers Microchip Questions

I'm Christian Jacobi,

chief technology officer for systems design at IBM.

Let's answer your questions from the internet.

This is Microchip Support.

[upbeat music]

From the Explain Like I'm Five subreddit.

How are microchips programmed to know

what is a one and zero?

So let's step back and talk about

why zeros and ones are so important in computing.

You can really encode all sorts of information

as a long series of zeros and ones.

Take the ASCII character set, for example.

It enumerates all the characters, the letters,

special signs in a long list,

and then it assigns a combination

of zeros and ones to represent each of these letters.

And then you can take a whole book, for example,

and just string all the letters, and characters,

and spaces, and commas, and exclamation marks

to create a long sequence of zeros and ones.

In a computer chip,

a zero is usually represented by no voltage,

and a one is represented by some voltage,

like one volt or 1.5 volts.

Transistors in a computer chip can then modify the signals

by either switching a transistor on or off

and performing certain computations.

You can build specific circuits using transistors,

like adders or multipliers,

but you can also make it programmable

so that you get software to run

and perform ever more complex operations

on these strings of data

that you're sending into the computer chips.

RolfCopter4 asks,

Just how on earth does a transistor physically work?

Well, the easiest way to think of it is like a valve.

You have an input and an output

and a valve handle.

In electronics with a transistor,

we call the input and output source and drain,

and the handle is called a gate.

An electrical signal connected to the gate

opens up the channel between the source and the drain,

or it keeps it closed so that no electricity can flow.

On a modern chip like this,

there are billions of transistors,

and of course they don't switch

like very slowly like a handle,

but they can switch billions of times per second.

Eatbeefnow, Why there are only few chip makers

in the world?

Well, the modern chips are designed

in very complicated processes,

like we're down to five, four, two nanometer design points,

and the manufacturing is extremely complicated.

Building fabs that can manufacture chips like that

is extremely costly.

And driving the development work

to get to the next technology node

from say three to two, and from two to 1.4

is an extremely costly undertaking as well.

So that's why we've seen a big consolidation

over the last 10 or 15 years,

and we only have a few companies

who can really afford to do the development work

and build those fabs.

So some of the big manufacturers today

are TSMC in Taiwan, Samsung in Korea,

and Intel here in the US.

All these companies are also building fabs in the US

and other places in the world,

but that's where they originate.

Ventynine asks, Why do computers get slow with time?

Let me bust that myth a little bit.

When you own a computer over a period of time,

you're loading more and more programs onto the computer,

you are getting firmware and software updates,

you're loading more and more data on it.

So it's not that the hardware gets slower,

you're just asking the hardware to do more

because you're kind of accumulating

a lot of junk on the device.

It's not like the chips wear out

and the hardware gets slower.

It's just you're asking more of it.

From hachface, Stupid question,

why do they need so many new data centers anyway?

Not a stupid question.

With what's going on in AI over the last few years,

we've really seen an explosion

in new data center construction,

and so let's just step back.

AI has had some really big breakthroughs

over the last three, four, five years,

and it's really driving worldwide productivity

for knowledge workers.

Now, we spent trillions and trillions of dollars in wages

for knowledge workers on the globe,

and if we can make knowledge workers more productive

by only a few percent, that is a massive market,

a hugely valuable market,

many trillions of dollars worth.

And so you see a lot of companies building data centers

to capture their share of that market.

Now, these data centers are really complex.

They need enormous power supplies

because they get filled with computers

for storage, general purpose computing,

and then of course a lot of GPUs for all the AI processing.

So while these data centers

are really huge infrastructure projects,

with like big buildings

and power supplies and cooling,

inside of the data centers

ultimately there are millions of chips.

Memory chips, storage chips,

general purpose processors,

and of course lots of GPUs for the AI processing.

Cincilator is asking, What are all those billions

of transistors in my CPU actually doing?

So transistors are really microscopic.

A modern day transistor has only a few nanometers of size.

A human hair is 100,000 nanometers in width.

So when we are talking about five nanometers,

that's like a tiny, tiny, tiny fraction

of the width of a human hair.

These billions of transistors,

each one of them is a tiny switch,

and we can form gates, and gates or gates that combine

two or three or four signals

and compute the and of all these signals,

or the or of all these signals, right?

All four of them, one, then an output is one,

or is one of them not a one,

then the output would be zero of the and gate.

Then we take these gates

and form more and more complex circuits.

We can build small adders, we can build multipliers,

we can perform those computations in a loop.

We can then add program function

so that you can actually program the chips

and perform ever more complex computations.

And so because we're putting all of these circuits

and multiple cores and a lot of memory on these chips,

it adds up to billions of transistors.

Minoshi asks, How can chips have billions of transistors

but have very few external wires connecting them?

It really matters how much data you need

to get in and out of the chip

versus how much computation you perform on the chip

with each piece of data.

Take this chip for example here.

These are all the connections on the backside.

They are combining the power supplies for the chip

as well as the input and output signals,

and then there are billions of transistors

and a lot of memory on this chip

to perform the actual computations.

DickheadNL is asking, Why do computer chips warm up?

The computer chips consist of transistors,

and every time they do a switch,

there's a tiny current flowing through the transistor

and the metal stack, and when that current flows,

a few electrons get moved around

and they push against the atomic structure of the metal,

and that creates friction

almost as if your hands are rubbing together.

That friction is causing the heat in the chips.

From the Ask Science subreddit,

If transistors are so small like a few atoms,

then how do we build them and put all of them on a CPU?

We start in the manufacturing process with a blank wafer,

and then we put some photo resistive material on the wafer.

We coat the whole wafer with that.

And then a mask that has all of the fine structures

of the design is used to shine a light

on the photo resist,

and then we etch out the areas

that have been blocked from the light,

and we can deposit metals, or we can dope the silicon

and create the semiconducting properties.

And then this happens in many, many, many layers.

First, we build what's called the front end,

which are the transistors

using repeated steps of photo resist,

shine light on it, etch it, deposit,

and then after the transistors

are done with multiple layers,

we then put the metal stack on top

which connects all of the transistors.

Nowadays with the fine structures that we have,

the two nanometer transistors,

it actually matters what kind of light

we are using for that imaging.

We're today using extreme ultraviolet light

because the wavelength of the light itself

has to be small enough to even be able

to show the fine structures that we need on these chips.

The machines that do all that work are massive,

like the size of an entire room

because they need to super precisely position the wafer.

They need to position the mask.

They need to have the laser light position,

and all of that needs to be like really in lockstep

to be able to create these super fine structures

of nanometer size.

Then there are super fine machines

that can cut the wafer into individual chips,

and we call that dicing.

And so you'll get these chips,

and then these individual chips

are put on what we call a module.

That's the little green board with two chips,

and then there's a metal stack inside here

that interconnects the two chips on this module,

as well as connects them to the underside

where we have the pins

that are driving the inputs and outputs of this chip.

A Reddit user asks,

How was the first computer chip created

with no computers to create it?

Well, really the first computers

were designed by hand on a piece of paper.

The circuits were drawn out on a piece of paper,

and then people would connect the different components

with little wire that they would solder

to the different components.

I myself, when I was at Saarland University in Germany,

was in a computer class where we had

what were called wire rep boards.

You would put components

into the wire rep board on one side,

and you would connect little wires on the backside

to interconnect all these components.

And we build a small calculator

using that primitive technique.

But in the 70s, whole computers were built

with these wire rep boards.

Nowadays, of course, we have very powerful computers,

and we can use these powerful computers

to build ever more powerful computers.

We're using huge computer farms, for example,

to validate the functional correctness of chips,

or to optimize the physical implementation.

InternalGoal955 is asking, AI conquered software coding,

and hardware design is next.

How do we prepare for inevitable displacement?

Well, I don't really look at it that way.

I think the word conquered

is really strong here, too strong.

AI tools are really powerful tools

that make us engineers more productive.

That's true in software engineering.

That's also true in chip development

and hardware engineering.

But it's another set of powerful tools

that we're building here that makes us better

and allows us to build better chips going forward.

I don't think it's gonna displace us.

It's gonna make us more productive

and enable us to build better chips.

ExileNorth asks, Why is silicon so important

in the manufacturing of computer chips?

Is there any viable alternative?

If not, why?

So modern manufacturing processes for semiconductors,

in particular computer chips,

cell phone chips, et cetera, are based on silicon.

That is the technology that has evolved

very far in terms of how many transistors

we can put on a chip,

how power efficient these chips can be,

how we can manufacture them in a very reliable way.

There are different elements in the periodic system

that can be used as semiconductors.

Silicon is one of them, germanium is another one.

But for the most powerful computer chips,

we're really dependent on silicon.

A semiconductor is a material

that doesn't conduct electricity like metal does,

but that can be configured to sometimes conduct

and sometimes not conduct.

That's the word semi.

So you can build a transistor with the gate,

and depending on what signal you put to the gate,

the semiconductor is either conducting or not conducting.

That is the fundamental building block for modern chips.

Rodabi asks, What advancements are made every year

that allow us to make faster processors?

A whole slew of things

across the whole stack of chip development.

The silicon node that's at the base,

like is it a five nanometer,

a four, three, two nanometer chip?

That changes all the time.

Then micro architects like myself,

we're inventing new ways to connect all these transistors

and build faster processors

at the micro architecture level.

They're figuring out how to make memory faster,

how to make storage faster, how to make network faster.

And in the combination of all those things,

computers are getting faster, faster and faster.

A micro architect is one discipline

in the broad field of computer engineering.

A micro architect is somebody

who basically lays out the big picture architecture

of the chip before it then gets built

into the different components and subunits

that end up making up the billions of transistors.

Dudewiththebling asks, Theoretically,

how small can a microchip be fabricated?

If you go back to computers from the 1930s and 40s,

they were built using magnetic relays or vacuum tubes.

Then in the 50s and 60s,

we developed integrated circuits

with the transistors on silicon chips, for example.

In the span of my career over the last 25 or so years,

we've moved from over 100 nanometer transistors

to five and two nanometer transistors nowadays.

There's no really strict limit

on how far we can continue to drive this,

but there's research going on here, right?

Nobody knows exactly how we'll build these chips

in 10 years or 15 years

because there's gonna be some scientific breakthroughs.

But let me tell you, 15 years ago, people didn't know

how we would manufacture the chips

that we have today with two nanometers.

That was an unknown.

So I believe we'll see the innovation continue

and research breakthroughs enable us

to continue to shrink the transistors,

and therefore add more and more transistors

on each of those chips.

So we are now at two nanometers,

and we're entering really the research

for the sub one nanometer timeframe.

We're calling that the Angstrom age,

and we're really now talking about transistors

of the size of just a few atoms.

R2002 asks, Semiconductor super cycle,

are we peaking or just starting?

Crash coming?

Well, who knows?

As we've talked about,

we're building massive new data centers,

and that is driving a lot of demand,

and it's really hard to build additional supply

for chip manufacturing just because these fabs

are so enormously complex and expensive.

So what we're seeing is a demand surge

from the new data centers,

and a bit of a supply crunch

because it's hard to build more manufacturing fabs.

How that plays out over the next few years

is anybody's guess.

Microchips have always gone in cycles.

Memory costs, for example,

has always gone up for a few years,

gone down, gone up again.

Right now, we're in what we call a super cycle.

With all the construction of new data centers,

there's so much demand for microchips,

memory, processors, GPUs,

and it's really hard to scale up

the manufacturing capabilities

because these fabs are so incredibly expensive,

that we're really seeing a surge in demand

driving the current cost of the microchips up.

Are we peaking? Are we crashing?

That's really anybody's guess.

I personally believe AI is such a transformative technology

that this cycle is gonna continue for a while.

DoomCrystal asks,

If we can't put any more transistors on a microchip

because the transistors are physically too small,

why don't we just make bigger microchips?

There's physical limits to how big we can make chips,

but then also there's commercial limits.

The bigger the chip, of course,

the more expensive it is.

But let's talk about the physical limits.

When manufacturing chips, we're using masks

to create the fine structures on the silicon wafer,

and these masks can only be produced in a certain size,

and so building chips above 750 or 780 square millimeters,

it's really hard,

and those chips are already very large

and therefore expensive.

Aiseadai is asking, What is the difference

between a GPU and CPU?

So let's step back.

There's many different types of chips.

There's memory chips, there's chips in a camera

that recognized the light

and turned the light into electrical signals, et cetera.

A CPU is a historically very versatile type of microchip

that is programmable and can execute all kinds of software.

That's really the heart of your laptop, for example,

or the heart of a traditional server computer.

GPUs are a different specialized kind of chip.

They came about maybe 20 so years ago,

and really were designed for graphics

used, for example, in either gaming

or in applications like computer aided design.

It turns out that the capabilities

that you have in GPUs foremost,

like real, strong high performance

computing capabilities

are also very relevant to AI processing.

And so the modern AI models

have actually been kind of built around the GPUs

because the math at a certain level

is similar to the kinds of math

that you do in graphics processing.

Prgmmr7 asks, Could someone explain

all the different types of chip design engineers

and the differences?

Well, I don't think I can explain all the different types,

but I can give you a good taste of it.

It starts with the people who develop the silicon process,

like how the silicon node

and the chip manufacturing works,

and then we have the engineers who design the chips.

Starts with a micro architect

who sort of lays out the big picture

of how the chip should work.

Then logic design engineers

implement the different functions in the chip,

the floating point units and the caches, for example.

Verification engineers make sure

that the logic design is functionally correct

and produces the correct results

when it computes on the data.

Physical design engineers take the logic design

and turn it into what we call a layout.

It's really like where do which transistors go?

Which function goes where?

How is it all interconnected using the metal stack?

And then as the chip gets manufactured,

you have all sorts of engineers and disciplines

to actually put a system around the chip.

So take this AI accelerator card.

Somebody designs the card.

Somebody designs the module on which the chip sits.

Somebody puts it all together and validates it,

and you have design for test engineers

who make sure that the chip

and the card works from manufacturing.

So you have all these disciplines

that bring it all together

and make sure that we have

functioning computers in the end.

Pyros_it asks, What were the tech leaps

that make computers now

so much faster than the ones in the 1990s?

Really, computer engineering has so many facets

and everything gets better all the time.

So it's faster transistors, smaller silicon nodes.

It's better designed in the processes themselves.

It's faster memory, faster network, faster storage.

Everything gets better.

If you kept one thing the same as it was in the 90s,

your computers today would still run very slow.

So it really takes all of it

to come together in a full system design

to create these breakthroughs.

PuddingComplete3081 asks,

Why does Moore's Law keep ending every decade

while computing power somehow keeps exploding anyway?

Moore's law was postulated not really as a law,

but more as an observation that about every two years,

we can double the number of transistors

that we can put on a chip.

That law is still around,

I mean, it still kind of works,

despite it has slowed down a little bit, right?

We're not doubling every two years,

but we can continue to grow the numbers

of transistors you can put on a chip.

What really has broken down is Dennard scaling.

Dennard scaling was a rule

that you can make transistors

smaller and smaller, put more of them on the chip,

and because of the transistors getting smaller,

they end up consuming the same amount of power

than the less transistors in the prior generation.

That scaling has really ended,

and as we're putting more and more transistors on,

it's really hard to stay in the power budget

for the chips that we're designing.

And so that's why you're seeing, for example,

processors consuming more power now

than they did 10, 15 years ago.

So with chip design now,

one of the most challenging aspects

is how do we manage the power consumption of the chip?

With the Dennard scaling no longer working,

as we put more and more transistors into a chip,

they consume more and more power,

and so there's a few key challenges here.

First to get the power into the chip,

and then that power creates heat,

and so we need to extract the heat,

and that's why you see fans in your computers.

But that's also when you look at big data centers,

you see massive power lines go into the data centers,

and then you see cooling towers, for example.

They use a lot of water to cool the air in the data center,

or to even bring cold water directly to the chips

to cool the chips with water.

Bons4y is asking, How are microchips made

with no imperfections?

I'll tell you the truth, when you're designing a chip

with billions of transistors,

there will be imperfections.

And we're designing to deal

with the imperfections into the chip design.

So for example, when you're designing a memory element,

you are not just designing the say one megabyte of memory,

you're designing maybe 10% more.

Then you have switches inside

where you can block out a bad memory cell

and use a spare cell that we have put into the chip.

Or think of some strange numbers of cores on a chip,

like you could have a chip with 28 cores, for example.

Well, typically what you would find

is there's actually 30 cores on the chip,

and then we look at which of these cores

are actually working,

and if only 28 of them are working,

we can sell that as a 28 core chip.

If only 16 are working,

you could sell it as a 16 core chip.

So we just need to prepare for that,

have redundancy built in,

and then structure the offerings

so that we can also sell partial good chips.

United_Nobody_2532 thinks,

Putting chips in people's brains would be great.

Well, let's separate what's actually happening today

versus what might happen in the future

versus some science fiction.

We've put chips into the human body for decades already.

Think of a pacemaker.

The pacemaker is a microchip.

It measures the electric signals in your heart

and it recognizes something that is not working right,

and it can send a pulse

to make the heart, you know, beat.

Modern pacemakers also contain memory, and take traces,

and are sort of like a ECG inside your body

that can be read out at a doctor's office.

We have these kinds of things.

We have hearing aids.

There's already research happening, for example,

to have artificial eyesight where a camera is connected,

the microchips in the camera can be connected

into the visual cortex of the brain.

So we're seeing a lot of these things

where I'll just say loosely, we can mitigate disabilities,

or we could have situations where like, you know,

a patient has a stroke and a chip could be used to repair

certain sections of a damaged brain, for example.

That already is happening,

and a lot of research is happening in that space as well.

Where it gets a bit more complex and controversial

is when it comes to actually enhancing

the capabilities of the brain.

To me, the brain is a finely tuned instrument

that has emotion, and intuition,

and experience, and knowledge,

and it makes us think, it makes us be innovative

and makes us human.

And I don't know whether, you know,

putting an additional chip

that could overload the brain with all the information

that's out on the internet would actually help or hurt.

It might just overload the brain,

besides all the ethical concerns it would create.

A Reddit user asks,

Why does making chips require clean facility?

Modern transistors are nanometers in size.

A dust speck is thousand times that.

Imagine as you're producing the chip

that you have a dust speck settle on the wafer

and blocks out thousands of transistors.

Well, then the chip won't be able to work.

That's why chip manufacturing facilities

are super, super clean room

so that you don't get the contaminations

onto the chips that you're producing.

DataNurse47 asks, Those who develop chips,

what was your career path like?

Well, like in any industry,

there can be many different career paths.

Mine started as a computer science student

at Saarland University in Germany,

and then I joined the IBM Development Lab

in Boeblingen in Germany,

and I kind of learned chip design as part of my job.

I then had an opportunity to move to New York

and develop next generation mainframe chips,

and from there I kind of grew

and went through different aspects of different chips.

I designed processor cores, I designed caches,

I designed IO circuits.

I kind of moved around all sorts of different areas.

And then as my responsibility,

and frankly, my experience grew,

I ended up in my current role as CTO.

So I started as a computer scientist.

Many engineers start

with an electrical engineering background.

As a computer scientist, I was more thinking

in terms of how programming works,

and then I learned the electrical engineering part

as part of doing my job.

So those are all the questions for today.

Thanks for watching.

[upbeat music]