I'm Mark Cerney,

Lead System Architect, at PlayStation 5,

and I'm here to talk about what we made,

and how we made it.

[booming electro music]

[character gasps]

My background is as a game developer,

I worked on anything from Marble Madness,

in the golden age of the arcade,

to Crash Bandicoot on the first PlayStation,

to Marvel's Spider-Man on PlayStation 5.

40 years of development. My God.

Okay. Moving on.

As Lead System Architect of the PlayStation consoles,

I focus on the chips on the motherboard,

and the software systems that directly use them.

It's very technical work,

but the ultimate users of these systems,

are developers trying to make games.

So it's great to have worked with a lot of teams,

over the years,

and understand a bit about what helps them,

and what just gets in their way.

[character grunts] [sword swooshes]

The first step in developing any console,

is a brainstorming session.

The hardware team has a list,

all the features that couldn't make it

into the previous console,

as well as of course a number of new ideas,

but there's also a longer list.

And that longer list, is all of the things,

that the game development community would like to see.

[upbeat electro music]

The SSD.

We've been getting requests for an SSD,

all the way back to PlayStation 4.

In particular, Tim Sweeney,

who is the visionary founder of Epic Games,

he said, hard drives were holding the industry back.

And he didn't say hard drives though,

he said, rusty, spinning media.

The game developers wanted to get away from hard drives,

because that's where all the game data is stored.

I mean, whatever it is,

character models, animation,

music, movies,

it's slow to get to,

and if you need seeks,

meaning you have to load a lot of little pieces of data,

from here and there,

it's particularly slow to get to.

If it takes 30 seconds or a minute to load into a level

that's just lost time.

So-called fast travel,

that's still taking 15 seconds or 30 seconds,

to get from one side of the map to the other side.

A bit of this is less obvious, it's filler.

It's that elevator ride, that hides the load.

It's that unskippable movie,

it's that narrow passageway that the player character

has to slowly ease their way through.

Now. I've been there myself,

as a designer on some of these projects,

where the map had to be just,

these twisty little passageways,

whose sole purpose,

was to hide the loads off of the media.

On PlayStation 4,

fast travel can take anything, depending on the game,

from 15 seconds to a minute.

On PlayStation 5, much faster,

anything from a fraction of a second to a few seconds.

[characters yelling] [rift crashing]

That means no more subway rides in Marvel's Spider-Man,

which is really a shame.

I liked those subway rides.

[train clanking]

The big challenge of hardware,

is working through the sheer number

of details involved, in creating a console.

And SSD, as an example,

will the game be entirely resonant on the SSD,

or is the SSD a cache,

and the game is on Blu-Ray,

or some other kind of optical media?

Do you want to use SLC, or MLC,

or TLC or QLC?

What kind of functionality do you want to have,

in your custom flash controller?

And how much do you want to have in your main custom chip?

Developers asked for an NBME SSD,

with at least a gigabyte a second of read speed.

And we looked at that,

we decided to go for maybe five or 10 times that speed,

it's always good to have a high target there.

Bringing the game development community

into the hardware development process,

is something that's pretty recent, historically speaking.

Over the years,

that was a dialogue that personally, as a developer,

I've really been wanting to have.

I'm looking for the developers

that give me the hardest time,

and the ones who really have strong opinions,

about what it is that they need,

to make the game that they've been dreaming of.

Those are just brutal meetings to be in,

but they're good to have,

because at the end of the day,

you're making a stronger console.

We decided on a strategy,

we called Integrated IO.

The biggest feature of Integrated IO,

at least from a developer standpoint,

is invisible compression.

Developers have to spend a lot of time and effort,

loading data in, off of media,

whether that's a hard drive or an SSD.

With the integrated IO, we take care of all of that.

The developers hand off their data to our publishing tools.

And then the game runtime just asks for the data,

and we handle the decompression for them.

We put as much functionality as we could

inside of a custom unit, in our main chip.

The impact of this compression strategy is huge.

It varies from game to game,

but a few standouts are Subnautica,

which is 14 gigabytes on PlayStation 4,

and to just four gigabytes on PlayStation 5.

Control: Ultimate edition,

which is 50 gigabytes, on PlayStation 4,

and about half that size, on PlayStation 5.

What we found out, with the launch titles is that,

for example, with Miles Morales,

even though it had added

all of these high resolution models and textures,

for the PlayStation 5 version,

The PlayStation 5 version was actually a bit smaller

than the PlayStation 4 version.

That growth in size from the assets,

had been offset by the improved technology, for compression.

[upbeat electro music]

The CPU.

The choice of the CPU is really important.

Pretty much anything you see, hear, or feel,

the CPU is involved in.

Animation, AI,

physics, collision.

Even things which have their own dedicated units,

like graphics or the 3d audio.

The smarts is still in the CPU.

Now, ideally you just use one incredibly fast CPU,

but that technology doesn't exist.

And so pretty much any device you use,

your PC, your phones, your console,

generates the necessary performance,

through using multiple CPU's.

Simulating something like New York City is really complex.

Particularly when you consider

that every pedestrian needs their AI,

and their animation,

all executed at what might be 60 frames a second.

Eight Zen 2 cores, really makes a difference there.

The impact of those eight CPU cores,

can also be felt in Returnal's battle scenes.

There are literally thousands of bullets in flight.

Destruction AllStars is using Zen 2's power,

for vehicle deformation and destruction.

The physics involves quite a bit of CPU processing.

There's lots of options for core count.

The question is how much of your custom chips

you want going into CPU's,

and how much you want going into the GPU?

When we did our tour,

we had some devs that would really like 16 cores,

but game development is so graphically oriented,

that when we explained that we could do that,

but the GPU would get smaller,

they instantly back off.

[upbeat electro music]

The GPU.

As a player,

you immediately notice the characters in these games,

hair, eyes,

skin, facial animation,

Let's see where this thing will take us.

We've come a long way,

from what could be done on PlayStation.

GPU's are fiendishly complex.

There's features that are added

year after year after year,

and this has been going on now for two, or three decades.

When we talk to developers,

most of the conversation is about GPU's.

About this alphabet soup of features,

that are incredibly important

to the graphics engine programmers.

They're looking for more performance.

They're looking for more flexibility,

in how they use that performance.

They're looking for new features,

and they're looking for modifications

to the features, that they already have.

We ended up with a customized AMD Radeon RDNA 2 GPU,

capable of running at 2.23 gigahertz and 10 teraflops.

That's 10 trillion floating point operations, a second,

as well as having a custom, cutting edge, feature set.

Epic Games have created a new software technology,

called Nanite, that focuses on the detail,

that the new hardware technology enables.

The idea behind this technology,

is that no matter how far you'd zoom into something,

no matter how close you get to something,

you're still seeing, like the real world,

more and more detail.

We made a lot of customizations to the GPU.

Some of them are smaller,

like, when PlayStation 5 crashes,

it's a lot easier to track down the source of that crash,

than it was on PlayStation 4.

Some of those customizations are giant,

like backwards compatibility with PlayStation 4.

Backwards compatibility is difficult,

because there are hundreds of essential

GPU features in PlayStation 4,

that the developers rely on.

And for their games to run flawlessly on PlayStation 5,

each of those features needs to be properly included.

We also need to sometimes insulate the games,

from the new PlayStation 5 capabilities.

There was a case early on,

where we ran a multimillion selling

PlayStation 4 game, on PlayStation 5,

and found out that the player character,

suddenly, was just running too fast.

And what was happening, was the power of PlayStation 5,

was translating into a higher frame rate,

and it broke gameplay.

And so to fix that particular bug,

we had to put in knobs,

that would allow us to dial in

just how much performance that game could handle.

[upbeat electro music]

Ray Tracing.

Without a doubt,

the most exciting new GPU feature is Ray Tracing.

One of the big uses of Ray Tracing is global illumination.

In Metro Exodus, light's streaming through a window,

it hits the floor,

it hits a wall,

and that bounce light, is what illuminates the scene.

Another big use of Ray Tracing is reflections.

Like being able to see Spider-man reflected

in the buildings that he's crawling over.

It's an impressive effect,

but it also shows why a new pipeline is required.

If you look at the 50 year history of games,

it splits pretty neatly into two eras.

The first era,

Pac-Man, [Pac-Man bleeps]

Sonic the Hedgehog, [rings jingle]

is games that are built, entirely out of flat components.

The second era,

Crash Bandicoot,

Uncharted.

These are games built out of 3d triangles,

with effects on them.

Thanks to Ray Tracing.

We're now entering a third era,

and the visuals have a capacity

to be like nothing we've ever seen before.

Ray Tracing is radically different technology.

It's pure computation.

There's a database in RAM,

that contains a description of the video game world,

mostly triangles and boxes.

And then there's support in the GPU hardware,

for seeing if a line intersects that geometry.

If the hardware is fast enough,

that unlocks new approaches to lighting,

and shadows and reflections,

The catch is, to use this, as a developer,

you have to build a whole new real-time graphics pipeline.

Ray Tracing was not one of the key features

that the developers asked for.

Mostly, I think,

because they had difficulty believing

they'd have enough Ray tracing performance,

to do anything interesting with it.

You can view it as a trade-off,

how much effort does it take to adopt

that new rendering pipeline,

versus how much difference does it make, visually,

to what the player is experiencing.

Developers immediately dug into the new feature set,

and at our very first PlayStation 5 game showcase,

we had six titles, that were using Ray Tracing.

Ray Tracing can also be used for shadows.

In the real world.

the edges of shadows are not perfectly crisp,

because the light that casts those shadows, isn't a point.

The light source itself has a size to it.

Call of Duty: Black Ops Cold War,

uses Ray Tracing and a lot of additional mathematics,

to create very realistic looking shadows,

with the proper softness.

[upbeat electro music]

Tempest 3d Audio tech.

One of the new features, that we're very excited about,

in PlayStation 5, is 3d audio,

and 3d audio is like nothing you've ever heard before.

The best way to experience it, is with headphoness.

[Rhino roars] Wait. Whoa, whoa, whoa.

[glass shatters] [rhino grunts]

Conventional stereo audio,

feels as if the sound is coming from inside your head.

With 3d audio,

it feels as if it's coming, from the world around you.

[lasers zapping]

We wanted 3d audio,

to be something that anyone can experience.

Something that would work with ordinary headphoness,

that you might already own, or with TV speakers.

We also wanted to be sure,

that there were no trade-offs for the game developers.

We didn't want to put them in the position of, for example,

having to choose between

having more enemies on the battlefield,

and supporting 3d audio.

Our solution for all of that,

was a dedicated 3d audio unit,

in our main custom chip.

[dramatic music] [explosions booming]

The custom engine, that drives Tempest 3d audio tech,

supports hundreds of sound sources,

and each sound source

can need massive amounts of flooding point calculations.

[dramatic music] [machine guns blasting]

Luckily, we have a way of doing that,

and GPU's are extraordinarily good,

at floating point calculations.

So what we ended up using for 3d audio, is a compute unit,

just like the ones in the GPU,

but this time, customized for 3d audio.

Part of what we do with 3d audio,

is create a sense of locality,

which is to say that you have a very good idea,

of where the sound is coming from.

3d audio also creates a sense of presence,

which is to say,

a feeling that you are immersed in the world of the game.

[beast roars]

Presence creates a lot of possibilities

for the game director,

and artistically, where they want to take the game.

To create that feeling of presence,

took quite a bit of technical learning.

Rain can't just be a rain sound, that's being replayed.

Instead, it needs to be synthesized,

from a dozen dynamic 3d audio sources,

surrounding the player, [rain teeming]

[upbeat electro music] Dualsense Wireless Controller.

Just like the target for audio,

was to bring in more of your sense of hearing,

the target with the Dualsense Controller,

was to bring in more of your sense of touch.

We did that in two different ways.

We added haptics,

to bring the player more into the world of the game,

explosions, footsteps,

and we added the adaptive trigger,

to bring you more in contact

with what you have, in your hands.

Like anything else,

there's a lot of different ways,

to incorporate those features into a controller,

so we did massive iteration on it.

The haptics that we ended up with, in the Dualsense,

work by playing tailored vibrations, through the controller.

Those can be sent to the left side or the right side,

which gives you additional feedback,

about what in the world of the game,

that you're interacting with.

Those vibrations aren't audio,

but they are mostly an audio frequencies,

and so typically, they are created by the sound designers.

The haptics also really bring home,

what your feet and hands are doing.

Sometimes that's obvious, like in Spider-man,

you're shooting webs.

Sometimes that can be surprising though.

My favorite moment in Ghost of Tsushima,

was that sensation, when you're riding on your horse,

and you can really feel the difference,

between galloping across dirt, or mud, or stone.

The other big new feature in Dualsense, is adaptive trigger.

You can feel what the player character

is holding in his hands.

The way we do that,

is by changing the resistance of the trigger.

Is it easy to pull?

Is it hard to pull?

We can also make it feel like the trigger is fighting you.

With previous controllers,

alt-fire required a different button,

or a different button combination,

but with Dualsense,

we can achieve that functionality,

by looking at how far the player is pulling the triggers.

In Returnal, for example,

you select from one of two firing modes,

whether you shoot a basic shot,

or the Doom Bringer,

depends on how much you depress the trigger.

[weapon zapping] [energy exploding]

Deathloop has a really interesting use of adaptive trigger.

When your gun jams,

you can feel it immediately in the controller.

Or when you're out of bullets, there's no resistance.

You just get a click.

Because there's two triggers, with separate control,

and also the haptics are independently specifiable,

for the left and right side of the controller.

Dual wielding works quite well.

You can really feel the weapon

that you're holding, in each hand.

[swords clashing] [upbeat electro music]

That completes the tour,

of the technologies behind PlayStation 5.

At the end of the day,

we're providing tools to game creators,

and as for how they choose to use those tools.

Well, the fun starts now.