Support NeoGAF

[Zuzu]

Nvidia has once again jumped way ahead of the competition with DLSS 4. DLSS 4 is using a transformer-based model for image upscaling compared to the CNN-based model that was used for DLSS 2 & 3.

So where does this leave PSSR? PSSR is using a CNN-based model and you have to wonder whether this is ultimately a dead-end. Should Sony reassess their decision to go with a CNN-based model? Perhaps they seriously need to consider building a transformer-model in conjunction with improving their current CNN-model?

Maybe the future should be a transformer-based PSSR for Playstation 6? What do you think?

 

[diffusionx]

lol all of the sudden everyone is an expert on these advanced techniques

 

[RoboFu]

does it matter? you have what 2-3 more years of ps5?

 

[Magic Carpet]

I have not seen a single frame of PSSR in person but the answer to OP is yes.

 

[winjer]

Do you even know what type of model Intel, Sony and AMD are using in their AI training?
I don't think I ever saw that information.

 

[Zuzu]

does it matter? you have what 2-3 more years of ps5?

I think it does if we want the best image quality for Playstation moving forward. It's likely too late for the PS5 Pro to receiver a transformer-based model but the decisions they make now could have ramifications for the PS6. It might be good to start to look into and develop a transformer model now so it's ready for the PS6.

 

[squidilix]

PSSR are under 6 month, and already catch the DLSS 3.2 in some ways and people already arguing "Lol, PS5 Pro with PSSR is doomed"...

chill ffs... you think?

And what about FSR4 ? CNN or Transformer Model? FSR4 is already dommed too ? Switch 2 is doomed because it using old version of DLSS?

 

[omegasc]

did you mistake GAF for a chatGPT prompt?

 

[intbal]

I'm sorry, OP. I just don't understand how this is going to help PSSR.

 

[Embearded]

Do you even understand what all of this means?

 

[Crayon]

I'm sorry, OP. I just don't understand how this is going to help PSSR.

On the other hand I don't see how that could hurt.

 

[Mister Wolf]

They gotta walk before they run.

 

[Buggy Loop]

Do you even know what type of model Intel, Sony and AMD are using in their AI training?
I don't think I ever saw that information.

Sony and Intel’s patents clearly refer to CNN model

No idea for AMD’s upcoming one but yea.. who expects anything from them other to copy Sony and 5 years late to the party?

 

[Loomy]

Nvidia has once again jumped way ahead of the competition with DLSS 4. DLSS 4 is using a transformer-based model for image upscaling compared to the CNN-based model that was used for DLSS 2 & 3.

So where does this leave PSSR? PSSR is using a CNN-based model and you have to wonder whether this is ultimately a dead-end. Should Sony reassess their decision to go with a CNN-based model? Perhaps they seriously need to consider building a transformer-model in conjunction with improving their current CNN-model?

Maybe the future should be a transformer-based PSSR for Playstation 6? What do you think?

What is the difference between CNN and Transformer models?

 

[mitch1971]

D L SS!

Framerates in disquise!!

 

[analog_future]

Yes, Cerny just needs to press the Transformer button. Don’t know why he didn’t think of this sooner

 

[Mister Wolf]

What is the difference between CNN and Transformer models?

More stable.

 

[Kings Field]

 

[Topher]

Was anyone asking why Nvidia doesn't use a "transformer model" before DLSS 4?

 

[Senua]

Was anyone asking why Nvidia doesn't use a "transformer model" before DLSS 4?

I bet O onQ123 was

 

[diffusionx]

What is the difference between CNN and Transformer models?

nobody posting here knows

 

[gerth666]

Some people really watch too much digital foundry.
Why is dlss and pssr constantly compared? Playstation is never going to use dlss

 

[Senua]

nobody posting here knows

One is gooderer

 

[Topher]

Some people really watch too much digital foundry.
Why is dlss and pssr constantly compared? Playstation is never going to use dlss

Because of an infamous, ill-advised PSSR vs DLSS/4090 thread one guy made. After that....

 

[viveks86]

Depends on how heavy a transformer model is. We will need to benchmark dlss4 on an older gen gpu that is comparable with the pro to see what the difference in overhead is. Cerny’s biggest hurdle seemed to be finding a “lightweight” CNN that worked. Certainly something to consider for future hardware iterations, if they hit a limit with CNNs.

 

[winjer]

Sony and Intel’s patents clearly refer to CNN model

No idea for AMD’s upcoming one but yea.. who expects anything from them other to copy Sony and 5 years late to the party?

Where did you see that?

 

[gerth666]

Because of an infamous, ill-advised PSSR vs DLSS/4090 thread one guy made. After that....

I've been gaming since the sega master system, so a good 30 yrs or so, and never understand the platform warrior mentality.
I say this as big playstation fan.

 

[Gaiff]

Was anyone asking why Nvidia doesn't use a "transformer model" before DLSS 4?

I was.

One day I walked up to Jensen and said, "Hey, DLSS is good, but it would be better if you upgraded the model to the Transformer one." Jensen replied, "Wow! You are a genius my friend."

And that's the story of how NVIDIA switched to the Transformer model for DLSS.

 

[viveks86]

Given the performance hit seen here, it won’t be worth it on a current gen console. They will need way more TOPS (and who knows what else, memory/bandwidth perhaps?) to make the upgrades worth it. May be worth considering for PS6.

Edit: I was wrong. Lower settings are now giving better results. Also this chart is more appropriate for the topic

 

[onQ123]

Nvidia has once again jumped way ahead of the competition with DLSS 4. DLSS 4 is using a transformer-based model for image upscaling compared to the CNN-based model that was used for DLSS 2 & 3.

So where does this leave PSSR? PSSR is using a CNN-based model and you have to wonder whether this is ultimately a dead-end. Should Sony reassess their decision to go with a CNN-based model? Perhaps they seriously need to consider building a transformer-model in conjunction with improving their current CNN-model?

Maybe the future should be a transformer-based PSSR for Playstation 6? What do you think?

Find out on the next episode of Dragon Ball Z

 

[onQ123]

I bet O onQ123 was

I prefer the Voltron model

Spoiler: but on a serious note

I have no idea what's going on here

 

[Edgelord79]

lol all of the sudden everyone is an expert on these advanced techniques

Welcome to the internet.

 

[Buggy Loop]

What is the difference between CNN and Transformer models?

AI image and video models have moved from CNN to Transformer models, it's just the way the model is trained and learns.

I'm not AI specialist, but the gist of it



They require a shitload more data to train but that is not pertinent for us because that's Nvidia's job with their supercomputers. The model requires more parameters and computation to work, but the ~5-10% different in performance penalty is vastly offset by the visual quality upgrade.

Where did you see that?

Integrating XeSS with Velocity and Luminance Adaptive Rasterization

Integrating XeSS with Velocity and Luminance Adaptive Rasterization

www.intel.com

It's even in their logo, a full-connected neural network (CNN)

from 2023

They describe additions to the basic CNN/RNN model for changing training weight and so on. They all have their tweaks but the foundation is the connected network as a baseline.

 

[Buggy Loop]

Given the performance hit seen here, it won’t be worth it on a current gen console. They will need way more TOPS (and who knows what else, memory/bandwidth perhaps?) to make the upgrades worth it. May be worth considering for PS6.

False

IF you were to just slap transformer model quality vs CNN quality presets, then you have a loss of performance. But still we're talking ~5%-10% which is nothing really, especially not on consoles where you probably cap framerate with a lot of leftover frametimes.

But peoples have downclocked a 3050 to 3TF (guess for what comparison ) and manage to run the model in Cyberpunk 2077. It's so good in image quality that the performance hit does not matter if DLSS balanced / performance look as good as CNN quality. So it is a net gain in performance and image quality. The model somehow handles super low internal resolutions way way better than old model.

If a 3TF downclocked 3050 can, a PS5 & Xbox series can definitely.

And performance metrics we have seem already outdated as upcoming drivers improve it.

Shit even ultra performance is legit with this model.

 

[poppabk]

Wait until AMD releases their Gobot model before making judgement.

 

[lakitu1982]

I couldn't give a damn, just bring me the games.

 

[DeepEnigma]

 

[yurinka]

I'm sorry, OP. I just don't understand how this is going to help PSSR.

Here you have Arcee, the Transformer model to better fit DEI standards:

 

[yurinka]

But peoples have downclocked a 3050 to 3TF (guess for what comparison ) and manage to run the model in Cyberpunk 2077.

At 3 frames per second and 240p resolution maybe?

 

[vkbest]

Depends on how heavy a transformer model is. We will need to benchmark dlss4 on an older gen gpu that is comparable with the pro to see what the difference in overhead is. Cerny’s biggest hurdle seemed to be finding a “lightweight” CNN that worked. Certainly something to consider for future hardware iterations, if they hit a limit with CNNs.

Any model running on few ms is a lightweight model. I'm surprised people don't understand this basic thing. A heavyweight model is for example stable diffusion.

 

[rofif]

I dont care. First they have to fix silent hill

 

[viveks86]

False

IF you were to just slap transformer model quality vs CNN quality presets, then you have a loss of performance. But still we're talking ~5%-10% which is nothing really, especially not on consoles where you probably cap framerate with a lot of leftover frametimes.

But peoples have downclocked a 3050 to 3TF (guess for what comparison ) and manage to run the model in Cyberpunk 2077. It's so good in image quality that the performance hit does not matter if DLSS balanced / performance look as good as CNN quality. So it is a net gain in performance and image quality. The model somehow handles super low internal resolutions way way better than old model.

If a 3TF downclocked 3050 can, a PS5 & Xbox series can definitely.

And performance metrics we have seem already outdated as upcoming drivers improve it.

Shit even ultra performance is legit with this model.

Thanks for the evidence. I stand corrected. Hadn’t thought about lower settings giving better results now.

 

[Buggy Loop]

At 3 frames per second and 240p resolution maybe?

Hur dur dur

Why come in here with shitposts?

At 3T, 4GB VRAM and older driver

1080p (720p, T/CNN): 40.38 / 47.29 (~ +3.6ms of cost)
1440p (720p, T/CNN): 32.59 / 41.13 (~ +6.3ms of cost)


This is not a video of the guy with stats above but goes into dlss 4 settings on a 3050



Plenty of performance to be gained at lower preset.

So yea, arguments that low TOPs RTX cards cannot run the model evaporated

 

[Robb]

 

[Skifi28]

Buzzword threads are the best!

 

[saintjules]

 

[reezoo]

PSSR achieving DLSS3 will be a huge win fit Sony and PS6. They can continue to improve after that.

 

[kazuyamishima]

Something like this ?

 

[SKYF@ll]

PSSR has a slight shimmer when the player is stationary.
But it does an excellent job of upscaling the finer details, and the quality is good when the player is moving.
Here's a comparison image of a Jedi Survivor created with help from Boji. *DLSS 3.8 *PSSR the latest ver.
I'm looking forward to DLSS4 and future updates to PSSR.

 

[YCoCg]

It needs more time to train and be implemented properly.