What's the next big thing after ray tracing? (it's NeRFs)

Started by Legend, Jun 24, 2022, 06:53 AM

previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Legend

Game graphics are always improving and fundamentally changing. The jump from 2D to 3D was huge, the jump to HD was huge, the jump to PBR was huge, and the current jump to ray tracing/global illumination is a huge shakeup.

But what comes next?


Depending on how long the current gen lasts, I think the next evolution will be the death of polygons and the rise of AI rendering.





This video is made entirely from a neural network. There is rasterization or ray tracing involved. Instead 185 raw pictures from the real world are "converted" into a neural net that directly produces these results. It runs at like 0.2 frames per second and can only work with a static scene in a limited space, but Nvidia has a lower quality real time version you can try yourself. Getting Started with NVIDIA Instant NeRFs | NVIDIA Technical Blog




Of course this tech is still pretty young and we could easily see a different AI approach find far more success.

Legend

Here are some fun online demos you can try for yourself of lower quality versions: Baking Neural Radiance Fields for Real-Time View Synthesis


the-pi-guy

It's Nerf or nothing!  8)

kitler53

Jun 24, 2022, 03:36 PM Last Edit: Jun 24, 2022, 03:42 PM by kitler53
i hope not actually.   i've about hit my limit on graphics.

1. photo realistic graphics are boring.   i like how much more detailed games are then back in the 90's but i never want to play a "photo realistic" game.  i want to play an artisticly stylized game.

2. i've already hit a limit where good graphics are hurting gameplay.  back in the day when worlds were very barren it was very clear where the game wanted you to go and what to do next.   while the extra detail is pretty in a screenshot i've already run into a lot of games were insane amounts of detail make the game difficult to play.  when the screen gets very detailed and lush it is difficult to distinguish the background from the gameplay.  you can get lost.  you can lose the NPC trying to shoot you in the face.  It makes the games play worse.

at this point a lack of graphical fidelity never pulls me out of the game or breaks immersion.  what breaks immersion for me is all of these super lush and beautiful background objects not having any physicals to them at all.  they are immovable objects that break all laws of physics and lead to absurd moments of gameplay that is very immersion breaking.

As an example, this was a memorably bad moment for me:  I was playing horizon zero dawn and in sunfall.  there are a bunch of tents consisting of a bit of cloth strung on a stick.  alloy can walk on top of them and the tents have no physics at all.  it was as if the tents were made of solid steel instead of flimsy cloth.



for me this it a lot like the old "invisible walls".   i get why it's there,.. technically games can't be completely true to life but this has that "uncanny valley" feeling.  the cloth on those tents should have bent under the weight of alloy,.. collapse even.   it's not acting as expected and it's immersion breaking.

...so if the gaming R&D guys want to find the next "big thing" i suggest finding a way to get "physics" integrated into all asset creation.   We've already seen quite a few games go big specifically because they have at least some elements of physics baked into their game design (rocket league, zelda botw).   finding a way to make physics available to all game would be a real "next gen" moment for me from both the "graphical fidelity" perspective but also a "what's possible with gameplay" perspective.


Featured Artist: Vanessa Hudgens

Legend

1. photo realistic graphics are boring.   i like how much more detailed games are then back in the 90's but i never want to play a "photo realistic" game.  i want to play an artisticly stylized game.
Outside of abstract art styles like Return of the Obra Dinn, photo realistic rendering techniques help games regardless of their art style.



Journey for example looks so freaking good partially because the sun feels so real. Without high quality shadows, deformable sand, and physically accurate specular highlights, it just wouldn't be as good.

2. i've already hit a limit where good graphics are hurting gameplay.  back in the day when worlds were very barren it was very clear where the game wanted you to go and what to do next.   while the extra detail is pretty in a screenshot i've already run into a lot of games were insane amounts of detail make the game difficult to play.  when the screen gets very detailed and lush it is difficult to distinguish the background from the gameplay.  you can get lost.  you can lose the NPC trying to shoot you in the face.  It makes the games play worse.
I agree. A lot of techniques for subconsciously guiding the player are diminished with improved graphics. The current solution of pressing a button to enter high contrast mode isn't the best (witcher sense, generic item pinging, etc.)

at this point a lack of graphical fidelity never pulls me out of the game or breaks immersion.  what breaks immersion for me is all of these super lush and beautiful background objects not having any physicals to them at all.  they are immovable objects that break all laws of physics and lead to absurd moments of gameplay that is very immersion breaking.

As an example, this was a memorably bad moment for me:  I was playing horizon zero dawn and in sunfall.  there are a bunch of tents consisting of a bit of cloth strung on a stick.  alloy can walk on top of them and the tents have no physics at all.  it was as if the tents were made of solid steel instead of flimsy cloth.



for me this it a lot like the old "invisible walls".   i get why it's there,.. technically games can't be completely true to life but this has that "uncanny valley" feeling.  the cloth on those tents should have bent under the weight of alloy,.. collapse even.   it's not acting as expected and it's immersion breaking.

...so if the gaming R&D guys want to find the next "big thing" i suggest finding a way to get "physics" integrated into all asset creation.   We've already seen quite a few games go big specifically because they have at least some elements of physics baked into their game design (rocket league, zelda botw).   finding a way to make physics available to all game would be a real "next gen" moment for me from both the "graphical fidelity" perspective but also a "what's possible with gameplay" perspective.
Improved physics could also come from the death of polygons, but for now NeRFs are even more static than ray tracing.

If you didn't know, there's a physics game that more or less does what you're asking for.

kitler53

If you didn't know, there's a physics game that more or less does what you're asking for.


it's certainly not a bad example.  it's certainly helps facilitate the idea of what i'm saying but even so this is pretty unrealistic too.

it looks to me like this is based on voxels?   things can certainly be destroyed but everything destroys the same.   like in the first couple of seconds on that video you'll see the "leaves" of the tree has the same basic destruction as the brick wall being driven into.   driving though that brick wall was waaaay to easy if the idea is realism.  and relative to my prior example this kind of destruction would be completely inappropriate for fabrics.  the tent wouldn't "shatter" it would bend and possibly tear. 

i guess some amount of "material science" is what i'm talking about. 
- windows would shatter
- rocks would cleave
- fabric would bend and tear
- books would move and be able to be opened to a page
- pencils would snap in half
- Grass would get trampled
- tree trunks would be static while the branches would sway
- ...and i feel it's especially "unimmersive" how in shooters bullets never actually deform the environmental objects being shot depending on the material being shot.


stuff like that i guess.   i would still expect a game like TLoU to have "ambiance" objects in the game but if you try to interact with it in any way you realize it has the physics of a "bronze statue".   

maybe another way to put it is bethesda brags about how you can pick up a coffee cup in their game.   that's like,.. something to brag about compared to most games.   but even in a Bethesda game (at least the one i played) if you throw the cup off a 3rd story balcony it doesn't shatter when it hits the ground.  it bounces.

if the goal is to "increase graphical fidelity of games and induce a sense of realism" it's these sorts of details that would be a bigger deal to me than and even higher detailed table that is "nailed" to the floor and wouldn't crumble if i drove a 20 ton truck on top of it.  on the contrary,.. the more realistic the environment looks the more critical i'm becoming of how unrealistic it reacts.  i didn't have these concerns when everything looked so cartoony because,.. well,.. it looked like it should have cartoon physics.



Featured Artist: Vanessa Hudgens

Legend

it's certainly not a bad example.  it's certainly helps facilitate the idea of what i'm saying but even so this is pretty unrealistic too.

it looks to me like this is based on voxels?   things can certainly be destroyed but everything destroys the same.   like in the first couple of seconds on that video you'll see the "leaves" of the tree has the same basic destruction as the brick wall being driven into.   driving though that brick wall was waaaay to easy if the idea is realism.  and relative to my prior example this kind of destruction would be completely inappropriate for fabrics.  the tent wouldn't "shatter" it would bend and possibly tear.  

i guess some amount of "material science" is what i'm talking about.  
- windows would shatter
- rocks would cleave
- fabric would bend and tear
- books would move and be able to be opened to a page
- pencils would snap in half
- Grass would get trampled
- tree trunks would be static while the branches would sway
- ...and i feel it's especially "unimmersive" how in shooters bullets never actually deform the environmental objects being shot depending on the material being shot.


stuff like that i guess.   i would still expect a game like TLoU to have "ambiance" objects in the game but if you try to interact with it in any way you realize it has the physics of a "bronze statue".  

maybe another way to put it is bethesda brags about how you can pick up a coffee cup in their game.   that's like,.. something to brag about compared to most games.   but even in a Bethesda game (at least the one i played) if you throw the cup off a 3rd story balcony it doesn't shatter when it hits the ground.  it bounces.

if the goal is to "increase graphical fidelity of games and induce a sense of realism" it's these sorts of details that would be a bigger deal to me than and even higher detailed table that is "nailed" to the floor and wouldn't crumble if i drove a 20 ton truck on top of it.  on the contrary,.. the more realistic the environment looks the more critical i'm becoming of how unrealistic it reacts.  i didn't have these concerns when everything looked so cartoony because,.. well,.. it looked like it should have cartoon physics.


Yeah it's based off voxels and not at all realistic, but at least it's reactive.



Control has great destruction physics where it matters. You can't destroy everything but it all feels pretty realistic. Shoot the corner of a concrete wall and you'll break a chunk off.


You're right though that no game has everything. In general there is a "right" way to play games and everything outside of that doesn't have physics. TLOU2 though has a lot of the things you mentioned (spoilers): The Last of Us 2: All the Tiny Details You Missed - Push Square


A cup bouncing after a large fall is laughably bad.