Ray casting can be pretty cool. Variable rate shading has some nice applications. Deep neural networks inside games can be powerful.
Deep learning super sampling however is the holy grail. Or at least a version of it will be. Hence from here on I'll just say deep learning max settings emulation (DLMSE).
At this point we've all seen examples of neural networks increasing the resolution of pictures or interpolating frames, but the secret sauce of DLMSE is that the neural network is trained per game.
The setup is simple. Develop a variant of the game that renders the game to two separate screens/files. One screen has the game running at max settings beyond what the best gaming computers can do, while the other screen has the game render at average or even lower settings. Worse resolution, worse anti-aliasing, worse textures, worse geometry, worse shadows, etc. Could even lower the framerate if its not too much work on the devs.
Thousands of hours of gameplay are then recorded and shipped off to the cloud for neural net training. The net has the clearly defined goal of making the lower quality video look like the max quality video. It may take a lot of time/processing power but this is something that neural nets are fairly good at.
Once trained, the neural network can run on player computers and consoles to bump up the game graphics with only a minor performance hit. A PC with a 2 teraflop GPU could make the game look better than a 10 teraflop GPU running the game natively.
It even has applications for game streaming. The cloud can stream players a low bandwith lower resolution video and local hardware using DLMSE could bump it back up to highest quality 4k pixels.
For this reason, I predict next next generation consoles will include dedicated hardware for deep neural networks, just like new Teslas.
Deep learning super sampling however is the holy grail. Or at least a version of it will be. Hence from here on I'll just say deep learning max settings emulation (DLMSE).
At this point we've all seen examples of neural networks increasing the resolution of pictures or interpolating frames, but the secret sauce of DLMSE is that the neural network is trained per game.
The setup is simple. Develop a variant of the game that renders the game to two separate screens/files. One screen has the game running at max settings beyond what the best gaming computers can do, while the other screen has the game render at average or even lower settings. Worse resolution, worse anti-aliasing, worse textures, worse geometry, worse shadows, etc. Could even lower the framerate if its not too much work on the devs.
Thousands of hours of gameplay are then recorded and shipped off to the cloud for neural net training. The net has the clearly defined goal of making the lower quality video look like the max quality video. It may take a lot of time/processing power but this is something that neural nets are fairly good at.
Once trained, the neural network can run on player computers and consoles to bump up the game graphics with only a minor performance hit. A PC with a 2 teraflop GPU could make the game look better than a 10 teraflop GPU running the game natively.
It even has applications for game streaming. The cloud can stream players a low bandwith lower resolution video and local hardware using DLMSE could bump it back up to highest quality 4k pixels.
For this reason, I predict next next generation consoles will include dedicated hardware for deep neural networks, just like new Teslas.
