Genie 3, real time video games from Google Deepmind

[Legend]

https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/

Still looking through it, but there is definitely a catch.


First thing I noticed is that a lot of the content looks like it's from a video game. Specific lighting and animation things you don't have in CGI. So I don't know if this is because it was trained on tons and tons of video game content, or if it has something under the hood related to this. Will keep reading.

Second thing is that the framerate of the environment seems to be decoupled from the camera. 

The waves update at a lower framerate. And there is artifacting as new cliff faces are revealed behind existing cliffs. I will keep reading but I think it's using reprojection.

[Legend]

Can't find a technical breakdown of the architecture, but it's definitely generating content and displaying content in different ways.

Check out the window and the trees as they first become visible. They are being generated over multiple frames before they stabilize. But once stabilized, the trees can come in and out of view without needing to be re-stabilized.



Wish I could find a breakdown of what they changed from Genie 2.