Article · AI translation
Thanks to IT Home readers 刺客, 咩咩洋, and 乌蝇哥的左手 for the tips!
IT Home reported on October 28 that Google DeepMind unveiled its latest world model, Genie 4, on Tuesday. It is currently in research preview. The latest model can generate the most photorealistic interactive virtual worlds to date, and users can enter prompts to modify scenes while a simulation is running. The model can also generate real-time audio synchronized with the virtual world, including music, ambient sounds, and interactive dialogue.
IT Home notes that several months before Genie 4's release, AI video startup Runway launched its second world model, GWM Worlds 2, an interactive model that dynamically responds to user input and also supports audio generation. Meanwhile, a trend toward consolidation has begun to emerge in the world-model sector. World Labs, founded by Fei-Fei Li, was acquired by AMD; Amazon is in talks to acquire Decart; and Rivian has just acquired Third Dimension to build in-house simulation capabilities for autonomous-driving tests.
Launching Genie 4 is Google's latest move to maintain its lead in a fiercely competitive field.
More than a year ago, Google released its predecessor, Genie 3, as a research preview. In addition to much greater visual realism and new audio capabilities, Genie 4 supports free-form edits, such as changing the weather or camera angle in real time or adding people and objects. It can maintain a coherent, consistent scene for up to two minutes, twice as long as its predecessor, with better physics and more lifelike virtual characters.
Integration between Google's large model Gemini and Genie 4 has also increased. Gemini acts like a director, planning steps and checking frames, while Genie 4 generates the visuals. In a research experiment related to the Gemini robotics project, the Gemini 4 Argon model broke a task into multiple steps, which Genie 4 then simulated in real time. The experiment demonstrated a possible approach: validating scenarios in a simulated environment before testing robots in the real world.
Google research director Tejas Kulkarni said at a briefing: “In the future, we will increasingly see these kinds of hybrid world models, with Gemini writing code and text and Genie 4 reading them in real time to simulate reality at different levels of abstraction.”
As with other world models, its main proposed commercial applications include gaming, interactive learning, and robotics. One envisioned example of interactive learning would let users enter a painting depicting a historical scene and talk with its figures about the events recorded in the painting.
However, “trusted testers”—the first users able to try Google's frontier products—have found a potential new use: film prototyping.
Google DeepMind engineer Diego Rivas said at a media briefing: “The value of real-time capabilities is especially clear in situations where you need to maintain a continuous creative train of thought. For example, they are useful during ideation and storyboarding, and at every stage that calls for a rapid burst of ideas.”
Google has also launched Veo, an AI video-generation model that can create high-definition short videos with synchronized audio from text, image, or video prompts. The tool has continued to gain popularity among film and television creators.
Rivas added that users may eventually be able to use Gemini Omni, a multimodal large model, to turn prototype material into finished creative assets. The fully multimodal Gemini can generate video from images, audio, and text. He also noted, however, that these capabilities are still some way from practical deployment.
Throughout the briefing, Rivas repeatedly emphasized that Genie 4 remains in an early stage of development and that the experience offered by a finished product has yet to be determined. The team also acknowledged Genie 4's shortcomings: the longer a simulation runs, the more likely details and objects are to drift. The team has yet to find a reliable way to ensure audio accuracy while keeping text in the virtual world clearly legible.
Genie 4 is currently available to trusted testers as a research preview. Over the coming weeks, the model will gradually become available to Google AI Ultra subscribers and through Project Genie, Google's AI tool for generating interactive virtual worlds from text or image prompts. Early partners using the Gemini application programming interface (API) and Google AI Studio will also gain access.