Article · AI translation
IT Home reported on October 18 that, according to a report published today by technology outlet TechSpot, Sony recently filed a patent describing a way for AI to predict the next frame before the GPU finishes rendering it, reducing stutter in games while avoiding the additional latency introduced by frame generation technology.
IT Home learned from the original report that the technology could address a drawback of frame generation: the display would no longer have to wait for two frames to finish rendering before generating an intermediate frame between them. Instead, it would use an already generated frame to predict the next one, allowing interpolation without waiting for two frames and helping games maintain a more consistent frame output cadence.
Sony also uses a jointly trained encoder and decoder that converts rendered frames into embedded representations, providing compact numerical descriptions of the images that the decoder can use as cues to generate predicted frames. While a game is running, the system works in parallel with the graphics rendering pipeline. As the hardware renders the next frame, AI processes the previous frame and sends its prediction to the display buffer. The predicted image could therefore appear before the next frame finishes rendering.
The system can also use motion vectors and the player's recent controller inputs, making the history of how the player has controlled a character the basis for predicting its actions in the next frame. The system is trained on consecutive video frames. The neural network initially uses randomly initialized weights, takes one frame as input, and attempts to predict the next. The system then compares its prediction with the actual subsequent frame and calculates a loss value to measure the prediction error.
Whether the technology will prove useful in practice, however, depends on whether AI can quickly generate accurate images to fill gaps in frame output when rendering slows down.