Sony files patent for AI frame prediction to reduce game stuttering

Sony has filed a patent describing artificial intelligence that would anticipate upcoming game frames, with the aim of reducing stuttering. Its proposed system would produce an image from the latest completed frame while graphics hardware works on the next one.

Some existing frame-generation methods need two finished images to create an intermediate frame, adding latency while they wait for the second. Sony’s approach would make a prediction using an image already available, avoiding that wait. The filing supplies no measurements demonstrating its effects on latency or image quality.

The design pairs an encoder with a decoder, trained together. The encoder turns a rendered image into a compact numerical representation, which the decoder uses to construct a prediction. Working alongside the graphics pipeline, the AI would send that image to a display buffer before rendering of the next frame finishes.

Optional inputs include recent controller actions and motion vectors describing movement within an image. For training, the network predicts an upcoming frame and compares the result with the actual image that follows. Those video images provide reference material without requiring human labeling.

The filing also outlines an alternative without AI that moves blocks of pixels based on their motion to create synthetic frames. Neither approach’s practical ability to maintain visual quality is established by the patent.