Google boasts the power of Lumiére, artificial intelligence for video and image animation

Google introduced Lumiére, a research-based spatiotemporal diffusion model for video generation. Thanks to an advanced artificial intelligence (AI) system, it allows the creation of animated content with “realistic, varied and consistent movements.”

Unlike other generative video engines, the company’s Mountain View offering promises to replicate the levels of accuracy, smoothness, realism and complexity seen in cutting-edge AI tools that work with static images.

Amazon launches its own artificial intelligence image generator

A third of the leaders Fortune 500 He says that his main concern when using artificial intelligence systems is the possible violation of intellectual property.

The project is a collaboration between Google Research, the Weizmann Institute of Science, Tel Aviv University and the Israel Institute of Technology. The algorithm has been trained. with 30 million images and videos with subtitles freely available and from media content banks such as Unsplash, Freepik, Pexels and Pixabay.

His work is based on architecture Spacetime-U-Net (STUNet)which is capable of determining the position and orientation of elements in a photograph or video, detecting their movements and recognizing their changes during shooting. clips simultaneously. These qualities allow you to create animated videos and images, copy art styles, and edit specific areas of existing material.

“STUNet generates the entire temporal duration of a video on a single display in a single model pass. “This contrasts with existing video models that synthesize distant key frames followed by temporal super-resolution—an approach that inherently makes achieving global time consistency difficult,” explains the Google team.

How does Google’s Lumiere work?

Google Lumiére can convert text and images into videos, create multimedia based on the graphic style of a pre-created illustration, animate certain elements of photos, and edit certain areas of video using fast. The model generates up to 80 frames at a speed of 16 frames per second, which is converted into five-second videos with a resolution of 1024 by 1024 pixels. The tool does not allow you to create videos in multiple frames or with transitions between scenes. “Creating content with these characteristics using artificial intelligence systems remains an open challenge for future research,” the company admits.

Google researchers say the goal of Lumiére is to “enable novice users to create visual content in a creative and flexible way.” Nevertheless, resource is not yet available for testing and it is unclear whether it will ever be officially released to the market. Research group big tech warns that “there is a risk of misuse when creating false or harmful content using our technology. “It is critical to develop and apply tools to detect bias and malicious use.”