OpenAI has done it again. Since the launch of GPT-3, DALL-E, and then ChatGPT and its model GPT-4, the firm led by Sam Altman has become the vanguard of generative artificial intelligence. Now the company has surprised the world again with the release of Sora, its artificial intelligence model that generates video from text, thereby further pushing the boundaries of AI in the field of moving images, where it still seemed very far from delivering good results. Results.
Sora promises that she will turn written proposals into videos up to 60 seconds long. The first examples created are impressive, combining realistic images with other fantastical ones, although their first “hallucinations” have also been noticed. Let’s look at some examples of how it can be used, as well as the many questions it raises.
What is Sora and what are her characteristics?
Using deep learning techniques and neural networks, Sora can turn text into impressive and realistic videos if you so choose.
Unlike previous video generation models, which were still several steps behind their text and still image counterparts, Sora can generate videos of any resolution and aspect ratio, even up to 1080p.
An example released by OpenAI can be seen here:
Announcing Sora, our model that creates minute-long videos from text prompts: https://t.co/SZ3OxPnxwz pic.twitter.com/0kzXTqK9bG
— Greg Brockman (@gdb) February 15, 2024
this could be an AI “holy shit” moment. OpenAI just announced Sora, its artificial intelligence model for converting text to video. This video is not real, it is based on the plot of how “a cat wakes up its sleeping owner and demands breakfast…”? https://t.co/xKy3iQBKwT pic.twitter.com/HPm2p1jbgo
— Tom Warren (@tomwarren) February 15, 2024
One of Sora’s most notable features is its ability to simulate digital worlds. By experimenting with games like Minecraft, Sora can visualize entire environments and control characters in real time. This opens up a wide range of possibilities for creating video games and interactive entertainment.
Additionally, Sora can also perform image and video editing tasks such as creating loops, extending a video forward or backward in time, and changing the background of an existing video. These features make Sora a versatile and powerful tool for audiovisual production.
Here we can see a more quirky or creative example:
today we took another step closer to this… #Sora pic.twitter.com/oQJ31ZglWx
— Beeple (@beeple) February 15, 2024
Outperform your competitors in video generation
While there are other AI-based video generation models, such as Pika or Synthesia, Sora stands out for its ability to realistically simulate digital worlds.
Unlike other models that rely on generating a single image or video, Sora goes further and appears to be able to calculate the physics of every object in the environment to render images and videos based on these simulations. This gives Sora a significant advantage in terms of realism and visual quality.
Of course, he is also “hallucinating” and making mistakes, as we see here:
even the mistakes of litter are fascinating pic.twitter.com/OvPSbaa0L9
— Charlie Holtz (@charliebholtz) February 15, 2024
Another area where Sora excels over its competitors is the resolution and aspect ratio of the videos it produces. While other models have limitations in this regard, Sora can output video in any resolution and aspect ratio, making it more adaptable to different platforms and devices.
How to use Sora
OpenAI decided to initially limit its access to a program with limited access to only some users. This is due to the possible consequences and dangers associated with this technology, such as the creation of deepfakes and fake videos.
The firm said in a statement that it is “committed to ensuring the ethical and responsible use of Sora” and is working on policies and regulations for its safe implementation, but a few paragraphs later it assures that the consequences “cannot be predicted by the types of use and abuse to which this technology can lead.
What consequences and dangers do these models pose?
The possibilities that Sora offers are stunning, but they also carry obvious dangers when they are opened up to the public. On the one hand, the ability to create realistic, high-quality videos could revolutionize the entertainment and audiovisual production industries. Creating digital worlds and virtual characters is becoming more accessible and realistic.
However, there is a risk of using this technology incorrectly. Deepfakes, manipulated videos that appear real, can be made even more convincing by Sora’s ability to create realistic videos.
What seems clear is that with the launch of Sora, OpenAI is once again demonstrating its leadership in the field of generative artificial intelligence.