The developer of ChatGPT, OpenAI, has introduced a new AI tool called Sora that is capable of generating photorealistic videos based on text prompts alone. Sora, which means Sky in Japanese, can produce synthetic clips up to a minute long that are almost indistinguishable from real footage. OpenAI claims that the tool can respond to detailed requests and create complex scenes with multiple characters and accurate details.
To showcase the capabilities of Sora, OpenAI’s chief, Sam Altman, invited users on X (formerly Twitter) to submit prompts for videos, which the AI tool then generated. One of the notable examples includes a lifelike clip depicting an instructional cooking session for homemade gnocchi hosted by a grandmother social media influencer in a rustic Tuscan country kitchen with cinematic lighting.
However, despite the impressive realism of the videos, there are still some errors in the generated content. For instance, objects such as spoons and rings may appear or disappear mysteriously, while certain elements, like a rolling pin or a wooden wine bottle, may seem out of place. OpenAI acknowledges that its model sometimes struggles with physics and situations involving multiple characters, and there are instances where animals or people may spontaneously appear.
Additionally, the generated videos may contain humorous or unusual elements. In one clip, characters’ hands contort strangely, and the flames on a birthday cake do not move as the grandmother blows them out. Another example shows a basketball exploding when thrown through a hoop. OpenAI suggests that these bugs contribute to the generation of amusing content.
Sora is capable of understanding requests for specific camera angles or video styles, enabling it to produce both realistic footage and cartoon-like videos resembling those seen in Pixar films. OpenAI mentions an example of an extreme close-up shot of a 24-year-old woman’s blinking eye, set in Marrakech during magic hour and filmed in 70mm cinematic style.
The unveiling of Sora comes at a time when interest in AI tools has surged following the release of ChatGPT over a year ago. However, the introduction of this video generation tool raises concerns among artists, animators, and filmmakers about the potential impact on their professions. It also stirs fears about the proliferation of highly convincing deepfake videos on the internet.
Addressing these concerns, OpenAI emphasizes that its tool will reject requests to create videos featuring extreme violence, sexual content, hateful imagery, celebrity likenesses, or the intellectual property of others. Currently, Sora is only available to a select group of technical testers, artists, and filmmakers to help OpenAI identify any issues or opportunities related to the technology. The company plans to hire external security experts to evaluate the technology for vulnerabilities, as it acknowledges it cannot predict all the ways in which people might abuse it.
It is worth noting that OpenAI is not the only company releasing AI models capable of generating videos. However, what sets this particular product apart is OpenAI’s previous success with ChatGPT and DALL-E, an image generating tool, both of which have garnered millions of users. However, the company has faced criticism and legal challenges from artists and media outlets, including the New York Times, accusing OpenAI of disregarding copyright laws by using vast amounts of data to train its AI models. OpenAI argues that its usage falls within fair use rules.
As for the training dataset used for Sora, OpenAI has not disclosed the specifics, other than stating that it includes publicly available and licensed footage.
In conclusion, the introduction of OpenAI’s Sora represents a significant advancement in AI technology, capable of generating lifelike videos from text prompts. While the generated content showcases remarkable realism, there are still limitations and errors in the videos. As with any new AI technology, concerns about job displacement in creative industries and the potential misuse of these tools arise. OpenAI aims to address these concerns by implementing restrictions on certain types of content and soliciting input from testers and external security experts.
