Sora is also able to simulate artificial processesone example is video games, says OpenAI in the paper.
These capabilities can be elicited zero-shot by prompting Sora with captions mentioning Minecraft.
Sora performs thousands of calculations to predict how an object interacts with its environment.
AI-generated replica of Minecraft from OpenAI’s SoraImage: OpenAI
This creates a world model, according to Fan, which makes it perfect for generating video games.
The company notes several limitations, including that Sora does not accurately model the physics of many basic interactions.
However, Sora has already solved several issues that other video generators cant do.
Sora also has much better dynamic camera motion than other video generators.
Certain Sora demos do look oddly similar to existing video game worlds.
The question stands: what was Sora trained on?
OpenAI is facing a lawsuit for training GPT-2 and GPT-3 onThe New York Times articles, without payment.
However, GPT-2 was largely trained onOpenAIs WebText dataset.
WebText scraped Netflix over 42,000 times according todata made public on GitHub.
Crunchyroll, Hulu, and YouTube doubler were also mentioned as video formats used to train OpenAIs model.
However, Sora likely required more data than just this.
News from the future, delivered to your present.
Get your finest dark clothing and all your creepy things, part one ofWednesdayseason two is dropping in August.
