Try Odyssey's New AI-Powered 3D Interactive Worlds

Odyssey, a startup founded by self-driving pioneers Oliver Cameron and Jeff Hawke, has unveiled an AI model that allows users to "interact" with streaming video. The model, accessible in an early demo on the web, generates and streams video frames every 40 milliseconds, enabling viewers to explore areas within a video much like a 3D-rendered video game. At the core of this technology is a sophisticated world model, which predicts the next state of the world based on current conditions, user actions, and historical data. This model demonstrates the ability to generate realistic pixels, maintain spatial consistency, learn actions from videos, and produce coherent video streams lasting five minutes or longer. Odyssey's vision is to transform traditional forms of video entertainment, ads, education, training, and more into interactive experiences, all powered by its AI technology. The company envisions these interactive videos as an "early version of the Holodeck," a futuristic concept from Star Trek where virtual environments can be explored with high fidelity. Current limitations in the demo include blurry and distorted environments and instability, where the layout of the surroundings might change unpredictably as the user navigates. These issues make the experience feel like a glitchy dream, according to Odyssey's blog post. The environments can be toggled between various settings, such as a wooded area with a cabin, a shopping mall, and a parking lot. Despite these challenges, the technology is exciting because it opens new possibilities for interactive media. For example, users can navigate these AI-generated worlds using the WASD keys, similar to controlling a character in a video game. Odyssey's approach to developing this technology is unique. Instead of relying solely on publicly available data, the company has designed a 360-degree, backpack-mounted camera system to capture real-world landscapes. This method is expected to produce higher-quality models. The demo is currently limited to 2.5 minutes of continuous exploration, but users can reload and re-enter the experience. Underlying the demo's real-time generation is the deployment of clusters of Nvidia H100 GPUs in the U.S. and Europe, capable of processing the input and frame history to generate the next frame in as little as 40 milliseconds. The cost per "user-hour" ranges from $1 to $2, making the technology relatively affordable for widespread application. Ed Catmull, a co-founder of Pixar and former president of Walt Disney Animation Studios, sits on Odyssey's board of directors. When asked about the future of the technology in an interview with The Verge, Catmull acknowledged the current limitations but expressed confidence in Odyssey's progress. He believes the company is at the leading edge of the field and noted that the broader community's ongoing improvements will help refine the technology. While Odyssey's demo does not yet rival established video games or replace movies, it shows significant potential. The technology's ability to generate real-time, interactive environments from captured real-world data hints at a future where traditional video content could become highly immersive and interactive. However, there are still numerous technical hurdles to overcome, such as improving image clarity and ensuring consistent and realistic interactions. Several other tech companies and startups are also exploring world models, including DeepMind, World Labs founded by influential AI researcher Fei-Fei Li, Microsoft, and Decart. They envision applications ranging from entertainment to robotics training. However, the creative industry has mixed reactions to this trend. A recent Wired investigation highlighted how game studios, such as Activision Blizzard, are using AI to cut costs and address workforce attrition, leading to concerns over job disruption. A 2024 study commissioned by the Animation Guild estimated that over 100,000 U.S. film, television, and animation jobs could be affected by AI in the near future. Odyssey emphasizes its commitment to collaboration with creative professionals rather than replacement. The company believes that interactive video can complement existing media, opening doors to innovative storytelling methods. Last December, Odyssey announced software that allows creators to load AI-generated scenes into tools like Unreal Engine, Blender, and Adobe After Effects for hand-editing, further integrating AI into the creative workflow. In summary, Odyssey's AI model for interactive video is a promising step towards creating highly immersive and interactive media experiences. While the current demo faces challenges, the company's innovative approach and collaborative stance suggest a future where AI and human creativity can coexist. Industry insiders recognize the potential but also the need for significant technological advancements to realize this vision fully. Ed Catmull's involvement and the company's robust funding further underscore the seriousness and potential impact of this endeavor.