Runway launches its first global model, and adds original audio to its latest video model

The race to launch global models has begun with AI-powered image and video generation company Runway joining a growing number of models Start-ups and big Technology companies By launching the first one. The model, called GWM-1, works by making frame-by-frame predictions, creating a simulation with an understanding of physics and how the world actually behaves over time, the company said.

A world model is an artificial intelligence system that learns an internal simulation of how the world works so that it can think, plan, and act without having to train for every possible scenario in real life.

Runway, which launched earlier this month Generation 4.5 video model Which beat out both Google and OpenAI on the Video Arena leaderboard, said its global model GWM-1 is more “generic” than Google Genie-3 and other competitors. The company is promoting it as a model that can create simulations to train customers in various fields such as robotics and life sciences.

“To build a global model, we first needed to build a really great video model. We believe the way to build a global model is that teaching models to predict pixels directly is the best way to achieve general-purpose simulation. At enough scale and with the right data, you can build a model that has enough understanding of how the world works,” Anastasis Germanides, the company’s chief technology officer, said during the live stream.

Runway has released specific directions or versions of the new world model called GWM-Worlds, GWM-Robotics, and GWM-Avatars.

GWM-Worlds is a template application that allows you to create an interactive project. Users can set a scene through a prompt or image reference, and as you explore the space, the model creates the world with an understanding of geometry, physics, and lighting. The company stated that the simulation runs at a rate of 24 frames per second and at a resolution of 720 pixels. While Worlds may be good for gaming, it’s also well-positioned to teach customers how to navigate and behave in the physical world, Runway said.

With GWM-Robotics, the company aims to use synthetic data augmented with new parameters such as changing weather conditions or obstacles. Runway says this method can also reveal when and how bots might violate policies and instructions in different scenarios.

Runway also builds realistic avatars within GWM-Avatars to simulate human behavior. Companies like an act, Synthesis, Soul machinesAnd even Google They have worked to create human avatars that look real and work in areas such as communication and training.

The company notes that worlds, bots, and avatars are technically separate models, but it plans to eventually merge all of these models into one.

Along with launching a new global model, the company is also updating its fundamentals The Gen 4.5 model was released earlier this month. The new update brings native audio and long-shot and multi-shot generation capabilities to the model. Using this model, users can create one-minute videos with character consistency, original dialogue, background audio, and complex shots from different angles, the company said. The company said that you can also edit existing audio and add dialogues. Additionally, you can edit multi-shot videos of any length.

The Gen 4.5 update pushes Runway closer to competitor Kling’s all-in-one video suite, which It was also launched earlier this monthespecially regarding the original audio and multi-shot storytelling. It also indicates that video generation models are moving from prototype to production-ready tools. Runway’s updated Gen 4.5 model is available to all paid plan users.

The company said it will make GWM-Robotics available through the SDK. It added that it is in active talks with several robotics companies and enterprises to use GWM-Robotics and GWM-Avatars.