Google DeepMind Fuses Street View with Genie 3 to Create Interactive AI Worlds

BitcoinWorld

Google DeepMind Fuses Street View with Genie 3 to Create Interactive AI Worlds

Google DeepMind has taken a significant step in bridging the physical and digital worlds by integrating its Street View imagery directly into Project Genie, the company’s general-purpose world model. Announced at the Google I/O developer conference, the integration allows users to generate interactive, explorable environments anchored to real-world locations captured over two decades of Street View data.

From Street View to Simulated Reality

For 20 years, Google has collected over 280 billion images across 110 countries using camera-equipped cars and backpack-mounted trackers. Now, that vast dataset is feeding Genie 3, a world model capable of generating diverse, interactive 3D environments from text prompts or images. Jack Parker-Holder, a research scientist on DeepMind’s open-endedness team, explained to Bitcoin World that the combination of real-world data with generative simulation opens up powerful use cases for both robotics and human exploration.

“It’s really powerful for both the agent [and robotics] use case and for humans to play with,” Parker-Holder said. He described a scenario where a robot deployed in London — a city that rarely sees sun — could be trained on simulated sunny days generated from Street View data, so the sudden glint of sunlight off Victorian-era housing doesn’t disrupt its sensors. Similarly, a traveler planning a trip to New York City in winter could use the tool to visualize a snowy version of a specific block, adjusting weather conditions on demand.

Robotics and Autonomous Driving Training

Genie 3 is already being used by Waymo, Google’s self-driving car subsidiary, to simulate exceedingly rare events — such as tornadoes or unexpected animal encounters — for training autonomous vehicles. Parker-Holder noted that while Waymo has its own simulator focused on the car’s point of view, Street View integration allows shifting the perspective to other agents, like pedestrians or delivery robots, enabling more comprehensive training scenarios.

The ability to anchor simulations to real geographic locations could accelerate Waymo’s expansion into new cities around the globe, giving its AI driver exposure to diverse road layouts, signage, and environmental conditions without requiring physical fleet deployment.

Still an Experiment with Room to Grow

Despite the impressive demos — including an underwater simulation of a neighborhood — the technology remains experimental. Diego Rivas, a product manager at DeepMind, cautioned that Street View in Genie is still a work in progress. In samples shown to reporters, the environments were recognizable but video-game quality rather than photorealistic. The models also lack physics awareness: in one simulation, a woman running through a snowy Joshua Tree scene passed straight through cacti and bushes.

Parker-Holder acknowledged the gap, comparing Genie’s current accuracy to that of video-generation models from six to twelve months ago. “I think it’s something we will solve,” he said, noting that physics understanding emerges intuitively through passive observation, similar to how living beings learn.

Jonathan Herbert, director of Google Maps and a 12-year Street View veteran, emphasized that the real breakthrough is spatial continuity. When a user turns 360 degrees, the AI correctly remembers and simulates the environment behind them, then builds new environments on top of that understanding. “We have long thought about how we can build out the best and richest model of the world on top of Street View data,” Herbert said.

Availability and Next Steps

Google is launching Street View in Genie to select Ultra users in the United States starting today, with broader U.S. access rolling out over time. Global Ultra users will gain access over the next few weeks. The researchers’ goal, according to Rivas, is to put the capability into as many hands as possible, though he stressed that accuracy improvements remain a priority.

Conclusion

By connecting two decades of real-world imagery with generative AI, Google DeepMind is laying the groundwork for a new class of interactive simulations. While still in its early stages, the integration of Street View into Genie 3 represents a meaningful step toward AI systems that can understand, simulate, and interact with the physical world — with implications for robotics, autonomous driving, urban planning, and immersive education.

FAQs

Q1: What is Genie 3?
Genie 3 is Google DeepMind’s general-purpose world model that can generate interactive, explorable 3D environments from text prompts or images. It is designed for robotics training, gaming, and educational experiences.

Q2: How does Street View integration work?
The integration allows Genie 3 to use Google’s massive Street View image dataset — over 280 billion images from 110 countries — as a foundation for generating simulations anchored to real-world locations. Users can explore these environments interactively and adjust conditions like weather.

Q3: Is the simulation physically accurate?
Not yet. The current version lacks physics awareness, meaning objects may not interact realistically (e.g., a character running through solid objects). Google expects this to improve over the next 6–12 months as the model learns physics intuitively through more data.

This post Google DeepMind Fuses Street View with Genie 3 to Create Interactive AI Worlds first appeared on BitcoinWorld.