Issue #18

When an AI Can Watch a Game and Build a New One

March 8, 20264 Stories8min Read

Welcome back to AI Playground. Turns out watching someone play a video game is now considered training data.

Editor's Note

This week’s stories show the industry figuring out where AI actually fits in gaming. Some studios are stepping back from generative tools that don’t quite hold up yet, while others are pushing ahead with new AI systems designed to make games more responsive and dynamic. The result is a mix of caution and experimentation as developers test how far these technologies can really go.

01

AI trained to play video games in stepping stone to real world applications

AI trained to play video games in stepping stone to real world applications

Researchers have developed an AI that learns to play video games as a method to improve real-world applications, showcasing advanced problem-solving skills and adaptability in complex environments.

Game developers can leverage these AI advancements to create more intelligent NPCs and dynamic gameplay experiences, enhancing player engagement and immersion.

02

Developer use of generative AI may be declining

Developer use of generative AI may be declining

A recent report from the Game Developer collective reveals that the use of generative AI among game developers has decreased significantly year-over-year.

This decline may indicate that developers are seeking more reliable tools for game design, potentially affecting innovation and project timelines.

03

Xbox files for patent to have AI helpers play games for you

Xbox files for patent to have AI helpers play games for you

Xbox has filed a patent for AI helpers that can assist players during tough gaming sessions by taking control of the game temporarily, utilizing a multi-modal generative model to create inputs from various media types.

This innovation could enable game developers to enhance player engagement by allowing friends or AI to assist in challenging parts of games, ultimately making gameplay more accessible and enjoyable for a broader audience.

04

GAMING: Are game developers using AI? Players want to know

GAMING: Are game developers using AI? Players want to know

A recent survey revealed that 53% of game developers are currently incorporating AI technologies in their projects, showcasing a growing trend in the industry.

This shift allows developers to enhance gameplay experiences, streamline production processes, and create more immersive worlds, ultimately meeting player expectations for innovative features.

Deep Dive

When an AI Can Watch a Game and Build a New One

Google DeepMind quietly released something last year that feels like science fiction for game developers. It’s called Project Genie, and the basic trick is simple to describe: show it a short clip of gameplay, and it builds a playable game world from it.

Not a static level. A real environment with physics, collisions, enemies, and movement that responds to player input.

Genie works by training a world model on huge amounts of gameplay video. Instead of generating art assets or code snippets, the model learns how games behave. When a player presses left, the system predicts what the next frame of the world should look like. Gravity pulls characters down. Enemies react. Objects collide. The environment keeps updating in real time, frame after frame, like a tiny simulation engine that learned its rules by watching other games.

For indie developers, the practical change is wild. Prototyping a small platformer used to mean days or weeks of assembling assets and scripting logic. Now a developer can feed Genie a short clip, tweak a few parameters, and generate thousands of playable level variations in an hour. One indie dev used the model to spin up a roguelike prototype over a weekend, with enemy encounters and loot patterns shifting from run to run.

What makes Genie interesting is that it produces worlds that respond to players instead of just generating layouts. Older procedural tools could spit out terrain or dungeons, but they often broke once players started pushing the system in weird directions. Genie generates the world step by step while the player moves through it, so the environment keeps adapting.

This has caught the attention of engine makers too. Early experiments with similar world models are already showing up in development tools, where studios use them to generate environments, crowds, or test levels before committing to full production assets.

It is still early days. No one is shipping full commercial games built entirely this way yet.

But the idea that a game engine could learn how a world works simply by watching gameplay opens the door to something developers have chased for years: worlds that can invent themselves as you explore them.

Rate This Issue

Never Miss the Future of Gaming

Join AI researchers, game developers, and creative technologists exploring the frontier. The stories shaping AI and gaming, explained clearly each week. No noise.

By subscribing, you agree to receive weekly emails. Unsubscribe anytime.

Weekly
Newsletter
50+
AI Sources
100%
Free

No spam, unsubscribe anytime • Delivered every Sunday • Privacy protected