Can We Please Just Be Normal About AI

On world models and the temptation to view new things through a well-worn lens

When I talk privately with game devs and industry people, more and more of them admit to feeling exhausted by the way AI gets discussed online, especially in games enthusiast and industry circles.

One problem is that voices representing the most extreme worldviews tend to dominate the conversation. The stories told about AI's looming impact on games tend to reduce down into one of the following visions:

1. We are building God (and also The Matrix) inside a datacenter in Milwaukee

or

2. Machines that feast on stolen art are drying up the oceans, and the only source of water left on Earth will be the tears of unemployed former game developers.

Neither of these visions have much to do with observable reality.¹ And the more tightly people clings to one of these visions, the more difficulty they have in understanding and reacting reasonably to novel applications of AI tech.

Far and away the clearest example of this as it applies to games has been the recent emergence of “world models,” basically a new form of interactive video where individual video frames are generated dynamically in response to user inputs.

The first, most rudimentary version of these models appeared last year with Google’s GameNGen prototype (built around Doom). This was quickly followed by other AI-generated world models based on other games, like Etched and Decart’s Oasis, a hallucinatory AI simulation of Minecraft that was made playable to the public as a browser app.

Approached on their own terms, these were early demos of a technology with uncertain—but definitely interesting—possibilities.

But that’s not how many saw them. Here’s one especially confused reaction to the Oasis demo:

This is a person who is fundamentally missing the point. A demo like this is attempting to show what interactive entertainment might be like on computers that are probabilistic instead of deterministic. It’s not about literally making a new way to play Minecraft. The point is to inspire people to imagine: Wow, what if they developed this model further and trained it on things in addition to Minecraft?

But Alex thought he was looking at just another Minecraft clone. If you’re operating with a broken worldview of what AI is, you actually can’t understand what you’re looking at.

Here’s another example, this time with a person reacting to a similar world model tech demo from Microsoft that happened to be based on Quake II:

There’s a lot to unpack here. Is the idea that this tech is going to harm the developers of Quake II? The reaction only makes sense if you’re predisposed to view every new AI tool as de facto harmful to artists and developers.

In any case, at least one person disagreed with Ruby Ranger's take here: the actual lead developer of Quake II, John Carmack:

Any person with a shred of curiosity ought to be gobsmacked by an interaction like this. One of history’s most famous games programmers just replied to your sassy tweet to disagree with you. How might you respond?

Instead of asking why Carmack thinks the demo is impressive, the poster took it as a chance to explain to the creator of Doom why he’s wrong. This in turn prompted Carmack to write a thoughtful take on AI tool development and its likely impact on games:

Carmack is introducing a completely different vision for analyzing AI models: not as a uniquely evil or godlike type of computer, but as just another tool. In this case specifically, Carmack sees a demo that fits into a long tradition of software “power tools” that includes ones he himself invented.

You can disagree with Carmack’s take, but let’s adopt his model for a second. There’s a lot to be said for the idea of treating individual AI applications as just another power tool for devs. Slowly, surely, specific AI tools are emerging now that can do things like generate 3D character models from concept art, automatically detect joints for rigging, and speed up the animation process by interpolating frames when given a starting and ending point. Future versions of these apps will make game development faster and easier, but they're hardly set to make human creatives obsolete. To make really high-quality stuff using these tools, they need to be wielded by skilled human operators. And in many cases these tools aren’t even replacing legacy game dev software like Blender. They’re integrating directly with existing tech stacks—just one more tool in the toolbox for artists.

Certainly, many devs say that AI coding tools are dropping their costs and dev time now, and that stuff should only get better. But as it stands today, even AI power-users say that today’s AI tools aren’t good enough to build high-quality games—at least not yet.

---

One more model couldn’t hurt

I like the way Carmack thinks about AI, but I don’t want to make the case that he’s definitively, 100% correct. The point is that there are other ways of viewing AI, even if you add the “it’s all just tools” model to your toolbox.

Even that way of thinking is inadequate to describe some of the things that are beginning to be achieved with new foundational types of AI, especially the most cutting edge stuff happening in the world model space. If you haven’t already, you really need to watch this demo of Genie 3, Google DeepMind’s new world model:

This, of course, is the same core technology that powered the earlier experiments with Doom, Minecraft, and Quake II… except much more advanced. And it happened very quickly: less than a year after the publication of Google’s GameNGen paper. In fact, the connections go deeper than that. Last week I had a chance to chat with Genie 3 co-lead Jack Parker-Holder for the a16z speedrun newsletter, and he stressed that one big unlock for Genie 3’s advances came from his team’s collab with Shlomi Fruchter, one of the co-authors of the GameNGen paper.

Whatever you call this new branch of the AI tech tree, “interactive video” or “world models,” it’s obviously something different. And yet, this is the cynical, incurious way the only major video game site to cover Genie 3 framed it:

We are suffering from a dearth of imagination, or maybe just a lack of adequate mental frameworks to make sense of this stuff. Truly new things like world models are not AI Gods, or Art Regurgitators, or even Just Another Tool. They are something new. And when we encounter something that doesn’t fit neatly within our existing worldviews—our models for understanding reality—we have to attempt, in a spirit of curiosity, to understand that thing on its own terms.

Above: A Genie 3 “world” generated by Google DeepMind researcher Philip J. Ball

That’s it for this week. I’m gonna go play Quake II.

1. Recent data shows that concerns about AI’s impact on the environment are pretty seriously overblown—especially when comparing AI usage with everyday activities like eating hamburgers, heating your home, or taking showers. University of Oxford Senior Researcher Hannah Ritchie has done multiple excellent deep dives on LLM electricity usage and likely impacts on carbon emissions which are worth reading.

Image from Andy Masley’s blog on this topic