Abstract: World models excel at generating interactive environment visuals, while traditional game engines excel at executing precise rules, physics simulation, and state management. The two technologies operate at different layers, so the most sensible direction is to use them together.
Bottom line upfront: Traditional game engines are still necessary. Three representative world models—Genie 3, WHAM-RT/WHAMM, and Oasis—can already generate actionable visuals in real time at 10 to 24 FPS, but their temporal consistency is measured in seconds or minutes, and they make mistakes at the rules layer—object memory, damage resolution, health tracking, and so on. Game engines provide deterministic rule execution, physics calculations, state persistence, and multiplayer synchronization—exactly the foundation on which commercial games run. At this stage, world models are well suited to environment generation, visual prediction, and content creation assistance, while the engine continues to own the core rules. Together they form a hybrid architecture.
The rest of this article first explains the difference in operating logic between world models and traditional engines, then uses publicly available metrics from the three models to assess how far they’ve come, breaks down specific failure cases publicly disclosed by WHAMM, lists the engineering responsibilities handled by mature engines, and finally presents the division of labor in a hybrid architecture, scenario selection, and a runnable code example.

To Know: World Model and Game Engine Terminology Prep
| Term | Explanation |
|---|---|
| World Model | An AI model that predicts and generates the environment state for the next moment based on the current frame and player actions |
| FPS | In this article, refers to the number of frames rendered per second; in the context of game genres, it also refers to first-person shooter games |
| Resolution | The number of pixels in the horizontal and vertical directions of the image, for example 640×360, 720p |
| Context Window | The range of historical information the model can reference when predicting subsequent states; in world models, it’s often measured in seconds or frames |
| Autoregressive Generation | Each frame is generated based on the previously generated frames, so errors accumulate frame by frame |
| Physics Engine | A system that calculates physical constraints such as collisions, motion, and forces |
| State Persistence | Saving the states of players, quests, and world objects so they can be read and continued at any time |
| Multiplayer Synchronization | A mechanism that lets different players see and interact with the same consistent game state |
| Determinism | The same input always produces the same result; it’s a prerequisite for debugging, replay, and multiplayer |
| Game Engine | A development and runtime system that organizes rendering, input, physics, animation, networking, and resource management |
| Hybrid Architecture | A combined approach where traditional program systems handle rules and state, while generative models provide content |
1 AI 实时生成游戏世界:世界模型与游戏引擎的运行逻辑差异
1.1 世界模型从生成素材走向生成世界
过去 AI 在游戏开发中主要生成贴图、3D 模型、配音等素材,这些素材最终仍然交给引擎组装并运行。世界模型改变了这种分工:模型直接接收玩家的键盘或手柄输入,预测下一帧画面,循环往复,形成一个“可以玩”的环境。Google DeepMind 的 Genie 3、微软研究院的 Muse 系列(WHAM-RT/WHAMM),以及 Decart 与 Etched 的 Oasis,是这一方向的三个代表。
传统游戏的运行逻辑是开发者预先定义场景、模型、规则和物理系统,玩家的操作触发预设逻辑,引擎计算出新状态后再渲染画面。世界模型的逻辑是“观察 + 动作 → 预测下一帧”,规则隐含在训练数据学到的视觉动态里。前者的画面由状态计算而来,后者的画面本身就是模型对状态的近似。
1.2 AI 写代码和 AI 生成画面属于两条路线
2026 年 11 月 9 日,Anthropic 工程师 Felix Rieseberg 发布的 Saltline 常被拿来和世界模型一起讨论。这是一款可在浏览器中运行的小规模第一人称射击可玩原型,场景是雨夜的集装箱港口,包含一个任务关卡。根据开发者本人自述的估算,Claude Fable 5.2 在单次提示词、无人工反馈的条件下用了约 8 小时,消耗约 4.45 亿输入 Token(其中大量为缓存读取)和约 270 万输出 Token,按 API 定价估算约 550 美元;这些数字来自开发者公开帖子,属于其自述估算。
Saltline 的“3A”只是开发者设定的实验目标和视觉类比。它的技术路线是让 AI 生成约 1.4MB 的 HTML/JavaScript 代码,3D 模型、纹理和声音都由代码与 Shader 程序化生成,运行时由浏览器通过 Three.js 和 WebGL 在本地渲染。这条路线中,AI 在开发阶段工作,运行阶段仍是确定性的程序,本质上沿用了“引擎式”运行逻辑。世界模型则让 AI 在运行阶段逐帧生成画面,两者的成本结构和可靠性截然不同。

2 Technical Metrics of Three World Models for Real-Time Game World Generation
2.1 Comparing the Public Metrics of the Three Models
The numbers in the table below are metrics each team published in their official technical demo environments. The test conditions differ from one another, so they’re good for gauging order of magnitude but not for direct side-by-side ranking.
| Metric | Genie 3 (Google DeepMind) | WHAM-RT/WHAMM (Microsoft Research) | Oasis (Decart & Etched) |
|---|---|---|---|
| Release date | August 5, 2025 | 2025 | October 2024 |
| Demo environment | General-purpose environment generated from text prompts | Quake II-like interactive environment | Block-sandbox-like world |
| Frame rate | Up to 24 FPS | Above 10 FPS | 20 FPS |
| Resolution | 720p | 640×360 | Not officially highlighted as a key metric |
| Consistency/memory | Environment stays consistent for several minutes; visual memory reaches back about one minute | Context window of about 0.9 seconds (roughly 9 frames at 10 FPS) | Uses techniques like Dynamic Noising to suppress error accumulation |
| Model scale/openness | Weights not public | Research demo | 500M parameters, open-source code and weights |
| Generation method | Real-time autoregressive world generation | MaskGIT-based generation | Transformer latent-space diffusion + autoregressive |
2.2 Genie 3: Controllable World Generation for General Environments
Genie 3’s goal is a general world model: users describe a world in text, then jump in and explore it in real time—walk forward and the model generates what’s ahead, turn around and it generates a new viewpoint. It also supports Promptable World Events, where text prompts can change the weather, make objects appear, and trigger other world events. Google also showed off an experiment pairing Genie 3 with the SIMA agent, in which the agent completes goals through navigation actions inside the generated world—a sign that world models could become environments for training agents.
On January 29, 2026, Project Genie, an experimental web app built on Genie 3, opened to Google AI Ultra subscribers aged 18 and over in the US, offering world sketching, exploration, and remixing. According to the official disclosure, each generation session is capped at 60 seconds, character control may lag, the physics won’t necessarily be accurate, and Promptable World Events had not yet been integrated at that point.
2.3 WHAM-RT/WHAMM: Adapting to New Games with Minimal Data
Microsoft’s Muse (World and Human Action Model) was initially trained on Ninja Theory’s Bleeding Edge, using data that included more than 1 billion game images and controller actions—roughly 7 years of continuous gameplay. The real-time version carried the same technique over to Quake II, using only about a week of curated gameplay data for the new game, and reached generation speeds above 10 FPS with resolution bumped up to 640×360.
“One week of data” describes the scale of data used when adapting to a new game—it has nothing to do with training duration or the cost of training from scratch. What matters is this: an existing world model can adapt to a new interactive environment with a relatively small amount of new data, which is an important signal that the data barrier is coming down.
2.4 Oasis: An Open-Source 20 FPS Real-Time Interactive Experiment
Oasis is built from a spatial autoencoder, a Transformer-based latent-space diffusion backbone, autoregressive generation, and user-input conditioning. It supports moving, jumping, picking up items, and breaking blocks. The official real-time constraints are easy to grasp: 20 FPS means 50 milliseconds per frame, and hitting 25 FPS requires generating a frame in roughly 0.04 seconds.
Oasis is a technical demo and has not been licensed by Microsoft or Mojang for Minecraft. The team also notes that further speedups and lower large-scale serving costs will require more efficient inference infrastructure. That point lays bare the cost profile of world models: every frame is one inference, so however long a player plays, the GPU has to keep computing for that long.
3 Why Game Worlds Generated by World Models Go Wrong
3.1 The Concrete Failure Cases WHAMM Publicly Disclosed
Microsoft laid out the model’s limitations in detail in the WHAMM technical article, and this list is the most valuable first-hand evidence for judging the maturity of world models.
| Failure phenomenon | Specific manifestation | Root cause |
|---|---|---|
| Object forgetting | Objects outside the field of view get forgotten by the model | Context is only about 0.9 seconds (9 frames) |
| Enemies vanishing/appearing out of thin air | A player looks down at the ground and then back up, and enemies may disappear or reappear | There’s no persisted object state outside the frame |
| Inaccurate health stats | The health number may change incorrectly | The value is “guessed” from the frame, lacking counting logic |
| Enemy interaction errors | Damage calculation and hit detection have deviations | Rules are approximately learned from data, lacking deterministic resolution |
| Level boundary freeze | Only part of a single Quake II level was trained, and generation may freeze once you reach the end of the training data | Generation range is limited by training data coverage |
| Large-scale latency | Noticeable latency may occur when many users are online at the same time | The service pressure of real-time inference per frame |
Microsoft’s characterization of this is quite accurate: this is a generative model’s approximation of the environment, players are experiencing the model, and the complete experience of the original Quake II is still provided by the original game.
3.2 Consistency Problems Come From the Autoregressive Structure Itself
3.2.1 Memory Length Determines How Long the World Can “Remember”
World models can only predict the future based on the historical frames within the context window. WHAMM’s 0.9-second window means that after an enemy leaves the field of view for about 9 frames, the model loses the basis for its existence. Genie 3 extends visual memory to about a minute and maintains consistency for several minutes, which is already a clear improvement, but games often require that a player come back to the same room hours later and still see the chest that was opened—this demands a memory scale several orders of magnitude higher.
3.2.2 Rules Aim for “Looking Reasonable”
The model’s training objective is to make the next frame look reasonable. Rules that require precise accumulation—such as health deduction, ammo counting, and quest progress—only occupy a few pixels on the screen, and the model easily generates them as “plausible but wrong.” What a single line of hp -= damage in an engine guarantees correctly requires a lot of data and structural design to approximate in a world model.
3.3 Genie 3’s Action Space and Interaction Duration Limits
The limitations officially acknowledged by Genie 3 include: the range of actions an agent can directly perform is limited, complex interactions between multiple independent agents are still difficult, real-world geographic locations can’t be precisely reproduced, text rendering is unstable, and continuous interaction is mainly measured in minutes. A limited action space means complex skill combinations, inventory operations, and dialogue branches are hard to express directly; difficulty with multi-agent interaction directly affects online and crowd AI scenarios.
4 Core Capabilities That Traditional Game Engines Provide
4.1 Engine Engineering Depth: A Look at Unreal Chaos Physics
Unreal Engine’s Chaos Physics system covers rigid body dynamics, destruction systems, networked physics, cloth simulation, ragdoll systems, vehicle physics, fluid simulation, physics animation, and hair simulation. The value of these modules lies in being controllable, repeatable, and debuggable: when the same bullet hits the same wall at the same angle, the fragment calculations can be reproduced, developers can track down issues frame by frame, and networked physics keeps results consistent across multiple machines.
4.2 The Eight Categories of System Responsibility That Game Engines Handle
| Engine System | Primary Role | What World Models Currently Offer |
|---|---|---|
| Scene system | Manages maps, objects, and hierarchy | Implicitly expressed through the image; objects lack independent identity |
| Physics system | Handles collisions, motion, and forces | Approximately learned from data, with limited accuracy |
| Animation system | Manages character skeletons and motions | Image-level generation, making precise skeleton control difficult |
| Networking system | Handles multiplayer synchronization | Complex interactions among multiple agents remain a hard problem |
| Asset system | Manages models, textures, and sound effects | Assets are model weights, hard to swap out individually |
| State management | Saves player, quest, and world state | Constrained by the context window — roughly 0.9 seconds for WHAMM |
| Rendering system | Reliably outputs visuals at a specified quality | On the order of 720p/24 FPS, limited by inference compute |
| Debugging tools | Locates errors and performance bottlenecks | Output is probabilistically generated, making step-by-step breakpoint debugging difficult |
4.3 Determinism Is the Lifeline of Commercial Games
Determinism lets games support recorded replays, automated testing, and reproduction of bugs submitted by players; it’s also the foundation for lockstep multiplayer and anti-cheat. World model output carries randomness — the same input may produce different results. That’s a strength for creative exploration, but an obstacle for commercial games that need fair competition, save/load, and balance tuning. Seen this way, the difference between engines and world models isn’t image quality — it’s whether results can be promised.

5 Integrating World Models with Traditional Game Engines: Hybrid Architecture
5.1 Division of Labor Principles for Hybrid Architectures
The core principle of a hybrid architecture is: anything that needs to be “committed” goes to the engine, and anything that needs “diversity” and “generation” goes to the model. The engine holds authoritative state (health, inventory, quests, coordinates, multiplayer data), while the model handles dynamic environments, content generation, NPC behavior, and creative assistance. The model’s output must be validated by the engine before it can be written into state. Saltline already demonstrates one variant of this approach: a Large Language Model generates all the code during development, and at runtime everything is executed by deterministic programs.

5.2 A Minimal Example of Driving NPC Behavior with an LLM
The example below demonstrates the most practical piece of the hybrid architecture: the engine holds the authoritative state of the NPC, while the Large Language Model is only responsible for “proposing” the next action. The engine then validates the proposal against a whitelist and numeric boundaries before executing it. The interface uses the OpenAI-compatible format.
import json, openai
client = openai.OpenAI(api_key="YOUR_API_KEY", base_url="https://api.apiyi.com/v1")
state = {"npc": "港口守卫", "hp": 80, "player_distance": 12, "alert": False}
prompt = f"根据状态给出NPC下一步行为,只输出JSON,action取值 patrol/chase/talk:{json.dumps(state, ensure_ascii=False)}"
resp = client.chat.completions.create(
model="YOUR_MODEL_NAME", # Select an available model in the console
messages=[{"role": "user", "content": prompt}],
)
print(resp.choices[0].message.content)
View the full implementation: the NPC decision loop where the engine validates and the model proposes
import json
import openai
client = openai.OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.apiyi.com/v1", # OpenAI-compatible interface, so you can switch models to compare results
)
ALLOWED_ACTIONS = {"patrol", "chase", "talk", "retreat"}
class EngineState:
"""Authoritative state on the engine side: all numeric values are modified only here"""
def __init__(self):
self.npc = {"name": "港口守卫", "hp": 80, "max_hp": 100, "alert": False}
self.player_distance = 12.0
def snapshot(self) -> dict:
return {"npc": self.npc, "player_distance": self.player_distance}
def propose_action(state: dict, model: str) -> dict:
"""Let the model propose an action; the model can only read state, not change it directly"""
system = (
"你是游戏NPC的决策模块。只输出一个JSON对象,"
'格式:{"action": "patrol|chase|talk|retreat", "line": "一句台词"}'
)
resp = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": system},
{"role": "user", "content": json.dumps(state, ensure_ascii=False)},
],
max_tokens=200,
)
text = resp.choices[0].message.content.strip()
try:
return json.loads(text)
except json.JSONDecodeError:
return {"action": "patrol", "line": ""} # Fall back to a safe default action when parsing fails
def validate_and_apply(engine: EngineState, proposal: dict) -> str:
"""Engine validation: whitelist + rule constraints; reject invalid model output"""
action = proposal.get("action")
if action not in ALLOWED_ACTIONS:
action = "patrol"
if action == "chase" and engine.npc["hp"] < 20:
action = "retreat" # Rules are decided by the engine; chasing is forbidden at low HP
if action in {"chase", "retreat"}:
engine.npc["alert"] = True
line = str(proposal.get("line", ""))[:60] # Limit line length to prevent UI overflow
return f"{engine.npc['name']} -> {action}:{line}"
if __name__ == "__main__":
engine = EngineState()
proposal = propose_action(engine.snapshot(), model="YOUR_MODEL_NAME")
print(validate_and_apply(engine, proposal))
This pattern contrasts with the WHAMM failure case: HP and alert state are stored by the engine, so even if the model “forgets,” enemies won’t vanish into thin air; when the model outputs an illegal action, the engine falls back to safe behavior.
🚀 Quick start: When building an NPC decision prototype, you can use APIYI at apiyi.com to call OpenAI, Claude, Gemini and other models with the same API key. New users get free test credits, making it easy to compare line quality, response latency, and JSON output stability across different models.
5.3 Scenario Recommendations: When to Choose a World Model, and When to Stick with a Game Engine
5.3.1 Scenarios That Suit World Models
- Concept validation and inspiration exploration: Quickly generate a walkable world from text to validate art direction and atmosphere — Project Genie’s sketch and remix features are built for exactly this kind of need.
- Agent training environments: The Genie 3 and SIMA experiments show that generative worlds can supply AI agents with large numbers of diverse training scenarios.
- Short interactive experiences: Minute-scale interactive demos, marketing experiences, and research presentations all fit within the current consistency ceiling of a few minutes.
5.3.2 Scenarios That Must Rely on a Game Engine
- Games with save files and long-term progression: Quests, inventories, and progression stats all require state persistence.
- Competitive and multiplayer games: These need deterministic resolution, multiplayer synchronization, and anti-cheat.
- Games driven by physics-based gameplay: Destruction, vehicles, cloth, and similar mechanics depend on debuggable systems like Chaos Physics.
- Commercial projects that need live operations: Testing, profiling, and patch updates all depend on the engine toolchain.
5.3.3 Scenarios That Suit a Hybrid Architecture
- Dynamic content in open worlds: The engine handles terrain and rules, while the model generates side events, NPC dialogue, and environmental descriptions.
- AI-assisted development: As with Saltline, a Large Language Model generates code, shaders, and level logic during development, while the runtime is handed off to deterministic programs.
5.4 Decision Recommendations: Judging Maturity by Technical Metrics
| Dimension | Current Level of World Models (Official Disclosure) | Typical Requirements of Commercial Games | Verdict |
|---|---|---|---|
| Frame rate | 10 to 24 FPS | Usually 30 FPS or above; action games aim for 60 FPS | Engine wins |
| Resolution | 640×360 to 720p | 1080p and above | Engine wins |
| Continuous consistency | 0.9-second context up to several minutes | Tens of hours of play and save files | Engine wins |
| Content generation speed | Instant world generation from a text prompt | Traditional pipelines require modeling and level production | World model wins |
| Runtime cost | One GPU inference per frame | Player’s local hardware rendering | Engine is cheaper |
| Adoption approach | Research prototypes, experimental products | Ready for live operations | Hybrid architecture |
💡 Selection advice: If your team wants to bring AI into an existing engine first, the safest entry point is “verifiable generated content” such as NPC dialogue, quest text, and level descriptions. You can use APIYI apiyi.com to test multiple models in parallel through a unified interface, choose based on output quality and per-call cost, and then decide on the integration approach.
6 World Models vs. Game Engines: FAQ
Q1: What is an AI world model, and how does it differ from a video generation model?
A world model takes the current environment state and the user’s actions, then predicts and generates the environment for the next moment. A video generation model outputs a fixed clip that the user can’t intervene in midway. A world model, by contrast, receives new input at every frame, so users can continuously direct its movement and actions. Genie 3 is a representative example of this kind of model.
Q2: What’s the fundamental difference between a world model generating a game world in real time and a traditional game engine?
An engine computes state according to explicit rules, then renders that state into images, and the results are reproducible. A world model learns visual dynamics from data and generates images directly, so its results carry an element of randomness. Engines lead in determinism, state persistence, physical accuracy, and multiplayer synchronization, while world models have the edge in content generation speed and a lower barrier to creation.
Q3: When were Genie 3, WHAM-RT, and Oasis announced? Can I try them now?
Oasis was announced in October 2024, with open-source code, a 500M-parameter model weight release, and an online demo. Muse was announced in February 2025, and its real-time Quake II version followed shortly after. Genie 3 launched on August 5, 2025. Project Genie, built on top of it, opened on January 29, 2026 to Google AI Ultra subscribers in the US aged 18 and over, with each session capped at 60 seconds.
Q4: Where do game teams’ AI costs mainly go right now?
It depends on which stage the AI is working in. When a Large Language Model generates code during development, as with Saltline, the cost is concentrated in one-off API calls—the developer’s own estimate was around $550—and after that, players run the game locally without calling the model at all. World models do per-frame inference at runtime, so every play session keeps racking up GPU costs. For runtime features like NPC dialogue powered by a Large Language Model, you pay per call, and you can start with small-scale testing through APIYI apiyi.com’s pay-as-you-go billing to estimate the per-player call cost.
Q5: How do I quickly add AI-driven NPCs to an existing game prototype?
I recommend the “model proposes, engine verifies” pattern from Section 5.2 of this article:
1. Register at APIYI apiyi.com and get an API key plus free trial credits;
2. Set the sample code’s base_url to `https://api.apiyi.com/v1` and fill in an available model name;
3. Serialize the NPC state from your engine into JSON, send it to the model, and only accept actions from a whitelist;
4. Switch between different models to compare latency and output stability, then settle on your production setup.
Q6: What known limitations do world models have right now?
WHAMM’s context is about 0.9 seconds, it forgets objects outside its field of view, there are errors in health tracking and damage calculation, and generation may freeze once it reaches the boundaries of its training data; Genie 3’s continuous interaction is measured in a few minutes, its action space is limited, and complex multi-agent interaction remains difficult; Oasis officially states that it needs more efficient inference infrastructure to speed things up and lower serving costs. All of these limitations point to the same conclusion: for now, world models work well as a content layer, but the rules layer still needs an engine.
7 World Models vs Game Engines: Key Takeaways
- Metric positioning: Genie 3 reaches 720p/24 FPS, WHAM-RT delivers 640×360 at over 10 FPS, and Oasis hits 20 FPS.
- Memory weakness: WHAMM has a context of roughly 0.9 seconds — enemies and objects outside the view get forgotten.
- Rule weakness: Precise rules like health values and damage resolution are approximated from pixels, which makes them error-prone.
- Engine value: Physics, state persistence, multiplayer sync, and debugging tools form the foundation of commercial games.
- Two approaches: Saltline has AI write code while the program runs; world models generate frames one by one.
- Where they converge: The engine owns authoritative state, while the model handles proposals for environments, content, and NPC behavior.
- Getting started: Begin with NPC dialogue that can be validated, and use APIYI apiyi.com’s unified interface to compare multiple models.
8 Conclusion: Where Traditional Game Engines Stand in the Era of AI World Models
AI world models can already generate interactive game worlds in real time. The three representative models push frame rates to 10–24 FPS and resolution to 720p, while the data needed to transfer to a new game has dropped to about a week. At the same time, problems like WHAMM’s object forgetting, incorrect health values, and frozen levels show that what the model generates is an approximation of the game experience — rules, state, and online play still need the engine to guarantee them.
The main thread over the next few years will be hybrid architecture: the engine holds down determinism, the model expands the scale and variety of content, and large language model code generation during development keeps compressing prototype production cycles. For development teams, the most practical first step is to plug AI into the parts of an existing engine that can be validated, then evaluate results and costs using real invocation data. I recommend validating quickly through APIYI apiyi.com — the platform offers a unified interface for multiple models plus free credits for new users.
📚 References
- Google DeepMind – Genie 3 Technical Release: Genie 3’s frame rate, resolution, consistency, and limitations
- Link:
deepmind.google/blog/genie-3-a-new-frontier-for-world-models -
Description: Primary source for Genie 3 metrics and officially acknowledged limitations
-
Google – Project Genie Official Announcement: An experimental web app built on Genie 3
- Link:
blog.google/innovation-and-ai/models-and-research/google-deepmind/project-genie -
Description: Availability scope and the 60-second experience limit
-
Microsoft Research – WHAMM Real-Time World Modeling: Quake II real-time demo and its limitations
- Link:
microsoft.com/en-us/research/articles/whamm-real-time-world-modelling-of-interactive-environments -
Description: Source for failure cases such as 0.9-second context and object forgetting
-
Microsoft Research – Muse / WHAM Project: World and human action models
- Link:
microsoft.com/en-us/research/project/wham/wham -
Description: The scale of Bleeding Edge training data
-
Decart – Oasis Project Page: Open-source real-time interactive world model
- Link:
oasis-model.github.io -
Description: 500M parameters, 20 FPS, and architecture details
-
Epic Games – Physics in Unreal Engine: Chaos Physics feature overview
- Link:
dev.epicgames.com/documentation/unreal-engine -
Description: The capability scope of a mature engine physics system
-
Saltline Code Mirror and README: The developer’s own notes on data attribution
- Link:
github.com/realcodebear/saltline-fable-5.2 - Description: A reference for file sizes and the AI-generated code approach
Author: APIYI Technical Team
Technical Exchange: Want to compare notes on AI integration for game development? Reach out to us via the APIYI apiyi.com homepage — we’re happy to share what we’ve learned.