aifollow.news
Back Rohan Paul
Rohan Paul· @rohanpaul_ai · X· · Original publication time AI score54

MBZUAI’s Institute of Foundation Models releases PAN 2 world model for physical intelligenceMachine translation

AI introduction

PAN 2 is designed to let an agent simulate what might happen before acting. An LLM describes events, their order and camera movement in text; a 29B diffusion model trained from scratch turns that text into visuals. IFM reports that, in its tests, the full model got 75.1% of steps right and completed 34.0% of plans. Without the LLM, step accuracy fell to 8.7%, and no plan was completed.

Original post · Original

The article text is unavailable in this language; an existing version is shown.

@rohanpaul_ai

MBZUAI's Institute of Foundation Models (IFM) just released PAN 2, an open foundation world model for physical intelligence.

It's built towards simulative reasoning: letting an agent play out what happens before it acts.

It works with two kinds of reasoning, each with its own job:

> Conceptual: an LLM works out what happens, when, and how the camera moves, then writes it down as text. > Perceptual: a 29B diffusion model, trained from scratch, turns that text into what you see.

Take the LLM out, and multi-step tasks fall apart: 8.7% of steps right, no plan finished. Getting the order of actions right is the LLM's job.

With both, PAN 2 gets 75.1% of steps right and finishes 34.0% of plans in full, the highest among the models IFM tested. Every step has to hold for a plan to work.

More details in the technical blog:

https://t.co/l6xKPmKT9k

[1/1]

Source: Rohan Paul · x.com

Model
Found an error? Send a correction