Prediction is not understanding, until it is

A model can predict the next frame almost perfectly and still not understand the scene. The loss curve looks the same either way, and that is the problem.

When we train video world models, the objective is almost always some flavor of reconstruction: predict what the sensor will see next. The hope is that to get good at this, the model is forced to learn objects, permanence, contact, and cause. Sometimes that happens. Often the model finds a cheaper path, exploiting texture statistics and camera motion priors that generalize just well enough to keep the loss falling.

The failure only becomes visible under intervention. Ask the model what happens if the cup at the edge of the table is nudged, and the texture-statistics model produces a plausible-looking video where the cup does something physically incoherent, because nothing in its training pressure ever required a concept of "cup" that persists when pushed.

Our position is that the distinction is measurable, and should be measured directly. We evaluate world models not on held-out prediction error but on counterfactual consistency: does the model's imagined outcome of an intervention agree with reality when we actually perform it? On this metric, models with nearly identical validation loss separate by a wide margin.

A model that holds up under intervention is one you can plan with. That is the bar we use.