Inside Level 4: World Models, VLA Policies and the Sim2Real Gap

Level 4 of the Physical AI Ladder describes systems that perceive, reason and act autonomously on the real world. It is the state of the art of the sector and it is explained here in the third person, with its actual technical parts: what a world model is, what a vision-language-action policy is, why the gap between simulation and reality is a distribution problem and not a photorealism problem, where inference has to live and what separates a documented pilot from production.

World models versus predictive models

At levels 2 and 3 of the ladder the work is done with predictive models. A predictive model is a function that maps a feature vector to a bounded output: probability of a bearing failing in seven days, forecast consumption for the shift, class of the defect in an image. Three properties define it: it does not represent the dynamics of the process but a correlation between input and output in the observed regime; it contains no actions, so it cannot answer what would happen under a different decision; and it is validated offline with an error metric on a held-out test set.

A world model is something else. It encodes the dynamics of the environment: given a latent state and an action, what the next latent state is and which observation is expected, with its uncertainty. The reference definition in the sector is in the Cosmos platform paper NVIDIA published on arXiv on 7 January 2025 (identifier 2501.03575), where a world foundation model is described as a general-purpose world model that can be fine-tuned into customized world models. On 31 May 2026 the company announced Cosmos 3, with a mixture-of-transformers architecture and the weights of the Super and Nano variants published on Hugging Face under the OpenMDW-1.1 licence.

What it adds over a predictor is three concrete capabilities:

• Imagined trajectories: rolling forward the effect of a sequence of actions and comparing alternatives that were never executed.
• Training without touching the plant: planning inside the model, which makes learning viable when every real trial costs money, line time or risk.
• Counterfactuals: expressing what would have happened under a different decision, something a predictor cannot do because its formulation contains no action.

From that follows the consequence that matters most to a factory: a world model needs action data, not only observations. A history with process variables but no record of interventions is purely observational data, and with it you learn what usually happens, not what happens if you intervene. That is why the supervised closed loop, level 3, is a prerequisite for level 4.

The known limit of this family is compounding error: when predictions are chained, error accumulates, and a policy optimised against an imperfect model tends to exploit its flaws instead of solving the task. Practice compensates with short horizons, recalibration with real data and penalties in regions where the model has little evidence.

Vision-language-action policies and how they are trained

A policy is a function that turns the estimated state into an action, subject to declared constraints and oriented towards a declared objective. A vision-language-action policy is a particular case: it takes visual observations, proprioception and a natural language instruction, and emits low-level actions directly, such as pose increments or joint commands. The term is fixed by RT-2, published by Google DeepMind on arXiv on 28 July 2023 (identifier 2307.15818), whose contribution was to treat actions as text tokens and incorporate them into the training set of a vision-language model, so that pretrained knowledge transfers to control. That explains why they generalise to objects and to task variants described in words, instead of requiring a trajectory programmed per product.

The dual-system architecture

Implementations in the sector converge on separating two speeds. NVIDIA describes it in the Isaac GR00T N1 paper published on arXiv on 18 March 2025 (identifier 2503.14734): a System 2 vision-language module in charge of semantic reasoning, and a System 1 that is a diffusion transformer generating the actions. Figure AI published on 20 February 2025 the frequencies of its Helix model, and they are the most useful figure for reasoning about latency: the reactive System 1 runs at 200 Hz, one cycle every 5 milliseconds, and the semantic System 2 at 7 to 9 Hz. The split answers a physical fact: reasoning about what to do and executing a stable movement have timing requirements that differ by more than an order of magnitude.

Where the training data comes from

• Real teleoperation. A human drives the robot and the trajectories are recorded. It is the most expensive data and the most valuable, because it contains the real dynamics. The open reference dataset is Open X-Embodiment, published on arXiv on 13 October 2023 (identifier 2310.08864): 22 robots, 21 institutions and 527 demonstrated skills, aggregated from 60 earlier datasets. Figure stated around 500 hours of teleoperation for Helix.
• Human video and web data. It brings a variety of scenes, objects and semantic context that no robot fleet produces, at the cost of containing neither actions nor forces.
• Synthetic data. Generated in simulation or with world models. It covers rare cases cheaply and inherits the errors of the model that produced it.

The GR00T N1 paper states the recipe as a heterogeneous mixture of real-robot trajectories, human videos and synthetically generated datasets. Physical Intelligence reached the same place along another path with pi-0.5, published on arXiv on 22 April 2025 (identifier 2504.16054), which uses co-training on heterogeneous tasks to achieve open-world generalisation.

Imitation first, reinforcement afterwards

Base training is imitation learning over triples of observation, instruction and demonstrated action. It is stable and cheap to validate, but it has a ceiling: the policy does not surpass the demonstrator and degrades as soon as it strays from the states the demonstrator visited. Reinforcement learning comes next, to improve success rate and throughput on real tasks; in the chronology published by Physical Intelligence on its blog (consulted on 9 September 2026), that is the stated contribution of pi*0.6, from November 2025, within a sequence running from pi-0 in October 2024 to pi-0.7 in April 2026. The third line is adaptation from few examples: Google DeepMind separated action from reasoning in Gemini Robotics 1.5, on 25 September 2025, and in Gemini Robotics 2, announced on 30 July 2026, states adaptation to a new morphology in just a few hours and, typically, with fewer than 200 examples, plus a variant that runs on the device itself.

The limits that matter in a factory are four:

• the inference frequency bounds the achievable control bandwidth, so they do not replace the millisecond loop but rest on top of it;
• the output is probabilistic, so underneath it there has to be a deterministic safety envelope that does not depend on the model;
• out-of-distribution behaviour is hard to bound formally, which clashes with the certification of a cell shared with people;
• they do not solve the problem of knowing what state the plant is in: they consume perception and turn it into movement within a bounded task.

The sim2real gap is a distribution problem

The popular explanation says simulation fails for lack of visual realism. The academic review published on arXiv on 23 October 2025 (identifier 2510.20808), included in the 2026 Annual Review of Control, Robotics, and Autonomous Systems, puts it differently: the gap comes from the abstractions and approximations that inevitably introduce discrepancies between simulated and real environments. Every simulator is a model and therefore imperfect by construction.

In learning terms: a policy trained in simulation optimises expected performance under the distribution of states and observations the simulator induces, and on deployment it meets a different one. The aggravating factor is specific to control and does not appear in a classification problem: the policy determines which states it visits. A small error takes it to states poorly represented in training, where it makes a larger error, which takes it further still. The shift feeds back on itself.

The sources of that shift are several, and only one of them is visual:

• poorly identified dynamic parameters: friction, inertias, backlash, elasticity, delays and actuator saturation;
• sensor noise, resolution and latency, which in simulation are modelled as ideal;
• contact, friction and deformation, the worst modelled part of almost any simulator;
• and, last, appearance: lighting, textures and reflections.

The corollary is uncomfortable for anyone buying on the strength of a demo: a photorealistic simulator with the wrong friction still fails. The techniques that narrow the gap attack all four sources. Domain randomisation trains over a family of simulators with randomised parameters so that reality falls inside that family; the canonical formulation is Tobin and others on arXiv, from 20 March 2017 (identifier 1703.06907), transferring simulated images to real ones by randomising rendering, an idea later generalised to dynamic parameters. Synthetic data at scale is the use the NVIDIA glossary (consulted in September 2026) attributes to world foundation models. Identification with real data measures on the machine the parameters the simulator was guessing, to centre that family where the system actually is. And fine-tuning with real episodes adjusts the policy in the target environment.

The last three require measured plant data. This is where level 1 of the ladder returns: a poorly characterised delay is learned as a dynamic that does not exist. Timestamp quality and synchronisation across modalities are not an ingestion detail, they are a condition for the model to be correct.

What plant history does and does not provide

It provides four things and not a fifth, and they are worth separating carefully.

• Perception. Observational history serves to detect anomalies, classify defects and estimate variables that are not measured. It is the ground of predictive maintenance and of AI visual inspection, and it does not require leaving level 2.
• Identification of dynamics. It allows fitting the parameters of a process model in the regime the plant has actually covered. It says nothing about regimes never visited, and that silence gets mistaken for evidence.
• Validation. It is a bench of real scenarios and a source of rare cases nobody would have thought to simulate. It also calibrates the simulator by comparing real and simulated trajectories under the same inputs.
• Bounding the authority of the system. It defines the normal operating envelope and makes it possible to detect when the system is out of distribution and must hand over control. Without that envelope there is no criterion for when the model should not decide.
• What it does not provide: learning a policy. An observational history contains neither the alternative actions nor their outcomes. Without a counterfactual you cannot evaluate a decision that was not taken. The only way to generate that data is to run a supervised closed loop and record, per action, which data triggered it, which rule or model version produced it, what was written, what the equipment answered and which effect was measured afterwards.

There is also an underestimated quality requirement. At level 4 the problem stops being volume and becomes synchronisation across modalities: a policy that associates image and force learns the wrong correlation if the two streams are out of phase, and the error is invisible in the training metric. Completeness also changes nature: it is no longer a percentage of samples, it is coverage of the distribution. Ten thousand repetitions of the normal case do not cover the rare case that decides whether the system is deployable.

Latency and compute: why inference lives at the edge

The loop closes over a real machine, and that machine sets the clock. The diagram chains the four stages with the latency each stretch tolerates.

The order of magnitude is documented. The 5G-ACIA and ZVEI white paper on the integration of industrial Ethernet networks with 5G networks, from November 2019, puts short communication cycle times in industrial periodic communication in the range of 10 ms to 1 ms and 0.5 ms. The same document sets out that working clock domains, where robots and motion control live, require time synchronisation of 1 microsecond or better, tending towards 100 nanoseconds. The standard that makes it possible is IEEE 1588-2019, approved on 7 November 2019 and published on 16 June 2020 by the IEEE Standards Association.

Against those figures it is clear why inference cannot live in the cloud. A 200 Hz cycle has 5 milliseconds to capture, infer, write and check, and a round trip to a data centre consumes that entire budget in the best case. The decisive argument, though, is not the mean delay but its variance: a loop with variable delay cannot be tuned, it is either slowed down to stay stable or it oscillates. A non-deterministic network offers no upper bound. Hence the usual allocation of compute:

1. Reactive loop, in the cabinet or onboard. With no dependency on any external network and with defined safe behaviour if the upper layer disappears.
2. Semantic reasoning, at the plant edge. At 7 or 9 Hz there is room for a local accelerator and not for leaving the site. The open humanoid robot reference design NVIDIA presented on 31 May 2026 for academic research is explicit: it combines a chassis, hands and an onboard Jetson AGX Thor compute module. Along the same lines, Gemini Robotics 2 has included since 30 July 2026 a variant that runs on the device.
3. Training and evaluation, on a platform or in the cloud. Never in the critical path of a fast decision. That is where the world model lives when it is used to generate data or to plan without touching the plant.

The boundary most often violated is the first one: closing a millisecond loop through a platform. It does not fail for lack of bandwidth, it fails through delay variance, and the symptom gets blamed on the algorithm when the problem is transport. The concept sits in the edge AI glossary entry.

The real state of industrial deployments

Telling trial, pilot and production apart is what press releases blur. The table gathers the documented industrial deployments with the status the source itself declares.

Three readings hold up on that table: every task is logistics manipulation or fixture loading, not process work; performance figures are almost always published by the vendor and not by an independent third party; and no source describes a manufacturing line operated autonomously end to end.

The financial contrast calibrates expectations better than any adjective. Figure AI closed a Series C of more than 1 billion dollars on 16 September 2025 at a post-money valuation of 39 billion. Physical Intelligence raised 600 million at a valuation of 5.6 billion, according to The Robot Report of 25 November 2025, and Skild AI 1.4 billion at a valuation above 14 billion, according to the same outlet on 15 January 2026. The only humanoid maker with public accounts, because of its SPAC listing, reported 1.8 million dollars of revenue and a 140 million dollar operating loss in 2025, according to The Robot Report of 7 September 2026.

Meanwhile, conventional industrial robotics remains the real volume. According to the International Federation of Robotics, in its note of 25 September 2025 on World Robotics 2025, 542,000 industrial robots were installed worldwide in 2024 and the operational stock reached 4,664,000 units, with an average world robot density in manufacturing of 177 robots per 10,000 employees.

Why level 4 presupposes levels 1 to 3

Physical AI is AI that perceives, reasons and acts on the real world. In a factory it starts by capturing the data of the equipment that already exists properly. That is not a stylistic preference: it follows from everything above, because each level of the ladder produces something level 4 consumes.

• Without level 1 there is no reliable perception. Without change of state capture, source timestamping and synchronised clocks, delays are learned as dynamics and the model is wrong by construction.
• Without level 2 there is no ground truth. Without defined, comparable indicators there is no criterion against which to evaluate whether the policy improves anything.
• Without level 3 there are no action and effect pairs. Without a record of what was done and what happened next there is no intervention data, no counterfactual and no drift metric.

The failure mode that follows from skipping this is recognisable and expensive: the demo works with the demo part and the demo lighting, and in production, with variability of material, light and product, the system degrades. It cannot be corrected, because correcting requires the record of observation, action and outcome that level 3 produces, and nobody detects the degradation because measuring the effect is also a level 3 capability. The full framework is in the guide Physical AI in manufacturing: what it is, what it is not and why it starts with data and summarised in the physical AI entry.

What a Spanish factory should do today

The starting point has been measured. According to Metalindustria, on 22 May 2026, reporting on the third Barometer of industrial digitalisation and automation in Spain presented at Advanced Factories 2026, only 3.3% of factories describe themselves as fully digitalised. According to the Spanish statistics office INE, in data published on 22 October 2025, 21.1% of companies with 10 or more employees use artificial intelligence, and Fundación Cotec puts industry at 17.5% on the same date. In robotics the figure is better: according to the executive summary of IFR World Robotics 2025, from September 2025, Spain installed 5,086 industrial robots in 2024 and overtook France to become the third largest European market. The local ecosystem is organising too: Automática e Instrumentación reported on 4 September 2026 the creation of the ÁNIMA association, driven by AFM Cluster and AER Automation.

Against that picture, the sensible move is not to buy level 4 but to build what level 4 consumes. Five measures, all executable with the existing equipment, plus one on the calendar:

1. Capture on change of state, not by slow polling. Machine states with one second resolution or better: an eight second micro-stop disappears if you poll every thirty.
2. Timestamp at source, with synchronised clocks. Never stamp at ingestion. Without a common clock the history lets you count, but not explain what happened first.
3. Context on every value. Stable asset identity, quantity and unit, scale, data quality and the version of the signal definition. The structured approach is in the unified namespace and in the guide to industrial data contextualisation.
4. Record actions and effects, not only observations. It is the only thing that turns a history into training material for a policy. The concept sits in the closed loop entry.
5. Keep the episodes that went wrong. At level 4 the value of the data is in the tail of the distribution: failures, recoveries and manual interventions.
6. Plan the regulatory framework with dates. The Machinery Regulation (EU) 2023/1230 applies from 20 January 2027, according to the EU-OSHA record. The ISO 10218-1:2025 and ISO 10218-2:2025 standards, published in February 2025, replace the 2011 editions and absorb the content of ISO/TS 15066 on collaborative operation, according to The Robot Report of 18 February 2025. And according to the European Commission timeline consulted on 9 September 2026, general application of the AI Act starts on 2 August 2026 and the high-risk rules for AI embedded in Annex I products, a category that includes machinery, apply from 2 August 2028. To that add the ISA/IEC 62443 series for the cybersecurity of automation and control systems.

None of them requires buying a robot. All of them produce value on their own at levels 1 to 3, which is where we work, and they are the ones that leave a plant able to evaluate level 4 on its own terms when the time comes.

Frequently asked questions about level 4

To place a specific plant on the ladder, the digital maturity test returns the level and the next step in ten questions, and the industrial data platform describes the capture, context and closed-loop architecture everything above rests on. If you want to review what data your equipment produces today and what would be missing to support a model, write to us.