There is a well-known problem in robotics research that doesn’t get nearly enough coverage in the mainstream accounts of humanoid progress. It goes by the shorthand “sim-to-real transfer,” and it describes something straightforward in principle and frustratingly persistent in practice: a robot trained extensively in a virtual simulation often performs significantly worse when placed in the physical world. Sometimes it fails in ways that seem inexplicable given its simulated performance. Sometimes it performs the task correctly in structured conditions but degrades badly when anything unexpected happens. Sometimes it simply doesn’t work.
Understanding why this gap exists — and how seriously the industry is grappling with it — matters for anyone trying to form an honest view of where humanoid deployment actually stands. The sim-to-real problem is not an obscure research footnote. It is one of the central practical constraints on moving from impressive demonstration to reliable production operation.
Why Simulation Became Central
To understand the gap, it helps to understand why simulation became so central to humanoid training in the first place. The short answer is scale. Training a capable robot policy — the learned set of behaviours that determines how the robot responds to its environment — using only physical robot hardware is prohibitively slow and expensive. Physical robots break. Training runs that involve falls, collisions, or repeated stress on joints accumulate wear. An hour of physical training is an hour of robot time, with all the logistics and maintenance that implies.
Simulation sidesteps most of those constraints. A physics simulator can run faster than real time, operate in parallel across many virtual instances simultaneously, and reset instantly after every failure. A training run that would take weeks on physical hardware can be compressed into hours on a sufficient compute cluster. Robots can be made to fall thousands of times, encounter millions of object configurations, and attempt tasks in countless variations — all without wear, cost, or risk to hardware.
The technique that makes this tractable at scale is called reinforcement learning (RL). In RL, a virtual robot tries a task, receives a reward signal based on how well it performed, and gradually adjusts its behaviour to maximise that reward over millions of trials. When RL is run in simulation with enough compute and a well-designed reward function, it can produce impressive results: robots that walk with surprising agility, manipulate objects with reasonable dexterity, and navigate complex environments. The question is what happens when the policy trained this way is loaded onto a physical robot.
Where the Physics Diverges
The sim-to-real gap has several distinct sources, and they compound each other in ways that make the problem harder than any single cause would suggest.
The most fundamental is what researchers call the “reality gap” in physics modelling. Simulators approximate the physical world, but they do so imperfectly. Friction is a good example. In a simulator, the friction between a robot’s foot and the floor is typically represented by a single coefficient — a number. In reality, friction varies with surface texture, contamination, temperature, and the exact geometry of contact. A robot trained to walk on a simulated surface with a fixed friction coefficient will have developed a gait tuned, implicitly, to that specific value. Put the same robot on a real floor — with slightly different friction, minor surface irregularities, the give of a mat, or the slipperiness left by a cleaning crew — and the gait that performed perfectly in simulation may become unstable.
Joint dynamics are another example. The actuators (motors and associated mechanical components) that move a real robot’s limbs have backlash (small amounts of mechanical play), compliance (slight springiness), and latency (a delay between commanded motion and actual motion). These properties vary unit to unit, change as components wear, and depend on temperature. Simulators model them, but the models are approximations. A controller trained to exploit precise, low-latency joint response may behave erratically on hardware where the actual dynamics differ from the simulated ones by even small amounts.
Contact dynamics — what happens when two objects touch, grip, or collide — are particularly hard to simulate accurately. Grasping an object in simulation is clean; the simulated surfaces interact according to specified contact models. Grasping a real object involves the exact texture, deformability, and weight distribution of that specific object, the compliance of the robot’s fingers, and the precise alignment of the contact at the moment of touch. This is one reason why manipulation tasks that look fluid in simulation — or in a physical demo carefully curated for success — often struggle in real deployment with the variability of actual objects.
The Sensor Gap
Beyond physics, there is a sensor gap. A real robot perceives its environment through physical sensors — cameras, depth sensors, force-torque sensors in its joints and fingertips, inertial measurement units that track its orientation and acceleration. Each of these sensors has noise characteristics, calibration drift, occlusion patterns, and failure modes that are difficult to model precisely in simulation.
Camera images from a simulated environment look different from real camera images in ways that are subtle but significant. The rendering of light, shadow, surface reflectance, and depth cues in simulation does not perfectly match what a physical camera captures. A perception system trained on simulated images may perform well on simulated inputs but fail to correctly interpret real ones — not because the task is harder, but because the input distribution it was trained on doesn’t match what it actually receives.
This is not a hypothetical problem. Autonomous driving research encountered it extensively, and significant engineering effort went into techniques like domain randomisation — deliberately varying the appearance of the simulated environment across many dimensions (lighting, texture, colour, noise) so that the trained system becomes robust to variation rather than tuned to a specific simulated appearance. The same techniques are being applied in humanoid robotics, with partial success. Domain randomisation helps, but it doesn’t eliminate the gap; it shifts the problem from “system trained on wrong distribution” to “system trained on distribution that hopefully overlaps with reality.”
What “Sim-to-Real Transfer” Looks Like in Practice
The practical manifestations of the sim-to-real gap vary depending on the task and the robot. For locomotion — walking, running, climbing stairs — the gap has narrowed considerably over the past several years. Researchers at groups including Carnegie Mellon University, Berkeley, and ETH Zurich have demonstrated legged robots that transfer walking policies from simulation to physical hardware with reasonable reliability. The techniques that made this work include extensive domain randomisation, careful simulator calibration using physical measurements, and training approaches that explicitly teach the controller to be robust to uncertainty rather than to optimise for a specific simulated physics.
For manipulation — grasping, placing, assembling, using tools — the gap remains substantially wider. Manipulation tasks are harder to simulate accurately because they depend on contact physics and sensor fidelity in ways that locomotion, which mostly involves the robot’s own body dynamics, does not. A walking policy that transfers successfully is using the robot’s proprioception (its internal sense of its own joint positions and accelerations) as its primary feedback — and that internal sensing can be calibrated and modelled with reasonable accuracy. A manipulation policy that needs to grasp a real object accurately is depending on perception of the external world and contact dynamics with it, both of which are harder to simulate faithfully.
This distinction has real consequences for humanoid deployment. The tasks that humanoid robots are most commercially interesting for — assembly, packaging, handling varied objects in unstructured environments — are exactly the manipulation-heavy tasks where the sim-to-real gap is most significant. The tasks where transfer works best — walking around, navigating, simple repetitive motions in well-structured environments — are closer to what existing automation already handles reasonably well with non-humanoid systems.
The Industry Response
Companies and research groups working on humanoid robots are not ignoring this problem. The approaches being pursued fall into a few broad categories, each with real promise and real limitations.
The first is better simulation. Nvidia’s Isaac Sim platform, used by several humanoid companies, has invested heavily in improving physics accuracy and rendering fidelity. More accurate simulators reduce the gap at its source. The limitation is that perfect simulation of the real world is not achievable — there is always a residual gap between the most accurate simulation and physical reality, and that residual matters in tasks requiring fine precision.
The second is hybrid training: using simulation to establish broad competence, then using physical experience to fine-tune. The simulated policy provides a starting point that would be impractical to reach through physical training alone; the real-world training corrects for the specific ways the simulation was wrong. This works, but it reintroduces the cost and time constraints that made simulation attractive in the first place — if you need thousands of hours of physical training to correct for sim-to-real errors, the efficiency gains from simulation are partially offset.
The third is learning policies that are explicitly designed to be robust to uncertainty. Rather than training a robot to perform a task in a specific way under specific simulated conditions, this approach trains it to perform the task successfully across a wide range of variations, so that real-world variation falls within the distribution the policy has already learned to handle. This is conceptually appealing and has shown results in locomotion. For fine manipulation, the space of possible real-world variations is large enough that achieving robust coverage through simulation alone remains an open research problem.
The fourth — and the one that has attracted significant commercial attention in recent years — is collecting large amounts of real-world robot data and using it to train or fine-tune models directly. This is part of the motivation behind teleoperation programmes, where human operators pilot robots through tasks while the robot’s sensors record everything. The resulting data captures real physics, real sensors, and real variation. It is expensive to collect at scale, but several companies have concluded that the expense is worth it given the limitations of purely simulated training.
A Gap That Closes Unevenly
The sim-to-real gap is not a single problem with a single solution. It is a collection of related challenges — physics modelling accuracy, sensor fidelity, contact dynamics, domain generalisation — each of which has its own trajectory and its own relationship to the specific tasks being attempted. Some parts of the gap have closed significantly; others remain wide open.
What this means for anyone trying to evaluate humanoid progress honestly is that demonstration performance in a controlled environment is a weak predictor of deployment reliability in an uncontrolled one. A robot that performs a manipulation task impressively in a video — trained in simulation, demonstrated in a curated setting — may or may not perform that task reliably when the objects are slightly different, the lighting is different, the surface is different, or a human is moving nearby. The demonstration tells you something about what the technology can do under favourable conditions. It tells you considerably less about what it will do in the ones that actually matter.
The researchers and engineers who work on this problem every day understand the gap clearly. The coverage of humanoid robotics, which tends to focus on what was shown rather than what wasn’t, often does not. Closing that gap in understanding — between demonstration performance and deployment reliability — is as important as the technical work of closing the sim-to-real gap itself.