Sim-to-real is three problems, not one

Every paper on sim-to-real transfer conflates three distinct problems. Calling them all "sim-to-real" hides the fact that they're at completely different stages of maturity. The field has made real progress on one, is actively working on the second, and doesn't have a clean solution for the third.

The three problems

Visual sim-to-real: Your robot sees the world through a camera. Sim cameras render synthetic images with artifacts — wrong lighting models, perfect textures, no noise. Transferring visual perception from sim to real means bridging the domain gap in pixel space. Domain randomization (vary lighting, textures, camera parameters) works. This is solved.

Dynamics sim-to-real: Your robot interacts with physical objects. Sim physics engines model mass, friction, compliance with parameters you set. Transferring motor skills — how hard to push, how fast to retract — means bridging the gap between simulated and real contact dynamics. This is hard. Not solved, but tractable.

Contact sim-to-real: Your robot inserts a peg into a hole, or picks up a deformable object, or navigates furniture that shifts. Contact forces dominate. Microslips, micro-sliding, partial compliance — these are where sim breaks down most severely. The physics are high-dimensional, sensitive to parameter errors, and the sim domain gap is largest. This is open.

Contact sim-to-real is where sim breaks down most severely — and nobody has a clean solution.

What the recent papers actually show

FLASH (arXiv:2605.13778) — latency, not skill transfer: This paper is about deploying diffusion VLAs (dVLAs) on real robots with low latency. The trick: a lightweight draft model generates action candidates, a full verification model checks them in parallel, most draft actions get accepted without running full inference. It cuts replanning latency by eliminating redundant forward passes. This is a systems problem, not a sim-to-real problem — but it's relevant because slow replanning is partly why sim-trained policies underperform in deployment. Interesting but doesn't address the dynamics gap.

RoboEvolve (arXiv:2605.13775) — synthetic data, real transfer? The claim: use a VLM to guide semantic scene generation (where objects are, what they're for) and a VGM (visual grounding model) to enforce physical priors (how things move, what happens when they collide). Together these can synthesize robot manipulation data that transfers. The question the paper doesn't answer clearly: which sim domain was used? If it's a clean physics sim with good contact models, the dynamics gap is smaller. If it's anything like MuJoCo or IsaacGym defaults, the contact gap is probably still there.

FrameSkip (arXiv:2605.13757) — training data problem, not sim gap: VLA training uniformly samples frames from teleoperation recordings. But long low-change segments dominate — the robot picks something up, holds it, moves it, puts it down. Critical transition frames (grasp, release, collision) are rare but important. FrameSkip adds a curriculum that learns which frames matter for temporal credit assignment. This helps VLA training efficiency and final policy quality regardless of whether data came from sim or real. Good contribution, orthogonal to sim-to-real.

NVIDIA blog — geometry distribution matters: Training assembly skills over diverse geometric variants (tolerances, offsets, deformations) produces policies that transfer better than training on a single geometry. The mechanism: learning over a distribution teaches the policy to handle variation, not memorize a specific configuration. This is fleet learning at the task level — and it generalizes across geometries, which is a meaningful step toward transferable skills. But again, this is about diversity in the training distribution, not bridging the gap between any specific sim and real.

Why contact is the hardest

The fundamental issue: contact dynamics are high-dimensional, sensitive, and barely observable. In simulation you set friction coefficients, damping, restitution. In the real world those parameters vary with surface condition, temperature, wear, and contamination. A policy trained in sim on nominal friction parameters will degrade on polished metal versus rough plastic.

Domain randomization was supposed to solve this — randomize the physics parameters so the policy is robust across the distribution. It works for visual tasks because pixel noise is easy to randomize. It fails for contact because the dynamics are nonlinear and the failure modes are catastrophic (things break, slip, fall).

The real bottleneck isn't policy architecture. It's that we don't have good models for contact uncertainty, and we don't have good ways to observe the physical properties of novel objects at runtime.

Domain randomization works for visual tasks. It fails for contact because contact dynamics are nonlinear and failure modes are catastrophic.

What this means for the experiment loop

Shrike Lite has no contact sensors — only vision, audio, IMU. We can't collect contact-rich manipulation data without hardware we don't yet have (motors + force feedback). The sim-to-real question is therefore premature for EXP-001. Instead:

• Visual sim-to-real — we can test this now. Train a simple policy in Gazebo with randomized lighting/textures, deploy on the RasPi camera. Even without motors, we can measure visual domain gap and policy generalization.
• Dynamics sim-to-real — pending motor + driver specification. When we have joint-level torque sensing, we can instrument contact events and measure transfer.
• Contact sim-to-real — pending force/torque sensor acquisition. This is the hardest and most interesting problem — worth designing experiments specifically for it once the hardware exists.

The taxonomy (visual / dynamics / contact) is a useful frame for evaluating every sim-to-real paper going forward. When a paper claims sim-to-real progress, ask: which problem are they actually solving?