Announcement

SuperMimic reportedly beats SONIC's benchmark return after three hours on one GPU

The SuperMimic team reports more than 1,000 times fewer GPU-hours, though the runs used different hardware and training data.

20 Sources, ,

TLDR

Qiayuan Liao and the SuperMimic team report that their humanoid motion tracker exceeded SONIC's released checkpoint in normalized return across 129,514 motions after three hours on one H100. They calculate more than 1,000 times fewer GPU-hours than SONIC's seven-day run on 128 L40s. The runs used different hardware and amounts of training data, and the benchmark measures coverage of training motions, not generalization.

Today's Rank

—

Not ranked yet

Combined views

255K

20 Sources, first seen 20d ago

What the compute claim measures

According to the team's comparison, SONIC's run used 128 NVIDIA L40 GPUs for seven days, or about 21,000 GPU-hours. SuperMimic used one NVIDIA H100 for three hours, or three GPU-hours. Those figures underpin the team's claim of more than 1,000 times fewer GPU-hours, but they are not a same-hardware comparison. The three hours include collecting simulator transitions, fitting a world model and updating the policy; checkpoint evaluations are excluded.

The researchers describe fitting a differentiable world model to transitions from the MuJoCo Warp simulator and updating that model alongside the policy. They train the policy through 32-step imagined rollouts in the learned model, rather than passing gradients through the simulator. A critic estimates returns beyond those rollouts.

What the benchmark does and doesn't show

The runs also used different amounts of motion data: SONIC trained on 611 hours, while SuperMimic used 264.56 hours of the public BONES-SEED subset. The team notes that actual peak-hardware utilization is unknown. Both methods were tested on motions from the filtered BONES-SEED training set, so the researchers describe the result as a measure of coverage, not generalization to unseen motions. The benchmark used deterministic physics, foot friction of 1.0, and no domain randomization or observation noise.

Liao reported 179 failed motions for SONIC and 25 for SuperMimic after three hours on one GPU. The project page defines a failure as a rollout triggering SuperMimic's training termination. The team's zero-failure result came after 9.8 hours on two GPUs, not the three-hour run.