SIMPACT aims to turn simulated actions into real-world robot moves

The SIMPACT team says its method starts by creating a physics simulator from a real-world scene. An agent then proposes actions and refines them using simulation results before they are executed in the real world. The project site says a vision-language model helps optimize the action sequence.