Evaluation of 30 frontier embodied AI models finds generalist robot policies struggle with real-world manipulation · Digg

/TECH

Evaluation of 30 frontier embodied AI models finds generalist robot policies struggle with real-world manipulation

Researchers created the RoboDojo framework to address the gaps

Combined views

14.5K

5 posts, first seen 19h ago

71 likes

Evaluation of 30 frontier embodied AI models finds generalist robot policies struggle with real-world manipulation

Commentary on X

Highest ranked

Ashish Verma@AshishSuperVer1TECH

@MarioChan2002 First of all its really exciting and awesome. 😃 Any plans to expand Tasks and evaluation dimensions(Dynamic Environments, Collaborative Tasks, etc.) as this is what a Real Environments actually looks like.

Combined views

14.5K

5 posts, first seen 19h ago

71 likes

5 comments

57 saves

59 reposts

Sentiment

Positive100%0%Negative

Positive Read

Users praised the benchmark evaluation of frontier embodied AI models as impressive and timely work that usefully identifies real-world manipulation failures and failure patterns.

Based on 11 visible X reactions from 11 accounts; directional sample.

Useful Links

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies(RoboDojo)viaEric Jang

Digg Deeper

Ask a question below.

Published answers will appear here.

Commentary on X

Highest ranked

Ashish Verma@AshishSuperVer1TECH

Yuejiang Liu@liu_yuejiangTECH

@MarioChan2002 Very timely benchmark for the field!

Tianxing Chen@MarioChan2002TECH

@MilaidyNow Looking forward to your exploration!

Shilong Liu@Shilong_Liu_AITECH

@MarioChan2002 Impressive work!

View all

Chris Paxton@chris_j_paxtonTECH

Honestly its amazing pi-0.5 is still going so strong on leaderboards a year later. The best model here is Hy-embodied VLA, open source from the Tencent Robotics team https://x.com/chris_j_paxton/status/2075376104249106807/photo/1 https://twitter.com/MarioChan2002/status/2075072783667920967