Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

AA-Robotics v0.1 launches with eight robot-arm tasks for frontier language models · Digg

Announcement

AA-Robotics v0.1 launches with eight robot-arm tasks for frontier language models

Artificial Analysis says GPT-6 Astra led its eight-task leaderboard, but top models were far slower and costlier than humans.

6 Sources, 1h ago, first seen 1h ago

TLDR

Artificial Analysis launched AA-Robotics v0.1 as a research preview that gives frontier language models control of a six-axis robot arm. In its first eight tasks, models could view four cameras and write code to move the arm. It reports GPT-6 Astra scored 69, ahead of GPT-6.1 Sol at 65, while the best-performing models were roughly 100 times slower and 10 times more expensive than humans.

Combined views

—

6 Sources, first seen 1h ago

Combined views

—

6 Sources, first seen 1h ago

— likes

— likes

— comments

— saves

— reposts

— comments

— saves

— reposts

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

6 Sources

@ArtificialAnlysWelcome to the era of embodied AI. We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today. We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly. GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5. AA-Robotics v0.1 details: ➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected) ➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements ➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot Key findings: ➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail ➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task ➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much ➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive More results, clips, and how AA-Robotics v0.1 works follow in the thread below1h

6 Sources

@ArtificialAnlysWelcome to the era of embodied AI. We are launching AA-Robotics v0.1, our research preview that measures frontier language model performance on understanding, planning, and acting in the physical world. In v0.1, each model is given control of a 6-axis robot arm (5 joints and a gripper) and a simple objective. We’re releasing results for our first eight tasks today. We find that frontier language models in coding agent harnesses have recently become capable of directly manipulating robot arms for simple tasks. Today, they are far too slow and expensive for any kind of production usage - nearly two orders of magnitude worse than human baselines. We expect this to change quickly. GPT-6 Astra places first in AA-Robotics v0.1, followed by GPT-6.1 Sol, Gemini 4 Argon and Claude Opus 5.5. AA-Robotics v0.1 details: ➤ Our first eight tasks test a range of capabilities: control and dexterity (precise grasping and object handling), spatial reasoning (how objects interact with one another) and adaptability (how models respond to the unexpected) ➤ Models get only brief instructions: how to connect to the robot arm (via a simple HTTP API), the arm's name and how to submit an attempt. They can view the scene through four cameras and write and run code to plan and send movements ➤ Once a model submits an attempt, its end state is graded against the task's pass criteria. A pass scores 100; an incomplete attempt earns partial credit. All attempts are zero-shot Key findings: ➤ The GPT family tops the leaderboard: GPT-6 Astra scores 69, ahead of GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. Claude Opus 5.5 (49) and Claude Fable 5.1 (46) trail ➤ Today’s models are incredibly slow at our tasks, taking ~30 minutes on average to complete each tabletop task ➤ Cost varies widely: GPT-6.1 Sol is the cheapest at $1.93 per task on average; GPT-6 Astra, the most expensive, costs $16.36, more than eight times as much ➤ Compared with a human, today's best-performing frontier language models are ~100x slower and ~10x more expensive More results, clips, and how AA-Robotics v0.1 works follow in the thread below1h