Orion AI system deploys computer-using agents to automate laboratory computer tasks and scientific workflows

Original post

I'm very happy to see more AI4Science work that uses CUAs (CUAs4AI4Science?). This is probably one of the highest impact areas for applying CUAs!

Also great to see multi-agent systems help out here on long-horizon, something we also observed in MACU (https://jykoh.com/macu)!

chang ma@ma_chang_nlp

Excited to introduce 🌠Orion: Towards Lab Automation with Computer-Using Agents.

Give it control of your lab computer💻, and it can use software, analyze any experiment images, browse databases on Chrome exactly like you, and work for hours to analyze your experiments.

🌎:https://orion-science.github.io 📎:https://biorxiv.org/content/10.64898/2026.06.13.732095v1

6:27 PM · Jun 17, 2026 · 2.2K Views

chang ma@ma_chang_nlp

[1/n] The idea is simple — scientists perform crucial part s of research on a computer, and so can an AI scientist. Instead of using predefined APIs, we give the agent full control of both the computer screen and the terminal. This unlocks tremendous capabilities.

13h2082

BOOKMARKS1LIKES2

chang ma@ma_chang_nlp

This work is originally done at @Genentech with @StanfordAILab @hkunlp2020.

Grateful to all collaborators !! Hanchen Wang @hcwww_, Aviv Regev, Matt Bucci, Linh Trinh @trinhlinh94.

I am especially grateful to my advisors on this project, Aviv and Hanchen. They reviewed the paper draft word by word countless times, and their thoughtful discussions and edits consistently pushed the work to a higher standard.

12h12221

RETWEETS17

chang ma@ma_chang_nlp

Excited to introduce 🌠Orion: Towards Lab Automation with Computer-Using Agents.

Give it control of your lab computer💻, and it can use software, analyze any experiment images, browse databases on Chrome exactly like you, and work for hours to analyze your experiments.

🌎:https://orion-science.github.io 📎:https://biorxiv.org/content/10.64898/2026.06.13.732095v1

13h9.9K5935

REPLIES1

chang ma@ma_chang_nlp

[n/n] Overall, our work demonstrates the great potential of computer-using agents in biomedical laboratory settings. We found that integrating screen interaction and code execution makes a significant difference. Unlike in computer science, where API-based tools are often feasible, vast amounts of biomedical knowledge are embedded in scientific software and visual data. To enable full-scale laboratory automation, we believe the introduction of such agents is inevitable.

13h98

chang ma@ma_chang_nlp

[2/n] Orion is capable of self-evolution. Given any new software, it could learn with the help of tutorial. After training on a handful of cases, it could master complicated softwares, e.g. QuPath, and sophisticated mouse moves, e.g. draw ROIs.

13h1311

chang ma@ma_chang_nlp

[3/n] Orion’s GUI interface is natively multimodal. Benchmarks across a series of tasks (deep research, microscopy analysis, whole-slide image tumor segmentation) show that it excels.

13h851

chang ma@ma_chang_nlp

[4/n] At the end of the day, we want our agents to draw insights and form hypotheses like scientists 👩‍🔬. We gave Orion a massive set of cell-painting assays to explore, and within 100 hours, it generated 52 reports; human reviewers identified 22 as plausible mechanistic hypotheses. Consider this example: it spotted two subtypes of polyploid morphology in BIRC5 cell imaging and proposed a grounded hypothesis.

13h78