Meta’s Muse took roughly seven months to reach launch, and product chief Mona Sarantakos says much of the work went beyond making the agent useful. In a November 15 episode of Lenny’s Podcast, she recalled having a largely functioning product by spring, then spending substantial time on reliability, safety and security. Source Source
The stakes weren’t limited to a bad answer. Sarantakos described Muse as Meta’s personal AI agent, able to use personal context and take actions, including buying things on a user’s behalf. That gives “it misunderstood me” a rather more expensive range of possibilities. Source
The Raspberry Pi weekend
Sarantakos, VP of product at Meta Superintelligence Labs, traced the idea to Nat Friedman encouraging colleagues to try OpenClaw around late January or early February. Unable to find a Mac Mini, she and her husband set it up on a Raspberry Pi. She connected her email, calendar and home automation, then spent the weekend experimenting. Source
By Monday morning, she said, the agent had assembled a roundup of what to expect that week from sources including her children’s school information. Getting that useful snapshot with very little prompting convinced her to pursue the idea with Friedman. The motivating breakthrough wasn’t some grand science-fiction task. It was getting ahead of the school newsletter. Source
But that experiment wasn’t Muse running on a Raspberry Pi. It was the inspiration for a product that, she said, would need to be much safer, easier to set up and capable of serving a consumer audience. Source
A browser brought more than browsing
Sarantakos said the team initially concentrated on figuring out the product itself. Later, adding capabilities created fresh security work: Muse didn’t have a browser at first, and introducing one opened up new problems to address. Existing industry benchmarks didn’t feel sufficient for an agent handling personal context and purchases, she said. Source
The team’s internal phrase was “trust is a must,” Sarantakos said. She described the risk of an early launch with too many problems: “People will not trust this agent. And if you don’t trust the agent, you’re never going to use it.” Source
Development continued as employees and early users outside Meta tested Muse, Sarantakos said, with feedback coming in throughout the summer. Source
Even the agent team has human homework
The testing framework she described covers safety, capabilities, behavior and priority use cases. That includes tests for prompt injection, or attempts to hijack the agent with malicious instructions. Others check browser use, task completion, follow-up questions and understanding what a user wants. She described required tests and others where limited trade-offs are possible. Source
The team also uses groups of AI agents, or “agent swarms,” to examine data and feedback. They can generate artificial examples and help assemble test cases for tasks involving extended back-and-forth conversations. Sarantakos said those tools supplement dashboards with specific examples and insights. Source
Sarantakos said people must review the test data. She said humans must also write the team’s system prompt (the agent’s core instructions) and its specification for how the agent should behave. Even inside an AI-agent team, the AI doesn’t get to write every instruction. Source
The interview provides no independent safety-audit results. Sarantakos acknowledged that the field still has unknowns and that “we’re never going to be perfect.” Source
Comments