Working with agents should feel like working with a colleague. You should be able “speak to” them not just with text chats, but by gesturing at a screen together, talking live, etc.
Users point and draw on screen to modify interfaces
Working with agents should feel like working with a colleague. You should be able “speak to” them not just with text chats, but by gesturing at a screen together, talking live, etc.
Many users are excited about Cursor AI's Design Mode for point draw and voice or chat UI edits because it makes the tool cleaner and more capable, while a few object to AI replacing human work or note missing features like the SDK.
@mntruell 💯
Working with agents should feel like working with a colleague. You should be able “speak to” them not just with text chats, but by gesturing at a screen together, talking live, etc.
designing in code is now as easy as: click, chat, hold shift to multi-select.
works best with Composer 2.5
design mode must know shortcuts:
Toggle: ⌘⇧D
Tab → next sibling Shift+Tab → prev sibling Enter → first child Shift+Enter → parent
Add to chat: ⌘L
Drag → box/scribble Shift+click → multi-select Alt+click → add to chat
Working with agents should feel like working with a colleague. You should be able “speak to” them not just with text chats, but by gesturing at a screen together, talking live, etc.

@elonmusk @mntruell Grok Agents

@mntruell Try Grok Agents

@elonmusk @mntruell Are you bullish on the $SPCX IPO @elonmusk
I know I am - cant wait for it!

@ericzakariasson can I build cursor with cursor? pt. 2

@elonmusk @mntruell 78K7QEC9HXgUoT7U19LRKHSJaZt6eMPDGMmxJJHNpump

@mntruell

@elonmusk @mntruell What do you think about QNX

@mntruell @elonmusk 👍

@mntruell @elonmusk Moving past just typing text into a box makes the whole process feel way more natural.

@elonmusk @mntruell The xAI Cursor partnership is turning out to be amazing. Things are only accelerating!

@mntruell @elonmusk We're still using AI like it's email.
The future probably looks more like Zoom.

@elonmusk @mntruell 💯

@mntruell What version of cursor is this? Mine looks nothing like this nor does design mode work like this. 🙃

@HarshitKhemani yes

That points toward a much more natural “co-working” model for AI agents 🤝
Instead of treating agents like tools you prompt, it’s closer to a shared workspace where you can point, highlight, ask in real time, and correct instantly—almost like screen-sharing with a teammate. That would reduce friction a lot compared to back-and-forth text and make complex tasks (design, coding, planning) feel more fluid and collaborative.
The big challenge is making that interaction precise and reliable—so the agent understands gestures, context on-screen, and live conversation without misinterpreting intent.

@mntruell This is why I'd love to see a duplex voice mode in Cursor. Back and forth conversation with an orchestrator that manages agents under the hood.