Cursor's Sam Whitmore argues AI personal assistants lack the version control and staging mechanisms needed for real-world actions

VIEWS419LIKES7

need auto-sandbox / git for everything

been thinking about why I don't feel comfortable with AI being a true personal assistant still, even though the models have gotten so much better. i realized a lot of the reason for me is actually git / version control.

using an agent that is 99.999999% accurate vs 97% accurate doesn't really matter to me when the errors are high impact. with code, i can yolo a PR & know that its will not be destructive to my existing codebase. there's no similar staging mechanism for my calendar, shopping cart, etc.

in theory you can create calendar event drafts, but very few of the other useful activities (esp spending money) have a draft / easy review paradigm. the solves here will most likely be on the infra / scaffolding level. once you start making a universal shopping cart draft, you're basically inventing a new web browser, or an agent-first wallet / credit card.

3h41970

BOOKMARKS1

Shreya Shankar@sh_reya

A lot of the infra seems like it will have to be physical as well. E.g., i'm happy to let an agent buy clothes for me if (i) it can handle returns without me knowing about it, (ii) my bank balance doesn't go too low/i can instantly get money back that i didn't spend, or i only pay if i actually wear the clothes...it's actually not obvious to me with the right environment/workflow is for easy and instantaneous "rollbacks" in the shopping setting

Sam Whitmore@sjwhitmore

been thinking about why I don't feel comfortable with AI being a true personal assistant still, even though the models have gotten so much better. i realized a lot of the reason for me is actually git / version control.

using an agent that is 99.999999% accurate vs 97% accurate doesn't really matter to me when the errors are high impact. with code, i can yolo a PR & know that its will not be destructive to my existing codebase. there's no similar staging mechanism for my calendar, shopping cart, etc.

in theory you can create calendar event drafts, but very few of the other useful activities (esp spending money) have a draft / easy review paradigm. the solves here will most likely be on the infra / scaffolding level. once you start making a universal shopping cart draft, you're basically inventing a new web browser, or an agent-first wallet / credit card.

3h37641

RETWEETS1

Shreya Shankar@sh_reya

I love the credit card fraud detection example. it is an example of how changes in the physical world/environment ended up making machine learning succeed

Early cc fraud classifiers had a lot of people complaining because of false detections (nobody wants their legit transaction being flagged as fraud + losing cc access as a result). The solution was not to lower the classifiers false detection rate. It was to send users text messages, requiring them to confirm that they indeed wanted to make the transaction classified as fraud

Spam filters have low false detection cost as Sam pointed out

If you don’t have 100% accuracy you need some way to quickly and cheaply correct mistakes

will brown@willccbb

@sjwhitmore counterexample = spam filters / credit card fraud detection

we’ve all got implicitly comfortable with machine learning systems w non-zero error making consequential decisions about the info we see + things we pay for, bc there are proper remediation channels + costs get absorbed

1h15340

REPLIES2

will brown@willccbb

@sjwhitmore counterexample = spam filters / credit card fraud detection

we’ve all got implicitly comfortable with machine learning systems w non-zero error making consequential decisions about the info we see + things we pay for, bc there are proper remediation channels + costs get absorbed

Sam Whitmore@sjwhitmore

been thinking about why I don't feel comfortable with AI being a true personal assistant still, even though the models have gotten so much better. i realized a lot of the reason for me is actually git / version control.

using an agent that is 99.999999% accurate vs 97% accurate doesn't really matter to me when the errors are high impact. with code, i can yolo a PR & know that its will not be destructive to my existing codebase. there's no similar staging mechanism for my calendar, shopping cart, etc.

in theory you can create calendar event drafts, but very few of the other useful activities (esp spending money) have a draft / easy review paradigm. the solves here will most likely be on the infra / scaffolding level. once you start making a universal shopping cart draft, you're basically inventing a new web browser, or an agent-first wallet / credit card.

2h41050

gabriel@gabriel1

@sjwhitmore yes

Sam Whitmore@sjwhitmore

been thinking about why I don't feel comfortable with AI being a true personal assistant still, even though the models have gotten so much better. i realized a lot of the reason for me is actually git / version control.

using an agent that is 99.999999% accurate vs 97% accurate doesn't really matter to me when the errors are high impact. with code, i can yolo a PR & know that its will not be destructive to my existing codebase. there's no similar staging mechanism for my calendar, shopping cart, etc.

in theory you can create calendar event drafts, but very few of the other useful activities (esp spending money) have a draft / easy review paradigm. the solves here will most likely be on the infra / scaffolding level. once you start making a universal shopping cart draft, you're basically inventing a new web browser, or an agent-first wallet / credit card.

54m34440

Sam Whitmore@sjwhitmore

@willccbb makes sense! but i would argue that in some sense that is a good version control system, in that, as you said, there is a remediation channel w low barrier to action & quick reversibility (I can move spam to my inbox & back easily - & easily flag / unflag charges as fraud)

will brown@willccbb

@sjwhitmore counterexample = spam filters / credit card fraud detection

we’ve all got implicitly comfortable with machine learning systems w non-zero error making consequential decisions about the info we see + things we pay for, bc there are proper remediation channels + costs get absorbed

2h15310

will brown@willccbb

@sjwhitmore version control works great for code + many other things :) could totally imagine ppl being willing to auto-merge critical patches from agents + do human review afterwards if merge approval rates are high enough + rollbacks are easy

more stuff should use git

Sam Whitmore@sjwhitmore

@willccbb makes sense! but i would argue that in some sense that is a good version control system, in that, as you said, there is a remediation channel w low barrier to action & quick reversibility (I can move spam to my inbox & back easily - & easily flag / unflag charges as fraud)

1h4310

Hamel Husain@HamelHusain

@sh_reya @willccbb @sjwhitmore Human in the loop thoughtfully is undefeated 😃

Shreya Shankar@sh_reya

I love the credit card fraud detection example. it is an example of how changes in the physical world/environment ended up making machine learning succeed

Early cc fraud classifiers had a lot of people complaining because of false detections (nobody wants their legit transaction being flagged as fraud + losing cc access as a result). The solution was not to lower the classifiers false detection rate. It was to send users text messages, requiring them to confirm that they indeed wanted to make the transaction classified as fraud

Spam filters have low false detection cost as Sam pointed out

If you don’t have 100% accuracy you need some way to quickly and cheaply correct mistakes

1h4430

Sam Whitmore@sjwhitmore

@sh_reya 💯

Shreya Shankar@sh_reya

A lot of the infra seems like it will have to be physical as well. E.g., i'm happy to let an agent buy clothes for me if (i) it can handle returns without me knowing about it, (ii) my bank balance doesn't go too low/i can instantly get money back that i didn't spend, or i only pay if i actually wear the clothes...it's actually not obvious to me with the right environment/workflow is for easy and instantaneous "rollbacks" in the shopping setting

3h16410

Sam Whitmore@sjwhitmore

@willccbb yes!

will brown@willccbb

@sjwhitmore version control works great for code + many other things :) could totally imagine ppl being willing to auto-merge critical patches from agents + do human review afterwards if merge approval rates are high enough + rollbacks are easy

more stuff should use git

1h2310

Ξric Juta@ericjuta

@sjwhitmore auto review HITL with cursor mobile app pls mam

3h62