Gabriel Petersson, a researcher with OpenAI and Midjourney experience, argues that coding tools should embed AI agents capable of clicking through interfaces on every pull request to surface obvious UI and functional bugs before they reach reviewers.
every PR will obviously come with 100% coverage of AI app testing, that tries every button in the interface to make sure it works as expected
why are the coding apps not making AI testing first class feature, 80% of problems are obvious for AI if it tries the app itself
A reply notes that Cursor developers run their own AI editor against the product during feature work, creating a self-testing loop that matches the spirit of Petersson’s proposal, though no performance numbers or scope details have surfaced.
Petersson frames the 80 percent catch rate as a qualitative hunch rather than measured data, leaving open whether existing agents can reliably navigate complex UIs without added overhead or false positives.
Positive users praise built-in AI testing features in coding apps as a big win for reliable end-to-end development, while negative users call current implementations like screenshot loops exhausting and ineffective.
No Digg Deeper questions have been answered for this story yet.
using my codex to take screenshots of my screen with my electron app as feedback loop which works ok but it's just so stupid
every PR will obviously come with 100% coverage of AI app testing, that tries every button in the interface to make sure it works as expected
why are the coding apps not making AI testing first class feature, 80% of problems are obvious for AI if it tries the app itself
and no, no external app with fix this i just want this to be built in, this is how every feature should be developed end to end
also put more effort into perception and understanding interfaces this will be enormous alpha
using my codex to take screenshots of my screen with my electron app as feedback loop which works ok but it's just so stupid
@gabriel1 this is what we do at cursor using cursor to test cursor when building cursor
every PR will obviously come with 100% coverage of AI app testing, that tries every button in the interface to make sure it works as expected
why are the coding apps not making AI testing first class feature, 80% of problems are obvious for AI if it tries the app itself
@gabriel1 with automations you can trigger computer use. so we both use it to repro issues as well as verifying solutions
@ericzakariasson 🦦🦦🤯🤯🤯🤯🇺🇸🇺🇸 where
I think the testing frameworks are still not there. AI still sucks at playwright. Agents can get stuck in a loop trying to write UI tests just for it to not actually validate anything useful… I nuked our Playwright suite cause it provided 0 value and just increased iteration loop…
@gabriel1 Someone is gonna acquire @tryrevyl for exactly this reason
@ericzakariasson 🦦🦦🤯🤯🤯🤯🇺🇸🇺🇸 where
@gabriel1 I give my game projects basically a cli control interface that can do anything in the game, makes codex 100x better at correctness on first pass
@gabriel1 Codex does this well through computer use actually! You just need to ask it to test, just like a real dev
@gabriel1 Just a random thought. I would imagine working at one if the top labs means u get insider and early access to flagship models without token restrictions etc
If u quit and have to be a normal person, it probably feels like how we lost access to fable
@gabriel1 p slow
what you need is a state space explorer that just explores your app universe constantly. not a really slow ass ai
@gabriel1 True, I've been working on systematic loops to make feature and code testing with regression and trying different ways to make it fail so AI can fix it at @anymo_ai
I feel companies would love shipping reliable features with detailed reports
YES!! I think this is where proper routing/orchestration *could* be most useful (if you’re worried about token efficiency/speed). Like, you’re an LLM dude, BOIL THE OCEAN here. Oh it’s gonna take a million tokens and 5 minutes? I don’t care!!
If I’m still doing this much handholding in 2027 I’m gonna lose it…
@ericzakariasson @gabriel1 He said "first class" Maybe a feature that does this as an automatic loop would be useful (burns a lot of tokens though)
With agent orchestration it's possible, trying to build it in my harness
@gabriel1 I gave my electron app dev only hooks codex can use to control many aspects of the app directly
big win
@gabriel1 But I’ve seen that it struggles with mapping things properly though… like functionally it will pass but a button is like off center or fucked by the padding and such.. that’s been my biggest challenge even with the playwright MCP.
@gabriel1 cc @thsottiaux . I do wish codex had better regression testing. i’ve tried so many different ways to get Codex to test and look at things before it hands me the final product
@gabriel1 BTW I noticed that GLM-5.2 loves doing that (though it doesn’t have vision, but it’ll write itself multiple layers of harnesses to test the UI).
@gabriel1 Because its bad and slow and the environments are annoying
@gabriel1 Because it takes time to write the skills for the model to reliability perform the test. And then you have to run the models across different environments to gather data. Finally you post-train the model. All of it takes time.