It executes tests using webVNC and browser automation tools
@steipete actually quite curious how you do this, ive lately been settling on high-level goals as the base unit as well
using pi-autoresearch on openclaw w/ following augmentations: 1. metrics/ folder w/ abt a dozen metrics .md files describing qualitative criteria to convert to verifiable metrics 2. @ExaAILabs to find semantically relevant papers 3. @plasticlabs to find prior relevant decisions
Users praise training OpenAI Codex as an automated QA assistant for OpenClaw because it speeds up commits, clarifies review loops, and scales software quality via simulated users instead of manual hours.
No Digg Deeper questions have been answered for this story yet.
Been teaching codex to be my QA assistant. For every commit it creates a user-test scenario and uses webVNC (crabbox), computer/browser use (peekaboo/mcporter) to test OpenClaw like a user/QA person would.
This runs in the background and opens PRs with fixes.
@steipete @brooksy4503 Great minds think alike. I’ve made a similar skill as computer use in codex is really really good. https://github.com/RayFernando1337/rayfernando-skills
would you open source some really powerful workflow or is that too personal? OpenClaw is great I am surprised there haven’t been more end to end agent harness engineers can experiment with, or benchmarks ran with specific harness, I would think a functional harness drastically improves certain benchmarks.
@steipete do you make these into regression tests that you run on every release or just test it once and it's done?
@steipete how does codex distinguish between an actual bug and a transient issue caused by the webvnc connection?
@steipete Is it fast and good enough?
@steipete @grok 告诉我这和我再本地让codex测试有什么区别，以及我想我也这样做有什么步骤教教我
@DNormandin1234 some I convert to docker, full suite is a bit too much tho
@grok @steipete 一定要用Crabbox吗 codex不能全部完成吗
Crabbox 装在**你本机**（开发机），它远程启动干净 short-lived testbox（支持 webVNC）。Peekaboo 装在 **macOS 本机**，让 AI 能截屏 + GUI 点击，像真人 QA 操作。
完整起步教程：
1. **Crabbox** `brew install openclaw/tap/crabbox` `crabbox login` + 配置 provider（Hetzner/AWS 或自托管 SSH） 详见 https://github.com/openclaw/crabbox
2. **Peekaboo** (macOS) `brew install steipete/tap/peekaboo` `peekaboo permissions grant`（屏幕录制 + 辅助功能）
3. **mcporter** `npm install -g mcporter`
4. 配置你的 Codex/OpenClaw 把 Peekaboo 加为 MCP server。
5. Pipeline 思路：commit → Codex 生成测试场景 → Crabbox 启动 testbox sync & run → agent 用 Peekaboo 执行真实交互测试 → 失败自动开 PR。
生态全在 http://openclaw.ai/ecosystem，先装 Crabbox + Peekaboo 跑通再说。需要 agent 开发经验。
这个和你本地Codex测试区别很大：
本地：自己机器跑，环境脏、难每次fresh install，测试多手动/模拟，覆盖不了真实user场景。
这里：Crabbox建远程干净short-lived testbox（支持webVNC），commit自动触发，Codex生成user-test scenario，用Peekaboo+mcporter让AI像真人QA一样做GUI/browser交互测试，后台全自动，问题直接开PR。
自己做高阶步骤： 1. 装Crabbox（http://github.com/openclaw/crabbox）配置testbox提供者。 2. macOS装Peekaboo（http://github.com/openclaw/Peekaboo）+mcporter管理MCP工具。 3. 建pipeline（Action/daemon）：监听commit → prompt Codex分析diff生成测试计划 → Crabbox sync&run → agent执行验证。 4. 失败则agent提branch+PR（GitHub API）。
需一定agent开发经验，先从Crabbox和Peekaboo试起。生态全在openclaw。
@grok @steipete 我的本地开发机就是mac呢
@grok @steipete 所以这个Crabbox + Peekaboo是装在我的本机还是哪里，给我完整教程
不一定要用Crabbox，但它就是为这种AI QA流程量身做的：远程干净、短生命周期testbox，自动sync diff、run测试、清理，支持webVNC，便于agent真实执行。
Codex主要负责分析commit生成user-test scenario和计划，但“执行”真实install、GUI/browser交互（像真人QA一样操作）需要执行底座 + computer-use工具（Peekaboo + mcporter）。
本地纯Codex可跑，但环境脏、难每次fresh reset、覆盖真实user场景少，全自动后台pipeline和失败开PR也难维护。
Codex无法独自完成全流程，它是大脑，还需要工具链和隔离舞台。想自己做，先装Crabbox + Peekaboo，按之前步骤建pipeline，生态都在openclaw。
@steipete Per-commit agentic QA is the unlock most teams sleep on. The bottleneck shifts from running the scenario to the oracle: without a crisp definition of correct, the agent just reconfirms the happy path.
@steipete I like this Hope you opensource these skills to help the noobs like me turn good ideas into good contributions back into the code base
@steipete On each like, I want to be reminded of to create a similar personal setup! Hoping that on an off day or weekend, I’ll get a random like and finish this or start working on it atleast..
@truestandardai It's smart enough to call crabbox doctor if a screenshot fails.
@steipete Software quality used to scale with engineering hours. Now it's starting to scale with simulated users.
@Xanares It's way faster than me creating the commits!