Another banger analysis from @liangc_science that's a follow up to his BMS/Opdivo PDL1 historical wrong decision piece that was reproduced by agents. This time, 10 drug program decisions submitted to 2 AI agent teams (Opus 4.6/7, GPT5.4/5).
Findings:
- Agents in these cases are, fundamentally, consensus reasoners
- Opus and GPT come to the same conclusions in 9 of the 10 decisions
- Didn't rescue the cases where a contrarian call in the absence of robust data would have scored a major win
How to best leverage AI agents for decision making:
- Use AI as a check, not a decider
- Lean on AI hardest where the decision turns on data interrogation
- Don’t expect contrarian conviction
- The framework provided to the agents matters more than the model
Makes me wonder how we can harness the models to become more contrarians. Maybe something a la Open Collider from @cdriclion? The flip side being: there might still be a role for great founders/execs for a little longer.
https://liangchang.substack.com?utm_source=navbar&utm_medium=web
