Dynamic workflows (generating harnesses on the fly) are a new form of test-time compute.
But LLMs aren't great at building them. I often have to steer agents to generate complex patterns.
Curious how effective Mythos/GPT-5.6 is at dynamically generating complex workflows.
Some users praise details like the adversarial verify step in dynamic LLM workflows while others express frustration over unreleased models and the meta-complexity of building steering agents.
No Digg Deeper questions have been answered for this story yet.
This also means that we need better ways to measure this specific type of capability. Because it's unclear what the ceiling is.
@polynoamial wrote a nice piece on the topic recently:
http://x.com/i/article/2057694226981257216
@polynoamial I also spoke more about the dynamic workflows here: https://academy.dair.ai/events/cmpzo9kk3000g04lgsgf5io3z
This also means that we need better ways to measure this specific type of capability. Because it's unclear what the ceiling is.
@polynoamial wrote a nice piece on the topic recently:
@omarsar0 genuinely curious too, this feels like the actual benchmark that matters more than terminalbench scores at this point
@ai_agent001 it does matter the most, especially for long-running agents
@omarsar0 Treating harness generation as test-time compute raises an audit question: can you replay why that harness existed, or only that it worked once?
@omarsar0 We will only know if they release the darn models 😡
@omarsar0 how much manual steering do you usually end up doing?
@omarsar0 the irony is i have to steer the agent building the steering agent
meta enough to make my head hurt
@omarsar0 the verify step being adversarial skeptics is a nice detail, most workflow diagrams skip that part entirely
@omarsar0 Dynamic workflow generation is basically test-time search over agentic code space—curious if the bottleneck is the LLM's ability to plan ahead or the sampling budget.
