I love agentic map-reduce, but I don't love Claude's workflows feature. It's hard to steer the workflow, and it's not data-driven enough (e.g., reduce groups are often predefined, not emergent from the map outputs)
HOWEVER...I was inspired by its UI to ship something similar for DocETL. Satisfying way to visualize progress when using the CLI 😆
https://github.com/ucbepic/docetl
Big improvements shipping daily...our new goals are to (i) design a better UI/UX for **humans steering AI agents** to write DocETL queries (ii) rethink query optimization given that agents are the primary issuer of queries
I love agentic map-reduce, but I don't love Claude's workflows feature. It's hard to steer the workflow, and it's not data-driven enough (e.g., reduce groups are often predefined, not emergent from the map outputs)
HOWEVER...I was inspired by its UI to ship something similar for DocETL. Satisfying way to visualize progress when using the CLI 😆
@sh_reya yeah, the hard part is letting the reduce shape come from the data instead of the workflow author. otherwise it is just a prettier DAG with an LLM inside.
I love agentic map-reduce, but I don't love Claude's workflows feature. It's hard to steer the workflow, and it's not data-driven enough (e.g., reduce groups are often predefined, not emergent from the map outputs)
HOWEVER...I was inspired by its UI to ship something similar for DocETL. Satisfying way to visualize progress when using the CLI 😆
