Joël Niklaus Highlights Speculative Programmatic Tool Calling
Hugging Face engineer describes sPTC parsing and pre-launching tool calls during streaming.
Joël Niklaus of Harness Optimization at Hugging Face posted on X praising Speculative Programmatic Tool Calling. He called the idea neat and clean. The post states that most agent harnesses wait for a model to finish generating a REPL program before executing its tool calls. sPTC parses the program while it streams, pre-launches safe calls in a shadow REPL, and reuses the result if the final program makes that call. The tweet includes a 56-second terminal screen recording of a side-by-side demo UI comparing the speculative approach on the left with the standard approach on the right.
Combined views
7.7K
3 posts, first seen 1d ago

