Positive users express excitement about testing direct code execution in place of LLM tool calling from first principles, while negative users dismiss tool calling as unnecessary overhead with added serialization and token costs.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.
I would love a company with LLM training chops (ahem ahem) to put this to the test From first principles optimal seems something like this - input tokens are just “events” - things that happened in the world - output tokens are code - code runs in a repl so you get to reference variables from previous turns - the output language supports streaming data types and is parsed on the fly - so you can even stream token-by-token to (in our case) itx.slack.getThreadStream() or sth It would pin inference to a repl session but with websockets inference APIs, realtime voice models etc there increasingly is some pinning of non-gpu workloads around the inference anyway
@eisokant tool calling is just function pointers with extra serialization and a per-token bill
Been saying a version of this for 2 years now. We should give each model its own micro VM with its own code base and just stop doing tool calling all together. Models are more effective and efficient with code.
@eisokant Yeah imo this is the way to go. Tool calling is a hack.
Positive users express excitement about testing direct code execution in place of LLM tool calling from first principles, while negative users dismiss tool calling as unnecessary overhead with added serialization and token costs.
Based on 2 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.