PrimeIntellect introduces Renderers boosting RL throughput over 3x
PrimeIntellect introduced Renderers to resolve mismatches in reinforcement learning pipelines where trainers operate on tokens while environments generate messages. The system enforces explicit token-in and token-out handling with user-controlled templating. LMSYS Org collaborated on the effort. The change eliminates hidden chat-template rewrites and delivers more than 3x throughput gains on popular open models.
#11John Schulman@JOHNSCHULMAN2Glad to see this -- renderers are a foundational component of the LLM stack. Renderers map between tokens and messages, which are invariant to tokenizer and formatting details. Most APIs, datasets, and RL environments are defined in terms of messages. Getting the details wrong leads to train-test mismatches, caching inefficiencies, and prompt injection vulnerabilities. We included a renderers module in Tinker Cookbook, but it makes sense as a standalone library.
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
#64Nathan Lambert@NATOLAMBERTThe jinja chat template has always felt like a temporary equilibrium, so we've needed someone to take the reigns and try to build that out within the community.
Excited about this!
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
Harmony was the first attempt at this imo, but it never broke out of the OpenAI model ecosystem. I'm honestly not sure why, but would guess lack of community effort https://github.com/openai/harmony
Nathan Lambert@natolambertThe jinja chat template has always felt like a temporary equilibrium, so we've needed someone to take the reigns and try to build that out within the community. Excited about this!
@willccbb @vllm_project @sgl_project @huggingface @tinkerapi confirmado
will brown@willccbball chat templates are wrong, some chat templates are useful we found some CRAZY performance wins by patching official templates, and we're shipping them in a standalone library you can use with any RL stack w/ examples for @vllm_project @sgl_project @huggingface @tinkerapi
@willccbb @vllm_project @sgl_project @huggingface @tinkerapi src https://rlhfbook.com/teach/course/lec2-chap4-5-9/#14
Nathan Lambert@natolambert@willccbb @vllm_project @sgl_project @huggingface @tinkerapi confirmado
#120Taco Cohen@TACOCOHENA gift from the Gods. Dealing with multiple models and many envs in the same RL codebase while respecting correctness constraints (no train / inference tokenization mismatch) is becoming a huge pain.
I have a vibe-coded draft PR that does exactly this, but happy I won’t have to land or maintain it now. Let’s hope the field can really standardize on one abstraction.
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
@TacoCohen Very cool! I think tinker from @thinkymachines had that API as well
Taco Cohen@TacoCohenA gift from the Gods. Dealing with multiple models and many envs in the same RL codebase while respecting correctness constraints (no train / inference tokenization mismatch) is becoming a huge pain. I have a vibe-coded draft PR that does exactly this, but happy I won’t have to land or maintain it now. Let’s hope the field can really standardize on one abstraction.
@TacoCohen @hallerite @thinkymachines https://github.com/thinking-machines-lab/tinker-cookbook/tree/main/tinker_cookbook/renderers
http://base.py has the ABCs
some of our fav bugs on the road to `renderers`
read all about it: https://www.primeintellect.ai/blog/renderers
go render some tokens:
will brown@willccbbsome of our fav bugs on the road to `renderers` read all about it: https://www.primeintellect.ai/blog/renderers
#339will brown@WILLCCBBall chat templates are wrong, some chat templates are useful
we found some CRAZY performance wins by patching official templates, and we're shipping them in a standalone library you can use with any RL stack
w/ examples for @vllm_project @sgl_project @huggingface @tinkerapi
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
the core of the issue is that both encoding and parsing are many-to-one
vanilla TITO does prefix lookup in token-space, which misses many rendering collisions
the solution is to do lookup in message-space, then input prep in token-space, which we call bridge_to_next_turn
will brown@willccbball chat templates are wrong, some chat templates are useful we found some CRAZY performance wins by patching official templates, and we're shipping them in a standalone library you can use with any RL stack w/ examples for @vllm_project @sgl_project @huggingface @tinkerapi
@vllm_project @sgl_project @huggingface @tinkerapi we're intending for this to become a programmable source of truth for template implementations so that we can finally get rid of jinja
lots here already, but PRs welcome for all models!
will brown@willccbbthe core of the issue is that both encoding and parsing are many-to-one vanilla TITO does prefix lookup in token-space, which misses many rendering collisions the solution is to do lookup in message-space, then input prep in token-space, which we call bridge_to_next_turn
#339will brown@WILLCCBB@vllm_project @sgl_project @huggingface @tinkerapi from a live run:
#707Vincent Weisser@VINCENTWEISSERWe are open sourcing renderers
For RL, the inference server should be simple Tokens in, tokens out
renderers is the token-level chat templating layer to >render messages to tokens >parse completions to structure >bridge rollouts byte-for-byte > >3x throughput on openmodels
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
#1153Florian Brand@XEOPHONworking at prime is just "ugh i had this gnarly problem, let’s fix it and then make it available to everyone"
a ton of other things are coming, can’t wait to show it to yall :)
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.
#1186Johannes Hagemann@JOHANNES_HAGEnever again
Prime Intellect@PrimeIntellectIntroducing Renderers RL trainers work in tokens. Environments work in messages. Going back and forth corrupts sampled tokens, wasting compute on every agentic turn. With Renderers, we fix this mismatch. This unlocks >3x throughput on popular open models.