The update integrates FSDP2 with deep-ep expert parallelism.
Today we're releasing prime-rl v0.6.0 — enabling RL at trillion-parameter MoE scale on agentic workloads at the highest efficiency.
We've relentlessly optimized our RL infra.
The result: GLM-5 on agentic SWE tasks at 131k context and sub-5-minute step time.
Users are praising Prime-RL v0.6.0 for enabling efficient trillion-parameter MoE RL training with standout gains like sub-5min steps and major KL reductions.
No Digg Deeper questions have been answered for this story yet.
prime-rl is fully open source, and we're hiring systems engineers to take it further.
Read the full prime-rl performance deep dive:
https://www.primeintellect.ai/blog/rl-at-1t-scale/
Highly-recommended read.
It's exciting to see large-scale agentic RL becoming more accessible. Cool to see the infra layer for this is being built and I think this plays an important role in self-improving agents arc and "owning your AI."
every infra piece you need to know to do RL on GLM-5
https://www.primeintellect.ai/blog/rl-at-1t-scale
prime-rl can now train 1T parameters MoE blazingly fast, under 5 minutes per step, or 1k steps in ~3 days
To achieve this we shipped in our latest prime-rl 0.6.0:
* inference: wide-ep, fp8 inference, llm-d router, mooncake, kv cache cpu offloading
* training: fsdp2, deep-ep expert parallelism, dsa cp, fp8 training, router replay
* agentic rollout: we rewrote the core of our rollout orchestrator for better scalability
support for glm5, kimi, nemotron, ...,
prime-rl is open source but also end to end optimized to run on our dedicated RL infra and compute layer
Today we're releasing prime-rl v0.6.0 — enabling RL at trillion-parameter MoE scale on agentic workloads at the highest efficiency.
We've relentlessly optimized our RL infra.
The result: GLM-5 on agentic SWE tasks at 131k context and sub-5-minute step time.
> 3D-parallel (FSDP2 + CP + EP)
This is, at least conceptually, my favorite sharding. I wonder if at some point during development you also tried and timed others (like PP or TP with EP) and how they compared, even just informally, or if you went straight for this one and only this one and just optimized it a lot?
We disaggregate prefill and decode onto separate workers.
A long prefill used to stall decode for everyone. Now it doesn't.
One Mooncake store pools KV cache across all nodes, so any worker can reuse any prefix.
The router picks workers by a score over load, queue depth, KV usage and prefix overlap. You get cross-replica cache hits with balanced routing across the whole deployment.
In RL, inference is the bottleneck — we optimize for throughput, not latency.
High concurrency, FP8 precision, and wide expert parallelism over 32+ GPUs. Every GPU holds its own slice of experts and acts as its own endpoint.
Over a long run the trainer and inference policies slowly drift apart, and that mismatch can kill your training.
R3 (router replay) captures the routing decisions from the inference engine, replays them on the trainer - KL mismatch drops ~10x.
Huge thanks to the @vllm_project team, and @robertshaw21 in particular, for all the help along the way.
Also to the llm-d and Dynamo teams for the collaboration on routing and inference.
The trainer is 3D-parallel (FSDP2 + CP + EP), built on TorchTitan.
FSDP2 shards params, grads & optimizer state. EP keeps experts sharded and routes tokens with all2all instead of all-gathering ~80GB per layer. CP handles the 131k context and GLM-5's DSA attention.
@PrimeIntellect why yall using torchtitan and not JAX?
@giffmana @PrimeIntellect It was kind of both - TP/PP in torch is somewhat miserable so I try to avoid it as much as I can. CP has the advantage that with DSA it is almost communication free here, so it is kinda a no-brainer in GLM case, in other cases it was mostly from previous experience
@PatrickToulme @PrimeIntellect my proposal for jax rewrite got outvoted (1:everyone else)
@m_sirovatka @PatrickToulme @PrimeIntellect I didn't know cat counted as 1
@PrimeIntellect @m_sirovatka cooked 🔥
@m_sirovatka @PrimeIntellect > TP/PP in torch is somewhat miserable so I try to avoid it as much as I can
Literally the motivation behind my question lol
@m_sirovatka @PatrickToulme @PrimeIntellect literally 1984
@giffmana @PrimeIntellect this goes pretty far imo, not sure we will ever need TP for moe, maybe pp at some point but rn defo not worth the hassle
all three of them are very simple to implement together and maintain
@PrimeIntellect how in the world did they convince @m_sirovatka to write
@samsja19 1k steps in 3 days at 1T scale and that's the good news
