The system supports GRPO training and KV-cache decoding.
🤔Can LLMs reason by sampling continuous thoughts — not just tokens?
Introducing NF-CoT: Latent Reasoning with Normalizing Flows. It samples continuous chain-of-thoughts directly in the stream of LLM with exact likelihood -- powered by STARFlow.
🌐Page: http://nf-cot.vercel.app
Excited to share our follow-up work of LaDiR: Latent reasoning with Normalizing Flows (NF-CoT)!
Instead of iterative diffusion denoising, NF-CoT integrates STARFlow to generate continuous thoughts autoregressively in LLMs, like tokens — with exact likelihood, KV-cache-friendly decoding, and compatibility with policy-gradient RL training such as GRPO.
🤔Can LLMs reason by sampling continuous thoughts — not just tokens?
Introducing NF-CoT: Latent Reasoning with Normalizing Flows. It samples continuous chain-of-thoughts directly in the stream of LLM with exact likelihood -- powered by STARFlow.
🌐Page: http://nf-cot.vercel.app
This work was led by two amazing @PennEngineers master’s students, @Guancheng_Tu @EthanFu0355525.
Also huge thanks to our great collaborators @SuhaoYu1020, @tyao923, @haoqik322, @Lianhuiq, and @YizheZhangNLP!
More details: http://arxiv.org/abs/2606.06447 Code&Model: Coming soon.
NF-CoT is also cheaper than LaDiR.
Inference: 1.9× faster end-to-end, with 2.5× fewer FLOPs/sample. Training: 2.85× higher sample throughput, with 6.66× fewer FLOPs.
-> Latent thoughts are generated like tokens — autoregressively, cache-friendly, without iterative denoising.
🧐Why is this hard?
Simply moving CoT beyond tokens is not enough. A useful latent reasoning space hould still keep what makes token CoT powerful: • sampling diverse trajectories • scoring probabilistically • training with likelihood • decoding efficiently with KV caches
🤔Can LLMs reason by sampling continuous thoughts — not just tokens?
Introducing NF-CoT: Latent Reasoning with Normalizing Flows. It samples continuous chain-of-thoughts directly in the stream of LLM with exact likelihood -- powered by STARFlow.
🌐Page: http://nf-cot.vercel.app
Excited to share our follow-up work of LaDiR: Latent reasoning with Normalizing Flows (NF-CoT)!
Instead of iterative diffusion denoising, NF-CoT uses normalizing flows to generate continuous thoughts autoregressively, like tokens — with exact likelihood, KV-cache-friendly decoding, and compatibility with policy-gradient RL training such as GRPO.
🤔Can LLMs reason by sampling continuous thoughts — not just tokens?
Introducing NF-CoT: Latent Reasoning with Normalizing Flows. It samples continuous chain-of-thoughts directly in the stream of LLM with exact likelihood -- powered by STARFlow.
🌐Page: http://nf-cot.vercel.app
Prior latent-CoT methods trade off. For example, • Coconut-style feedback is efficient, but mostly deterministic. • LaDiR-style diffusion latents are stochastic, but iterative and likelihood-intractable.
NF-CoT keeps the sweet spot -- stochasticity + likelihood + efficiency.
🧐Why is this hard?
Simply moving CoT beyond tokens is not enough. A useful latent reasoning space hould still keep what makes token CoT powerful: • sampling diverse trajectories • scoring probabilistically • training with likelihood • decoding efficiently with KV caches
NF-CoT is also cheaper than LaDiR.
Inference: 1.9× faster end-to-end, with 2.5× fewer FLOPs/sample. Training: 2.85× higher sample throughput, with 6.66× fewer FLOPs.
-> Latent thoughts are generated like tokens — autoregressively, cache-friendly, without iterative denoising.
Results on Qwen3-8B-Base across 5 code benchmarks:
Avg pass@1 improves from 55.8 → 68.8 (+13.0), and reaches 70.1 after RL.
NF-CoT also outperforms the strongest latent baseline, LaDiR, by +7.1%. On MBPP+, pass@1 = 72.1 — matching the base model’s pass@128.
💡The idea: place a STARFlow inside LLM!
STARFlow is a SOTA normalizing flow built from Deep-Shallow Autoregressive Transformers.
In NF-CoT, shallow invertible layers map continuous latents into a deep reasoning space, where the LLM models them left-to-right alongside text.
Prior latent-CoT methods trade off. For example, • Coconut-style feedback is efficient, but mostly deterministic. • LaDiR-style diffusion latents are stochastic, but iterative and likelihood-intractable.
NF-CoT keeps the sweet spot -- stochasticity + likelihood + efficiency.
The whole model is trained end-to-end with NLL over both latent thoughts and text answers.
At inference time, it reasons autoregressively in the deep reasoning space, while still allowing latent thoughts to be inspected or decoded for better explainability.
💡The idea: place a STARFlow inside LLM!
STARFlow is a SOTA normalizing flow built from Deep-Shallow Autoregressive Transformers.
In NF-CoT, shallow invertible layers map continuous latents into a deep reasoning space, where the LLM models them left-to-right alongside text.
Moreover, the exact likelihood over both latent reasoning and text answers makes NF-CoT compatible with GRPO-style post-training with verifiable rewards — like explicit CoT, but in the continuous space and pluggable into existing RL frameworks.
The whole model is trained end-to-end with NLL over both latent thoughts and text answers.
At inference time, it reasons autoregressively in the deep reasoning space, while still allowing latent thoughts to be inspected or decoded for better explainability.
Results on Qwen3-8B-Base across 5 code benchmarks:
Avg pass@1 improves from 55.8 → 68.8 (+13.0), and reaches 70.1 after RL.
NF-CoT also outperforms the strongest latent baseline, LaDiR, by +7.1%. On MBPP+, pass@1 = 72.1 — matching the base model’s pass@128.
Moreover, the exact likelihood over both latent reasoning and text answers makes NF-CoT compatible with GRPO-style post-training with verifiable rewards — like explicit CoT, but in the continuous space and pluggable into existing RL frameworks.
The system supports GRPO training and KV-cache decoding.
🤔Can LLMs reason by sampling continuous thoughts — not just tokens?
Introducing NF-CoT: Latent Reasoning with Normalizing Flows. It samples continuous chain-of-thoughts directly in the stream of LLM with exact likelihood -- powered by STARFlow.
🌐Page: http://nf-cot.vercel.app
