Xavier Gonzalez releases a paper mapping Newton iterations to recurrent neural networks, Picard iterations to diffusion models, and Jacobi iterations to parallel LLM decoding

Fixed point iterations for parallelizing nonlinear dynamics is all the rage: - Newton for RNNs - Picard for diffusion models - Jacobi for parallel decode of LLMs But how do these techniques relate, and when should you use them? We show you how in our new paper 🧵