Elon Litman published a PyTorch demonstration of a loss term designed to report zero while producing gradients that change model weights during training. His article includes code and printed output showing the behavior. In an Oct. 21, 2026, post, he called the technique an “(unpatchable?) supply-chain zero-day.”
How the zero-valued loss works
Litman’s example starts with a training loop that imports a third-party UltraLossRegularizer and adds its result to the task loss before calling total_loss.backward() and optimizer.step(). He says the hypothetical attacker need not modify the starting checkpoint, model architecture, dataset or optimizer. The extra regularizer still has to be added to the loop.
To explain the trick, Litman uses (2**30 + u) - 2**30. He writes that, for sufficiently small u, float32 rounding makes the expression evaluate to zero, while PyTorch autograd passes through a gradient of one. In his construction, combining that behavior with another expression produces a scalar loss that reports zero but supplies a chosen gradient. Standard SGD then uses that gradient to move selected weights toward a specified next state.
His two-parameter script prints 0.000000 for the loss over four steps while its printed gradients and states change. Litman also reports adding a Game of Life term to 1,024 weights in a CNN. He writes that the auxiliary loss stayed at zero across 2,345 SGD steps while those weights followed the game’s update rule.
The disputed security label
Litman argues that model weights could serve as memory and SGD steps as a clock for computation during training. In a quote-post, moyix questioned the zero-day characterization: running another party’s regularizer already means running untrusted code, which could tamper with weights directly. In a follow-up, moyix asked whether importing and running attacker-controlled Python had already executed arbitrary code.
Comments