Xinyu2ML questioned the fairness of latent feedback’s inference-cost comparison in a November 14 quote-post. Inference is the work of running a trained model. Source
According to Xinyu2ML, the reported validation loss uses four parallel forward passes, corresponding to roughly 4× the prefill cost of a same-size standard Transformer. Prefill processes the prompt before generation starts. Per-token decoding—the work of generating each new token—remains close to 1× that comparator’s cost, the post says. That is not a license to relabel the result “four times slower”: the post does not establish a fourfold increase in total inference cost or latency. Source
What the training-savings announcement claims
huskydogewoof introduced Latent Feedback Is a Compute and Data Amplifier on November 13. The announcement describes the mechanism as feeding a Transformer’s own top-layer states back into its input, with the aim of getting more from the available training data. Source
On the same data, under a two-epoch budget, the announcement reports reaching the same loss with:
- 15% less training compute at 100 unique tokens per parameter.
- 46% less training compute at 400 unique tokens per parameter. Source
These percentages are not universal discount codes.
Under a two-epoch budget, the announcement labels a 15% compute saving at 271 million parameters as measured, while a fitted scaling law predicts 34% at 10 billion parameters and 45% at 100 billion. Those last two figures are extrapolations, not measurements at those sizes. Source
What the critique leaves open
Xinyu2ML praises the breadth of the comparisons across fixed-data, fixed-epoch and fixed-compute settings, noting that one relies only on scaling-law extrapolation. Still, the post suggests that scaling up a standard model may be the better choice. Source
The supplied alphaXiv page shows the paper’s title and “Loading PDF…” instead of methods or results. It provides no basis for independently checking either the training-compute accounting or the inference-cost comparison. Source
No response from huskydogewoof to this critique is included in the supplied posts, which also don’t establish wall-clock savings, GPU-hour savings or end-to-end serving performance.
Comments