The upcoming work expands on Artzi's existing arXiv research
@LChoshen @LChoshen , I would love to hear more about what you are doing and thinking about
@LChoshen This one of our efforts in along these lines: https://arxiv.org/abs/2505.15962
We have a big step beyond this that will go on arXiv in a few days, which I am pretty excited about
@yoavartzi I am soon moving to the new lab, so there's a lot of thinking (and a big branch should be pretraining), but it is also already active. I think the most concrete pretraining challenge is With context
@yoavartzi There's also the thought of what tricks are people doing post hoc, what if we pretrain? e.g. speculative decoding is ~ deepseek's next token prediction trick. Thinking and instructions there are paper that show it. What else do we do only at the end?
@yoavartzi This is cool, share or tag me when the new thing is out
@LChoshen This one of our efforts in along these lines: https://arxiv.org/abs/2505.15962
We have a big step beyond this that will go on arXiv in a few days, which I am pretty excited about
The upcoming work expands on Artzi's existing arXiv research
@LChoshen @LChoshen , I would love to hear more about what you are doing and thinking about
@LChoshen This one of our efforts in along these lines: https://arxiv.org/abs/2505.15962
We have a big step beyond this that will go on arXiv in a few days, which I am pretty excited about