Fully open models release complete training details, not just weights.
Do I understand it correctly that the OLMo from-scratch series is coming to an end?
If so, looks like NVIDIA stepped up just in time with Nemotron models as the only remaining fully-open (ie not just weight drop) from-scratch LLM team.
Positive users praise NVIDIA Nemotron for delivering fully open models with data, recipes and reports while negative users worry OLMo may lose its edge in sharing complete training details.
Two things make me think that:
1) I remember not too long ago some comm's about the new strategy, which sounded much more application/post-train focussed to me (sorry don't remember where, but from ai2)
2) large portion of the team gone
That being said, I'm an OLMo fan for the openness and reasonable quality, so I would be happy if my impression is proven wrong!
@giffmana i'd say don't count olmo out just yet :)
Do I understand it correctly that the OLMo from-scratch series is coming to an end?
If so, looks like NVIDIA stepped up just in time with Nemotron models as the only remaining fully-open (ie not just weight drop) from-scratch LLM team.
@giffmana Haven't looked into it too deeply but isn't stepfun also fully open?
@giffmana Great fully open pretraining efforts going on at Marin!
@giffmana I hope not! I really appreciate NVIDIA’s recent efforts across several areas of open-source AI, but I would hate to see OLMo disappear. It has been such a valuable open-source project, built by an incredibly talented group of people.
@giffmana Arcee?
@giffmana Apertus?🇨🇭 I heard there will be a v2.
@giffmana stay tuned. lots of exciting work underway is all i can say rn, and I think the community will be pleased with the outputs
@giffmana Did you see project Marin?
@giffmana Apertus?
@giffmana K2 from @IFM_MBZUAI
@giffmana They should get Nathan on board
@bygregorr @giffmana Pre and post Data, software, recipes, models (base, post, reward), full technical report
Hard to be more open and Nemotron
@giffmana not sure nemotron clears that bar the same way olmo did. olmo dropped the full pretraining dataset too, does nemotron actually publish the training data or just weights + recipes?
@giffmana 💚
@Laz4rz @giffmana Apertus is open data Arcee isn't (?) as far as I knoe
@fujikanaeda @bygregorr Yep, i double-checked before posting :)
But curiously, Nemotron 3 Nano Omni only partially:
@Kyle_L_Wiggers Nice looking forward then!
@GlennMatlin @giffmana Ah true, Arcee used prop @datologyai data iirc
Fully open models release complete training details, not just weights.
Do I understand it correctly that the OLMo from-scratch series is coming to an end?
If so, looks like NVIDIA stepped up just in time with Nemotron models as the only remaining fully-open (ie not just weight drop) from-scratch LLM team.
