Fine-tuning took six weeks on a GB300 NVL72 cluster
not sure the minimum required but i can tell you i used a full gb300nvl72 for ft (took ~6 weeks) and now i am hosting it on the same cluster . the minimum i've used to run in real disagg for a meaningful number of tokens is 12x8xB200 (8 prefill x 4 decode) but ideally you have much more
@_xjdr how much infra do you need to finetune / host K2.6?
Users praise the massive GPU cluster for fine-tuning K2.6 as a beast because of the high FLOPs and HBM needed to train and run it properly.
@_xjdr alrighty looks like i'm lowering my ambitions to laguna xs.2
not sure the minimum required but i can tell you i used a full gb300nvl72 for ft (took ~6 weeks) and now i am hosting it on the same cluster . the minimum i've used to run in real disagg for a meaningful number of tokens is 12x8xB200 (8 prefill x 4 decode) but ideally you have much more
@vikhyatk its a beast. even in fp4, it takes quite a lot flop and quite a bit of HBM to train and run properly
@_xjdr alrighty looks like i'm lowering my ambitions to laguna xs.2
Fine-tuning took six weeks on a GB300 NVL72 cluster
not sure the minimum required but i can tell you i used a full gb300nvl72 for ft (took ~6 weeks) and now i am hosting it on the same cluster . the minimum i've used to run in real disagg for a meaningful number of tokens is 12x8xB200 (8 prefill x 4 decode) but ideally you have much more
@_xjdr how much infra do you need to finetune / host K2.6?