Zephon’s TorchTitan integration keeps global batches identical when training resumes on a different GPU count

PyTorch says Zephon’s TorchTitan integration matched global-batch SHA-256 hashes when a 1B-parameter run resumed on different GPU counts. Zephon tokenizes, packs, shuffles and mixes data during training, avoiding a preprocessed corpus copy. Resuming requires unchanged data, tokenizer, seed, mixture, context length and global batch size; the lane count must divide evenly across data-parallel replicas, and optimizer-step batches across lanes. Losses may still differ because floating-point reduction order depends on GPU count.