Announcement
Tinker adds Dimuon optimizer for LoRA fine-tuning
In its SFT tests, Tinker says Dimuon often reached lower loss in fewer steps than AdamW.
TLDR
Tinker introduced Dimuon on October 15 as a new optimizer for LoRA fine-tuning. It says Dimuon often reached lower loss in fewer steps than AdamW in its SFT tests. Tinker said it was still experimenting with Dimuon and invited users to try it and share feedback.
Combined views
3K
4 Sources, first seen ago
