6h ago

Yoav Goldberg argues that BitFit bias-only fine-tuning succeeds due to the same underlying mechanisms as steering vectors

The methodology inspired research on tuning only normalization layers.

โ€”โ€”0โ€”โ€”
Original post
OP(((ู„()(ู„() 'yoav))))๐Ÿ‘พ#92(((ู„()(ู„() 'yoav))))๐Ÿ‘พ@YOAVGO

i cant believe I just realized this now, but the reason BitFit (bias only fine-tuning) works, is actually the same reason steering vectors work. or rather, bitfit offers a richer class of adaptations than steering vectors.

8:36 AM ยท May 26, 2026 View on X
REPLY(((ู„()(ู„() 'yoav))))๐Ÿ‘พ#92(((ู„()(ู„() 'yoav))))๐Ÿ‘พ@YOAVGO

@DimitrisPapail i remember this one!

Dimitris PapailiopoulosDimitris Papailiopoulos@DimitrisPapail

@yoavgo I loved BitFit. Was part inspiration for this work of ours https://arxiv.org/abs/2302.07937

3:52 PM ยท May 26, 2026 ยท 1.4K Views
4:25 PM ยท May 26, 2026 ยท 254 Views
REPLYDimitris Papailiopoulos#197Dimitris Papailiopoulos@DIMITRISPAPAIL

@yoavgo I loved BitFit. Was part inspiration for this work of ours https://arxiv.org/abs/2302.07937

(((ู„()(ู„() 'yoav))))๐Ÿ‘พ(((ู„()(ู„() 'yoav))))๐Ÿ‘พ@yoavgo

i cant believe I just realized this now, but the reason BitFit (bias only fine-tuning) works, is actually the same reason steering vectors work. or rather, bitfit offers a richer class of adaptations than steering vectors.

3:36 PM ยท May 26, 2026 ยท 3.9K Views
3:52 PM ยท May 26, 2026 ยท 1.4K Views
Yoav Goldberg argues that BitFit bias-only fine-tuning succeeds due to the same underlying mechanisms as steering vectors ยท Digg