Announcement
High-rank JEPA projections may not rule out backbone collapse
A post says λ-JEPA is designed to target the gap between a high-rank projection and a potentially collapsed backbone.
TLDR
A post questions whether JEPA prevents collapse in the representation used downstream. It argues that a high-rank projected representation does not guarantee a non-collapsed backbone, and says λ-JEPA is designed to target that gap.
Combined views
30.3K
1 Source, first seen ago
