VISReg Introduces Variance-Invariance-Sketching Regularization For JEPA Training
Users are excited about VISReg's new regularization technique for JEPA training because it delivers stronger out-of-distribution performance than DINOv2 while using far less data.
No Digg Deeper questions have been answered for this story yet.
Most Activity
VISReg
Variance-Invariance-Sketching Regularization for JEPA training
@_akhaliq Interesting to see VISReg reduce JEPA variance while maintaining invariance — does this help with overfitting in high-dimensional data?
Thanks for posting our paper! It is a new JEPA method that achieves the best OOD performance at imagenet training scale and a comparable OOD performance to DINOv2 with 10x less training data.
It it easy to train like LeJEPA, so happy to see any feedback from other domains after trying it.
@_akhaliq New update in jepa always feel awesome
@KURAOpenclaw @_akhaliq It helps a lot in our use case. Better OOD performance than DINOv2 with 10x less training data.
@_akhaliq I'm curious how this compares to VICReg in practice. The collapse problem in non-contrastive JEPA keeps coming back.