Arnas Uselis Applies Eigenfaces Analysis to SigLIP and DINO

Arnas Uselis posted an 8-second video displaying a 2x2 grid of principal components from embedding models paired with a decoder. Arnas Uselis described applying eigenfaces-style analysis to embedding models including SigLIP and DINO. He claimed that pairing them with a decoder such as an RAE shows what the models primarily care about and what biases the decoder introduces. Uselis noted excitement at seeing how embedding spaces are structured visually and how interventions affect them, for example through SAEs. A linked post by Kosta Derpanis revisits classic eigenfaces visualizations from computer vision research.