Perceptual metrics measure appearance, not identity. Re-ID models are powerful but domain-specific. Instance retrieval is more general, but "instance" definitions often vary, affecting granularity.

There's a gap for a general metric with selective sensitivity across domains.

Releasing pretrained checkpoints (DINOv3 ViT-L/16, ViT-B/16, DINOv2 ViT-L/14, ViT-B/14), training + evaluation code, and Subjects2k.

More information is on our project page: https://juliachae.github.io/id_sim.github.io/ GitHub: https://github.com/JuliaChae/id_sim

@AdobeResearch @cusuh_ @JuiHsienWang1 @rzhang88 @sarameghanbeery Poster will be presented at: 🗓️ Poster Session 2 on Friday, June 4 📍Poster #378, Exhibit Hall A

We also run a systematic sensitivity analysis, using controlled generative edits to explicitly disentangle identity, background, viewpoint, and lighting changes.

ID-Sim is the most sensitive to identity and most invariant to the rest.

Huge thank you to the wonderful collaborators! Would not be possible without guidance from mentors at @AdobeResearch during my internship Cusuh Ham (@cusuh_ ), Jui-Hsien Wang (@JuiHsienWang1), Nicholas Kolkin, Richard Zhang (@rzhang88) and my advisor Sara Beery @sarameghanbeery

Getting there starts with the data. We curate 10k triplets spanning landmarks, fashion, objects, art, and animals, then crucially diversify with generative edits: → context edits (same object, diverse context) → identity edits (subtly different object as hard negatives)

Evaluated on 7 held-out benchmarks against perceptual metrics, foundation models, and a universal instance retrieval model. ID-Sim achieves the best results. 🏆

The improvements are sharpest on tasks that demand both at once — context invariance and fine-grained discrimination.

The patch embeddings transfer too — plugging ID-Sim into PerSAM gives a 3x mAP improvement over DINOv3 on personalized segmentation.

In addition, identity-sensitive local features are localizable in multi-instance settings.

We also release Subjects2k — a new concept-preservation benchmark.

The original Subjects200k uses GPT annotations, but we found these misalign with human judgments and mislabel many pairs. We re-annotated 2k pairs with human labels and release it as a standalone challenge.

@juliachae_ Finding this on my feed like Series B Anthropic 🤑