23h ago

Researchers Launch CIRCUS Benchmark for Genuine Multimodal Image Retrieval

——0——
Original post
OPPasquale Minervini#713Pasquale Minervini@PMINERVINI

Check out this new project by @MattAttimonelli et al.! TLDR -- multi-modal retrieval benchmarks can be solved by only using one modality, and we tried fixing this by creating manually curated subsets where all modalities are required

6:11 PM · May 18, 2026 View on X
REPLYPasquale Minervini#713Pasquale Minervini@PMINERVINI

it's like the "hypothesis-only" SNLI baseline all over again (but in multimodal settings)

Pasquale MinerviniPasquale Minervini@PMinervini

Check out this new project by @MattAttimonelli et al.! TLDR -- multi-modal retrieval benchmarks can be solved by only using one modality, and we tried fixing this by creating manually curated subsets where all modalities are required

1:11 AM · May 19, 2026 · 1.5K Views
9:20 AM · May 19, 2026 · 113 Views