The model self-organizes without explicit category training labels.
Users are excited by experiments on AI models developing brain-like maps for faces, voices and sentences, praising the testable hypotheses and lack of capability trade-offs.
No Digg Deeper questions have been answered for this story yet.
2/ Like the real cortex: one continuous sheet, from retinotopy to the language network.
Most topographic models fragment this, a separate 2D map per layer, and for one modality only.
Topo-Omni instead puts vision, audio, & cognitive units of Qwen-Omni on ONE shared sheet!
11/ Links
📄 Paper: http://arxiv.org/abs/2606.09770 💻 Code: http://github.com/epflneuroailab/topo-omni 🤗 Model: http://huggingface.co/epfl-neuroai/topo-omni 🌐 Website: http://topo-omni.epfl.ch
10/
One spatial-smoothness rule, applied to a pretrained multimodal model, recovers cortex-like organization + generates new, testable hypotheses about the brain.
co-led w/ @HannesMehrer, collab w/ Lara Marinov, Ahmed Abdelaal, @akgokce0, supervised by @martin_schrimpf!
3/ Vision
Result: in the vision encoder, distinct patches emerge for faces, scenes, objects, and visual word forms: echoing the FFA, PPA, LOC and VWFA of your visual ventral stream.
Their response profiles correlate with real human fMRI (r up to 0.89).
5/ Cognition
Feed it text, and the deeper module finds THREE networks
🗣️ Language (sentences > nonwords) 🧮 Multiple-demand (math > sentences) 🤔 Theory of mind (false-belief > false-photo)
One spatial rule → dissociable cognitive networks. No brain data, no labels.
7/ We can drive or suppress clusters:
🔼 Drive face units → model "sees" faces almost everywhere 🔽 Suppress top 10% of face units → face recognition collapses to ~0%, while bodies/scenes/objects stay intact
These regions aren't just correlated → causally responsible.
8/ The big one: discovering NEW brain networks
We used Topo-Omni's sheet to screen video clips and found two clusters with no clear precedent in the literature:
🦅 An animals network (snakes, birds, big cats) 🏔️ A natural landscapes network (beaches, mountains, terrain)
9/
We tested these AI-derived predictions on independent human fMRI (Spacetop) and found right-lateralized prefrontal activation for both.
An AI model just predicted new organization in the brain. 🤯
6/ "But doesn't this hurt performance?"
We checked. vs. non-topographic baselines (same backbone, same task loss):
✅ Brain predictivity statistically tied on 11/12 visual ROIs (Natural Scenes Dataset) ✅ Best overall OmniBench accuracy
Topography is basically free!
4/ Audition
In the audio encoder: a speech-selective region tracks the human superior temporal gyrus (r=0.69), it responds to intelligible speech but drops for "quilted" speech.
A separate voice-selective region prefers human vocalizations, like the temporal voice areas.
@bkhmsi damn very interesting experiments!
@bkhmsi ohh i can't wait to read this. i wish tts was better at vocalizing high complexity text. gonna go write the paper on it. brb.
@bkhmsi @dyamins ignoramus
@bkhmsi wonder if the 'respond alike' rule assumes a metric that flattens things differently than human perception
@bkhmsi Not hype. Testable hypotheses, causal clusters, and no measurable cost in capability. More of this.