Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

World & Business

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Linear mode connectivity in neural network training · Digg

Linear mode connectivity in neural network training

A user calls linear mode connectivity one of the most underrated features of training neural networks.

TLDR

A user recalls a four-year-old post while calling linear mode connectivity one of the most underrated features of training neural networks.

Combined views

21.1K

5 Sources, first seen 14h ago

168 likes

Combined views

21.1K

5 Sources, first seen 14h ago

168 likes

rohan anil Dimitris Papailiopoulos Dan Roy

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

14h ago

first seen 14h ago

7 comments

130 saves

7 reposts

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

7 comments

130 saves

7 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

6101234567890123456789

6101234567890123456789

Related

An AI-compute joke gets a Hugging Face follow-up

One commenter says another may lack “quadratic attention or infinite flops” but has “unclenched thirst and fire.” The person addressed replies, “And also doesnt hack into hugging face.”

Rohan Anil Questions Market for Costly AI Initiative

Co-founder of CoreAutoAI responds in discussion on project funding.

Tech Voices Welcome Open Source Model Agnostic Era

Social media posts share a statement declaring the start of model-agnostic development.

5 Sources

@_arohan_Linear mode connectivity is one of the most underrated feature of training neural networks. I remembered this 4 year old post, and how it appears.

@DimitrisPapail@_arohan_ yeah it's pretty wild! i remember @jefrankle had a very cool paper using linear connectivity to argue about the quality of good sparse network for sparse models (aka lottery tickets) https://arxiv.org/abs/1912.05671

@roydanroyFirst linear mode connectivity (not named that until Frankle, Dziugaite, et al 2019) observation was by @_vaishnavh and it was on test loss when swapping the training data. LMC in Frankle, Dziugaite, et al 2019 was defined on train, IIUC, but we definitely observed it on test (and think it was in the abstract). There's been a lot of cool work since then.

@_vaishnavhAt that point, it was just a cute observation that showed up when I was trying to characterize the space explored by SGD for a fixed random init (in the hope of understanding why deep networks generalize well).

5 Sources

@_arohan_Linear mode connectivity is one of the most underrated feature of training neural networks. I remembered this 4 year old post, and how it appears.

@DimitrisPapail@_arohan_ yeah it's pretty wild! i remember @jefrankle had a very cool paper using linear connectivity to argue about the quality of good sparse network for sparse models (aka lottery tickets) https://arxiv.org/abs/1912.05671

@roydanroyFirst linear mode connectivity (not named that until Frankle, Dziugaite, et al 2019) observation was by @_vaishnavh and it was on test loss when swapping the training data. LMC in Frankle, Dziugaite, et al 2019 was defined on train, IIUC, but we definitely observed it on test (and think it was in the abstract). There's been a lot of cool work since then.

@_vaishnavhAt that point, it was just a cute observation that showed up when I was trying to characterize the space explored by SGD for a fixed random init (in the hope of understanding why deep networks generalize well).