Stella Biderman of EleutherAI highlights low ICML workshop acceptance rates, sparking debate over a standalone AI interpretability conference

While the numbers aren’t really comparable (short workshop submissions vs full papers), one wonders if Interpretability (mechanistic or other flavors) should have its own conference by now.

Apparently ICML Mechinterp had a lower acceptance rate than ICML this year. 25% vs 26.6%

Maybe Mechinterp should have its own conference now.

@BlancheMinerva @FazlBarez The email notification said 44% right?

@FazlBarez Yeah, the final acceptance rate is going to be < 20% for Mech Interp. We had 800 submissions and a little under 200 acceptances.

@BlancheMinerva Really?

@FazlBarez Wait, that’s a < 25% acceptance rate not < 20%. Still, ICML was 26.6% this year per the official emails.

@BlancheMinerva 800 submissions? It is a lot.

@BlancheMinerva @FazlBarez Gotta say if it is <20%, this is evidence that MI is not appropriate for a workshop. It is too broad; this is not what the workshops are for. It should be a conference, and ICML needs more targeted interp workshops.

@boknilev There used to be a "deep learning" workshop at neurips. they stopped having it after the acceptance rate dipped below the neurips acceptance rate. now DL has ICLR (and, well, neurips)

@BlancheMinerva Im still surprised, I expected approx 30% online with previous years afair?

@BlancheMinerva @FazlBarez Like there used to be a "deep learning" workshop at neurips. they stopped having it after the acceptance rate dipped below the neurips acceptance rate.

@roeiherzig @BlancheMinerva Eleuther runs a fellowship program with academics/industry researchers working with fellows. Presumably that can produce that 800 output

@BlancheMinerva alignment research officially harder to publish than generic ML papers now

feels backwards but says a lot about where the field is at