Fair Representation Learning Yields Useless Classifiers When Base Rates Differ
Picture each group as a stack of pancakes — one pancake per class label. Groups differ in how the two stacks are mixed (different base rates). Force the combined view to look identical across groups, and the two pancake types (one per class) have to fuse into one.
Alex Smola@smolixFair representation learning promises feature distributions that look identical across groups. We show that, if the groups have different base rates, the only way to make features look identical is to discard every trace of the label. In other words, you get useless classifiers.
What does this mean for "fair representation learning"? Instead of chasing the limit of perfect fairness, trade off fairness for accuracy and you can get a lot of both worlds. For more details see the blog post: https://alex.smola.org/posts/36-pancake-theorem/. Joint work with Daniel Matsui Smola.
Alex Smola@smolixFor a pancake free proof (using linear algebra instead, check out the paper) https://arxiv.org/abs/2605.09221.
For a pancake free proof (using linear algebra instead, check out the paper) https://arxiv.org/abs/2605.09221.
Alex Smola@smolixPicture each group as a stack of pancakes — one pancake per class label. Groups differ in how the two stacks are mixed (different base rates). Force the combined view to look identical across groups, and the two pancake types (one per class) have to fuse into one.