Task diversity is supposedly key to generalization in RL. But what does it do to continual RL, where agents face one new task distribution after another?

We find that past a point, more diversity actually inhibits continual reinforcement learning 🧵