One question I'm sometimes asked is how my research group picks problems. Do I come up with most of the ideas for new papers, or do the students? Neither!
I strongly believe that research is more effective if we pick projects, not problems. What's the difference?
- Projects are long-term research agendas that last 3-5 years or more. A productive project could easily produce a dozen or more papers (depends on the field, of course — in some fields papers represent a lot more work than in others).
- Projects are defined not by a research question but by a change we want to see in the world. For example, the goal of a current project in my group is to make AI more reliable. We may or may not succeed, but the point is that this is a much more ambitious scope than can be tackled in a single paper.
(Some fields have a norm that their job is only to describe the world, not change it. This is culturally jarring to me but even in that case I think projects are better defined in terms of a change you want to see in the research community, if not the external world.)
- Projects are best executed by a core team that stays together and provides intellectual continuity but with a diverse and varying set of collaborators for individual papers which helps constantly bring in new perspectives.
Why pick projects instead of problems? If your method is to jump from problem to problem, you face a tradeoff. You could pick small problems that you can tackle in a month or two, but in that case the resulting papers may not have much impact. Or your can go deep into a topic for many years (essentially what I've described as a project, but structured as a single paper), but that's extremely risky.
In my experience, once a research team is committed to a project, generating the research questions that individual papers in the project will tackle is fairly straightforward. Each paper in the project naturally generates a bunch of new questions and directions for future work. So generating new ideas is not the hard part, rather it is the profusion of ideas. How to select among them? Ideally some combination of intellectual curiosity and whatever best furthers the project's overall goals and vision.
