@yoavgo If you aim for short term gains from it, you've got to be short term relevant. If you plan something bigger, make big changes and if they are big enough they will follow. So, do you try to justify to reviewers and they disagree is that it? Do you look for short term?
I find it quite disturbing that prefix caching considerations make it really hard to justify any context optimization/organization innovation, as whatever you may want to do will wreck the cache