Poolside's Laguna training report details a REINFORCE-style agent framework, prompting technical analysis of classical variance reduction

There is a wealth of variance reduction literature, control variants, approximate/surogate gradients through a learned model of environment, that I’ve been waiting to see if it makes its way into modern RL, like REBAR, RELAX, “backpropagation through the void”