One user has proposed a deliberately small reinforcement-learning stack: PufferLib's version of proximal policy optimization, no asymmetric actor-critic setup, probably recurrent neural networks and a ceiling of 1 million parameters.
The recommendation arrived in one short post, followed seconds later by a reply specifying PufferLib PPO. It is a personal prescription, not a benchmark result or evidence that the same recipe fits every task.
What PPO contributes
PPO was introduced in 2017 as a family of policy-gradient methods for reinforcement learning. Its authors describe a loop that alternates between collecting data through interaction with an environment and optimizing a surrogate objective. Their design permits multiple epochs of minibatch updates, with the goal of balancing implementation simplicity, sample use and training time.
That background helps explain the post's “all you need” framing: it favors one established optimization approach over a more elaborate collection of algorithms. The user's additional constraints narrow the recipe further by rejecting an asymmetric actor-critic arrangement, tentatively adding RNNs and imposing a hard parameter limit.
Why PufferLib is the specific choice
PufferLib's documentation describes the project as a reinforcement-learning library with its own PPO variant. The current documentation says the implementation supports synchronous training and one-epoch asynchronous training. It also says the default PufferNet architecture uses MinGRU, a form of linear recurrence, alongside highway networks.
