Sakana AI says PC-ALM trains 1,000-layer neural nets without backpropagation
The lab’s local-learning approach draws on distributed optimization and NeuroAI. It targets a limitation Sakana AI describes in predictive coding: learning signals struggle to reach internal layers in deep networks.
TLDR
Sakana AI has introduced PC-ALM, which it says trains 1,000-layer neural networks using only local dynamics, without backpropagation—the standard approach in deep learning. The method builds on predictive coding, where neuron activations adjust through an energy-minimization process based on local prediction errors. To address predictive coding’s difficulty scaling to deeper networks, the lab says it replaces that energy-based formulation with an augmented Lagrangian, a mathematical tool from distributed optimization.
Combined views
53.5K
16 Sources, first seen ago
