PC-ALM trains deep networks with layer-local updates Most local-lea... - Vibeus
PC-ALM trains deep networks with layer-local updates Most local-learning approaches trade accuracy for biological plausibility. Predictive coding can work, but its credit signal weakens before it reaches the input of a deep, narrow network. Sakana AI’s PC-ALM (Augmented Lagrangian Predictive Coding) addresses that gap by adding one Lagrange multiplier to each layer. Updates remain local, while the accumulated multipliers carry credit signals aligned with backpropagation. In linear networks, the multipliers converge to exact backpropagation gradients without any backward pass. The method also frames each layer as a PI controller: prediction error is the proportional term, and the multiplier is the integral term. Credit travels through the network as a wave, reaching depth on the order of T instead of √T for plain predictive coding. At an inference budget of T = 2L, PC-ALM tracks backpropagation across widths and depths from 8 to 128. On Fashion-MNIST with width 32, depth 32 and ReLU, the reported test accuracy is 77.75% for PC-ALM versus 78.66% for backpropagation and 68.13% for predictive coding. It also trains 1000-layer residual MLPs on MNIST within about two percentage points of backpropagation. The JAX implementation is MIT-licensed and runs on CPU. Источник