Deepity: A C++ library showing Predictive Coding Networks can match Backprop (97.73% on MNIST in 60s) [P]

Reddit r/MachineLearning 09/02/26, 04:49 PM Tools

Summary

The Deepity C++ library implements optimized Predictive Coding Networks to achieve near-backpropagation performance on MNIST, with 97.73% accuracy in 59.5 seconds.

I've spent the last month building a local C++ machine learning library called Deepity to test alternative credit assignment algorithms; specifically Predictive Coding Networks (PCNs). While PCNs are fascinating for biological plausibility and continual learning, naive implementations are painfully slow. By implementing recent research (Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment) and utilizing algorithmic caching to bypass redundant forward projections during the inference settling phase, I managed to close the performance gap with backpropagation on my CPU when training on MNIST (50 epochs). PyTorch Backprop (Feedforward): 98.27% test accuracy in ~70s. Deepity DKPPCN: 97.73% test accuracy in 59.5s. Next up is porting these kernels to CUDA to scale up the architecture and testing its capabilities in continual learning scenarios where standard backprop struggles. If you are interested in local learning, alternative credit assignment, or HPC for ML, I'd love your feedback! GitHub: https://github.com/ra4ster/deepity Project Site: https://ra4ster.github.io/Deepity

Original Article

Similar Articles

@InnocFrancesco: Interested in local learning algorithms? In work accepted at #ICML2026 , we show that predictive coding computes the sa…

X AI KOLs Timeline

This paper shows that predictive coding networks compute the same gradients as backpropagation in the limit of width much larger than depth, bridging biological learning and standard neural network training.

Deep Label-Wise Attentive Temporal Convolutional Networks Improve Medical Coding

arXiv cs.CL

Proposes a deep neural model combining multi-layer temporal convolutional networks with label-wise attention for medical coding, achieving significant improvements in F1 and recall scores over previous state-of-the-art.

Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference

arXiv cs.LG

This paper presents a hybrid amortized inference method that accelerates hierarchical sparse predictive coding by combining a fast initial estimate with corrective refinement steps, achieving better efficiency than pure iterative or amortized approaches.

MNIS Fasion : 17.6 sec for 92.32% test accuracy on the official 10k set (trained on the full 60k) on an AMD Ryzen 7 PRO 8700G (8C/16T)

Reddit r/artificial

A developer reports achieving 92.32% accuracy on the MNIST dataset in 17.6 seconds using a binary XNOR ensemble model on an AMD Ryzen CPU, demonstrating significant speed improvements over traditional floating-point CNNs.

DeepCode: Open Agentic Coding

Papers with Code Trending

DeepCode is a fully autonomous framework for document-to-codebase synthesis that uses principled information-flow management to convert scientific papers into production-grade code, achieving state-of-the-art results on PaperBench and surpassing PhD-level human experts.