Tag
This paper tracks how different learning rules (backprop, feedback alignment, predictive coding, STDP) affect the alignment of CNN representations with human fMRI across training. It finds that backprop destroys V1 alignment in one epoch, while local rules preserve it, suggesting a trade-off between building higher-level representations and retaining early visual features.
Stanford CS224N course notes provide a clear introduction to the mathematics of backpropagation and gradient computation in neural networks, covering chain rule, computational graphs, and vectorized derivatives.
This paper tracks how supervised training with different learning rules (backpropagation, feedback alignment, predictive coding, STDP) degrades alignment between neural network representations and early visual cortex fMRI data, finding that untrained networks often match or exceed trained ones in V1 alignment.
Sakana AI presents DiffusionBlocks, a method that trains neural networks block-wise by interpreting forward passes as diffusion denoising, significantly reducing memory requirements compared to traditional end-to-end backpropagation.
This thread explains the intuition behind the Jacobian Matrix and its widespread applications in AI and machine learning, including backpropagation, normalizing flows, computer vision, and robotics.
The paper introduces closed-form predictive coding via hierarchical Gaussian filters that restore precision-weighted prediction errors, yielding faster and more efficient training without global error signals, outperforming backpropagation on certain tasks.
This paper evaluates the biological plausibility and representational alignment of feedback alignment algorithms in convolutional networks, comparing them to standard backpropagation on CIFAR-10. The authors find that modified feedback alignment methods converge on internal representations similar to those produced by backpropagation, suggesting functional success through mimicking representational geometry.
Andrej Karpathy's 'Neural Networks: Zero to Hero' is a free course covering neural networks from basics to modern architectures like transformers, with YouTube lectures and Jupyter notebooks. It includes hands-on implementations of micrograd and makemore.