“Mathematics of Neural Networks” is an excellent set of lecture notes for anyone who wants to study modern neural networks from a mathematical perspective.

It covers supervised learning, artificial neurons and activation functions (ReLU, sigmoid, tanh, swish, softmax), shallow and deep neural networks, stochastic gradient descent, weight initialization, vanishing and exploding gradients, convolutional neural networks, automatic differentiation, backpropagation, adaptive optimization algorithms (Adagrad, RMSProp, Adam), and concludes with modern geometric concepts such as manifolds, Lie groups, equivariance, group convolutions, and rotation-translation equivariant CNNs.

I recommend keeping it in your bookmarks as a useful reference whenever you need to explore the mathematics behind neural networks.

https://arxiv.org/abs/2403.04807

Mathematics of Neural Networks (Lecture Notes Graduate Course)

Source: https://arxiv.org/abs/2403.04807 Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Code, Data, Media

Code, Data and Media Associated with this Article

Demos

Demos

Related Papers

Recommenders and Search Tools

IArxiv recommender toggle

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv’s community?Learn more about arXivLabs.