deep-learning

This paper introduces Stiefel-AdamW, a geometry-aware optimizer for linear factorization blocks in deep learning that enhances stability and performance, validated on models like GPT2, ViT, and Mistral 7B.

This paper establishes two limitations of Maximal Coding Rate Reduction (MCR²) for out-of-distribution generalisation, showing that it can fail under distribution shift and that incorporating invariance principles does not eliminate this failure.

A comparative study of temporal deep learning architectures for physiological emotion recognition using multimodal wearable datasets, evaluating LSTM, TCN, and Transformer models under different sensing configurations.

TALON introduces a temporally aware longitudinal framework for radiology report generation that adaptively integrates variable-length patient histories using a dual-channel temporal fusion module to improve accuracy in detecting interval changes and persistent findings.

This article explains why the backpropagation algorithm in neural networks is implemented in a backward pass, clarifying the efficiency advantages over a forward computation based on the chain rule.

The author fine-tuned Qwen3.5 4B using LoRA with public and synthetic data to create a Jev-style model, achieving improved performance and open-sourcing the model and dataset.

The article introduces a visualization tool for the attention mechanism in Transformers, allowing users to customize input matrices and see step-by-step computations of attention scores. It is an initial version with plans for future updates based on user feedback.

The PyTorch Conference North America is announced, featuring sessions on new optimizers like Muon and TorchJD, tools like fastsafetensors and LMCache, with speakers from major tech companies and institutions, happening in San Jose on October 20-21.

An announcement for a talk at PyTorch Conference North America covering practical implementations of Muon, Dion, and Dion3 optimizers in training stacks.

This Twitter thread introduces Hugging Face's Kernels, a tool that allows users to select and replace optimized kernel implementations for supported layers in AI models without rewriting the entire model.

An open-source AI agent that automates the discovery, filtering, and ranking of ML research papers based on user interests, generating personalized reports to streamline the research process.

The article explains the engineering aspects of Mixture-of-Experts (MoE) inference, detailing token routing, expert batching, GPU distribution, and performance trade-offs for efficient serving.

A Stanford deep learning course on computer vision taught by Professor Fei-Fei Li is available on YouTube, discussing the evolutionary role of vision in developing intelligence.

The paper proposes a deep architecture that jointly optimizes cost functions and route-ranking models for customizable route planning using differentiable shortest-path search, demonstrating significant improvements over state-of-the-art methods on real-world datasets.

The paper proposes a Physics Informed Recurrent Neural Network (PIRNN) that predicts unobservable physical variables to improve time series forecasting in physical processes, demonstrated through groundwater level predictions.

This paper introduces a post-hoc weight rectification framework called JANUS to mitigate catastrophic forgetting in fine-tuning foundation models, achieving parameter space orthogonality for preserving historical performance while adapting to new tasks.

The paper proposes TriDim, a reusable block for EEG decoding that preserves spatial and temporal structures, achieving state-of-the-art performance across multiple datasets and improving existing models.

CoRe proposes a model-agnostic learning objective for multivariate time-series forecasting that uses frequency coherence and relational graph losses to improve prediction accuracy over standard methods.

This paper proposes the Knowledge-enhanced Agriculture-informed Neural Network (KAINN) framework, which integrates domain knowledge into deep learning models to improve the accuracy and interpretability of nitrous oxide emissions predictions in agricultural systems.

FoundAna introduces a foundation model for graph anomaly detection that integrates GNNs and transformers to achieve generalizable, cross-graph performance, demonstrating superior results on multiple benchmark datasets.