Adaptive Multi-Scale Goodness Aggregation for Forward-Forward Learning

arXiv cs.LG 05/20/26, 04:00 AM Papers

Summary

Proposes Adaptive Multi-Scale Goodness Aggregation (AMSGA), an extension of the Forward-Forward algorithm that improves stability, robustness, and generalization via multi-scale goodness aggregation, adaptive hard negative mining, and layer-dependent thresholds, achieving modest accuracy gains on MNIST and Fashion-MNIST.

arXiv:2605.18804v1 Announce Type: new Abstract: We propose Adaptive Multi-Scale Goodness Aggregation (AMSGA), a novel extension of the Forward-Forward (FF) algorithm designed to improve stability, robustness, and generalization in local-learning neural networks. AMSGA addresses several limitations of the original FF framework by introducing multi-scale goodness aggregation across local, intermediate, and global representations; adaptive curriculum-guided hard negative mining; layer-dependent adaptive thresholds; and a warm-up cosine annealing learning-rate schedule for improved optimization stability. Together, these modifications strengthen the FF paradigm while preserving its biologically plausible and memory-efficient properties. Experiments on MNIST and Fashion-MNIST demonstrate consistent performance improvements over the baseline FF algorithm, achieving up to +1.45% improvement on MNIST and +1.50% improvement on Fashion-MNIST without significant computational overhead. Our results suggest that local learning methods can become substantially more competitive when goodness estimation and training dynamics are carefully designed.

Original Article

Similar Articles

AHEAD: Advancing Multi-Class Label Aggregation with Interpretable Cross-Annotator Modeling

arXiv cs.LG

This paper proposes AHEAD, a cross-annotator learning framework for multi-class label aggregation that uses graph neural networks to model annotator reliability, achieving significant accuracy improvements on 10 real-world datasets spanning NLP, CV, Video, and Audio.

A Unified Framework for Gradient Aggregation in Multi-Objective Optimization

arXiv cs.LG

This paper presents a unified theoretical framework for gradient aggregation in multi-objective optimization, establishing convergence rates to Pareto stationarity. The authors introduce a sufficient alignment condition and demonstrate its application to existing and new algorithms, such as capped MGDA.

LoRA-GA$^2$: Low Rank Adaptation with Multi-step Gradient Adaptive Alignment

arXiv cs.CL

This paper introduces LoRA-GA2, a fine-tuning algorithm that leverages multi-step gradient information to improve the performance of Low-Rank Adaptation for large language models, achieving better results on benchmarks while preserving efficiency.

Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings

Hugging Face Daily Papers

SAGA framework uses frozen multimodal large language models to provide attribute-aware supervision for vision encoders via Group Relative Policy Optimization, improving zero-shot image retrieval by 3–6 points on fine-grained benchmarks.

SIGMA: Structured Noise-Effect-Aware Grouped Multi-Agent Aggregation

arXiv cs.AI

This paper proposes SIGMA, a hierarchical collaboration framework for cooperative multi-agent reinforcement learning that learns robust representations under noisy observations by exploiting cooperation structures through density-based grouping and aggregation methods.