FDSML: Formula-Driven Supervised Metric Learning with Parameter-Aware Triplet Loss

Hugging Face Daily Papers 10/29/26, 12:00 AM Papers

Summary

The paper proposes FDSML, a formula-driven supervised metric learning approach that uses parameter-aware triplet loss on a refined fractal dataset (FineFractalDB) to improve synthetic pre-training for image retrieval, achieving notable Recall@1 gains over FDSL on Stanford Online Products and Stanford Cars.

Formula-Driven Supervised Learning (FDSL) enables vision pre-training without real images, but existing methods cast pre-training as classification on discretized labels, discarding continuity in the generative process. We propose Formula-Driven Supervised Metric Learning (FDSML), which derives relational supervision from image-generation parameters beyond discrete class labels. FDSML combines FineFractalDB, our refined fractal dataset with higher intra-class variation and more stable structures, with a Parameter-Aware Triplet (PA-Triplet) Loss that assigns positives and negatives according to distances in the underlying Iterated Function System parameter space. This objective trains embeddings to respect parameter-derived relative similarity orderings. Experiments show that FDSML improves transferability to image retrieval while remaining competitive on downstream classification and improving robustness under several corruption settings. On ResNet-50, Recall@1 improves from 65.6% to 74.0% on Stanford Online Products and from 56.2% to 78.4% on Stanford Cars over FDSL pre-training. These results suggest that parameter-derived relational supervision provides a stronger synthetic pre-training signal than label-based classification for transfer beyond recognition.

Original Article

Similar Articles

On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification

Papers with Code Trending

This paper analyzes limitations in standard supervised fine-tuning (SFT) from a reinforcement learning perspective and proposes Dynamic Fine-Tuning (DFT), a simple gradient-rescaling method that improves LLM generalization and matches offline RL performance.

Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning

arXiv cs.LG

Proposes FedCMM, a framework for federated continual learning of multimodal LLMs that uses modality-aware elastic weight consolidation, local generative replay, and task-similarity-aware gradient aggregation to mitigate catastrophic forgetting.

Representation Fréchet Loss for Visual Generation

Papers with Code Trending

This paper introduces FD-loss, a method to optimize Fréchet Distance as a training objective for visual generation by decoupling population and batch sizes. It demonstrates that this approach improves generator quality and suggests FID may not always accurately reflect visual quality.

DFSC: Error-Controlled Differentiable Mittag-Leffler Propagation for Fractional Scientific Machine Learning

arXiv cs.LG

This paper introduces DFSC, a PyTorch environment built around a Mittag-Leffler Spectral Layer (MLSL) that enables error-controlled, differentiable fractional propagation in hybrid neural models. It separates known fractional dynamics from data-driven corrections, supports batched/dense/sparse operator paths, and provides certified error bounds in certain regimes.

AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss

Hugging Face Daily Papers

The paper introduces Adversarial Fréchet Distance (AdvFD), which adds a learnable adversarial feature space to static Fréchet losses to improve generator post-training, with real-feature whitening to stabilize optimization.