Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain

Hugging Face Daily Papers Papers
transfer-learning semi-supervised-learning noise-adaptation domain-adaptation generalization

Summary

This paper introduces Semi-Supervised Noise Adaptation (SSNA), a novel framework that uses synthetic noise domains (e.g., Gaussian distributions) as surrogate source domains to improve generalization in semi-supervised learning settings. The proposed Noise Adaptation Framework (NAF) establishes a generalization bound and demonstrates improved target domain performance.

Transfer learning aims to facilitate the learning of a target domain by transferring knowledge from a source domain. The source domain typically contains semantically meaningful samples (*e.g.*, images) to facilitate effective knowledge transfer. However, a recent study observes that the noise domain constructed from simple distributions (*e.g.*, Gaussian distributions) can serve as a surrogate source domain in the semi-supervised setting, where only a small proportion of target samples are labeled while most remain unlabeled. Based on this surprising observation, we formulate a novel problem termed *Semi-Supervised Noise Adaptation* (SSNA), which aims to leverage a synthetic noise domain to improve the generalization of the target domain. To address this problem, we first establish a generalization bound characterizing the effect of the noise domain on generalization, based on which we propose a Noise Adaptation Framework (NAF). Extensive experiments demonstrate that NAF effectively leverages the noise domain to tighten the generalization bound of the target domain, leading to improved performance. The codes are available at https://github.com/AIResearch-Group/SSNA.
Original Article

Similar Articles

Subliminal Learning is Non-Semantic Distillation

arXiv cs.AI

This paper investigates subliminal learning in language models, showing that biases can transfer from teacher to student via seemingly random synthetic data. The authors find that adding Gaussian noise to weights increases transfer, and that students inherit not just the semantic bias but also the type of intervention used, with implications for training safety and data auditing.