Tag
This paper develops a mathematical framework for analyzing information discarded by machine learning models under Lie group actions, introducing null fibers and stabilizers, with applications to data masking, model fingerprinting, and privacy-preserving computation tested on molecular and image tasks.
CrossBERT decouples representation learning from token reconstruction, enabling higher masking ratios and better sample efficiency, outperforming BERT on MTEB and GLUE benchmarks.