Tag
The paper proposes a rewriting scaling law that predicts loss from model size, training tokens, rewrite fraction, and data epochs while accounting for rewrite generation compute, finding optimal rewrite fractions increase with compute and that models trained on rewrites can outperform their rewriters.
该论文对合成数据上的隐私风险进行了实证研究，比较了数据级与训练级干预手段在能力与隐私之间的权衡，发现隐私感知重写相比 DP-SGD 或直接微调能提供更优的能力-隐私平衡。