hard-label-consensus

This paper introduces TallyTrain, a communication-efficient federated distillation method that transmits only the argmax class index per probe (hard-label consensus) instead of full softmax vectors, reducing bandwidth by up to three orders of magnitude while matching or surpassing the performance of soft-label distillation and Pareto-dominating standard federated learning baselines like FedAvg, FedProx, and FedDF.