Tag
This arXiv paper proposes Desired-Update-Aligned Synthetic Data (DASA), which uses activation-gradient feedback from a frozen reference model to optimize continuous synthetic input embeddings for LLM fine-tuning, questioning whether human-readable text is necessary. Experiments across Llama and Qwen models (1B–32B) show DASA matches or outperforms natural-language data under LoRA and delivers a 3.6–4.9x speedup over GRADMM.
This paper introduces a gradient-based speech-to-text alignment method applicable to any differentiable ASR model, including CTC, transducer, attention-based encoder-decoder, and speech large language models, requiring no training or model modification.
Proposes Gradient-Based Connections (GBC), a method that models multi-agent LLM systems as computational graphs and uses gradient signals to attribute errors to specific agents, enabling better system-level optimization.