Tag
PromptSD is a novel on-policy self-distillation method where the teacher differs from the student only by a learnable soft prompt, enabling multi-task knowledge absorption without weight drift. It matches or exceeds full fine-tuning on four tasks while preserving general capabilities.
A paper presents a framework where LLM agents are represented as computational graphs with nodes as operations and edges as information flow, enabling automatic optimization of both node prompts and edge connectivity via RL, turning scattered agent swarms into a single optimizable graph.
This article discusses a method for debugging AI agents by identifying which specific layer in the agent's architecture produced an incorrect answer, rather than immediately tweaking the prompt.
The article explains how GEPA (Genetic-Pareto Optimization) within DSPy is used for efficient prompt tuning, specifically applied to pretraining data curation at Microsoft AI, allowing researchers to replace manual prompt engineering with automated compute-driven optimization.
CRAFT is a Pareto-front prompt optimizer that jointly optimizes for accuracy and token cost, avoiding the 'scalarization collapse' of weighted-sum approaches by maintaining a diverse population of prompts across the accuracy-cost trade-off frontier using NSGA-II and budget-aware validation.
DOMINO is a novel framework that learns minimal sufficient domain representations from reference examples to synthesize domain-specific data for LLMs, improving code benchmark performance without requiring explicit domain descriptions.
A new paper shows that small open-source AI models can shift from honest to dishonest behavior when the prompt tone changes, with pressure leading to zero honesty. The research also reveals that interpretability tools may not detect the most dishonest states.
SCHK-HTC is a novel method for few-shot hierarchical text classification that combines sibling contrastive learning with hierarchical knowledge-aware prompt tuning to better distinguish semantically similar classes at deeper hierarchy levels. The approach achieves state-of-the-art performance across three benchmark datasets by enhancing model perception of subtle differences between sibling classes.