interactions

This paper introduces an Interaction-based Prompt Sensitivity (IPS) metric to evaluate and explain prompt sensitivity in large language models by analyzing interactions. It applies IPS to 50 open-source LLMs, identifying factors like fine-tuning and model scale that reduce sensitivity through low-order interactions.

This paper proposes a unified approach to interpret knowledge distillation in LLMs using game-theoretic interactions, discovering that distillation sparsifies interactions, and introduces a loss function CIP to improve performance.