Tag
该论文研究在上下文老虎机的离策略评估中，如何在预算约束下从多个标注源（领域专家、大语言模型）采购反事实标注，通过整数分配优化与单调改进算法，在合成临床和教育数据上将均方误差相对无标注基线降低了 20.58% 和 10.77%。
The paper proposes PFN-OPE, a prior-data fitted network that amortizes off-policy evaluation across contextual bandit tasks, handling both policy shift and reward shift in LLM deployment. On HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies, it achieves 2.0 to 9.3 times lower error than the best baselines in reward-shifted settings.
The paper proposes OTROPE, a likelihood-free off-policy evaluation method for LLMs that uses optimal transport to align limited human-labeled behavior-policy samples with unlabeled target-policy samples in semantic space, yielding a doubly robust-style evaluator that outperforms baselines under distribution shift.
This paper explores using a model's semantic ID hierarchy for off-policy evaluation in generative recommenders, showing that coarsening to code-prefix clusters improves estimation accuracy under production logging constraints.
A controlled, reproducible benchmark evaluating offline estimators for equal-cost top-k allocation, identifying when offline evaluation can be trusted and providing a practitioner's guide.
This paper introduces cross-domain off-policy evaluation and learning (OPE/L) for contextual bandits, allowing the use of logged data from multiple source domains to improve policy evaluation and learning in target domains with challenging conditions like few-shot data, deterministic logging policies, and new actions.
This paper studies off-policy evaluation (OPE) when decision subjects (agents) strategically modify their covariates in response to a policy. It proposes a method that uses local disclosure via post-hoc explanations to reveal agents' pre-strategic covariates and construct a doubly robust estimator for policy value.
Proposes Adwm, an autoregressive diffusion world model for off-policy evaluation of LLM agents, enabling reliable value estimates from pre-collected trajectories without online interaction.
The paper introduces Human-in-the-Loop Gated Bandit (HITL-GB) for short-term rental dynamic pricing, showing that historical pricing data under a prior policy is structurally equivalent to on-policy warm-up data, reducing cold-start from ~150 to ~30 episodes.
This paper demonstrates the robustness of refugee matching impact evaluations using off-policy methods like IPW and AIPW, confirming previous findings on algorithmic refugee assignment.