Tag
该论文提出一种基于约束上下文老虎机的个性化 LLM 路由框架，通过 primal-dual GLM-UCB 算法在线学习用户偏好，在最小化推理成本的同时以高概率保证响应质量满意度。
This paper introduces TailCert, a PAC-Bayes Bernstein-based procedure for certifying upper tail quantiles of policies selected from logged contextual-bandit data, and derives matched finite-class certification boundaries showing how sample requirements scale with overlap factor, policy-search complexity, and safety margin.
This paper introduces LAG, a closed-form index policy derived from an occupancy-measure LP for online task assignment with reusable resources, modeled as a contextual restless multi-armed bandit. The authors establish the first deterministic fluid limit for a contextual bandit and show LAG is asymptotically optimal, outperforming greedy assignment by up to 36% on synthetic instances.
该论文研究在上下文老虎机的离策略评估中，如何在预算约束下从多个标注源（领域专家、大语言模型）采购反事实标注，通过整数分配优化与单调改进算法，在合成临床和教育数据上将均方误差相对无标注基线降低了 20.58% 和 10.77%。
The paper proposes PFN-OPE, a prior-data fitted network that amortizes off-policy evaluation across contextual bandit tasks, handling both policy shift and reward shift in LLM deployment. On HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies, it achieves 2.0 to 9.3 times lower error than the best baselines in reward-shifted settings.
A paper proposing Residual Quantization (RQ) as a representation layer for contextual bandits, enabling nonlinear reward modeling with strictly bounded O(1) memory via gradient-boosted pseudo-targets and online depth adaptation through a shadow mechanism, beating non-RQ counterparts on 11 of 13 datasets while matching XGBoost and neural baselines at up to 1000x less memory.
This paper investigates how historical A/B test data can inform adaptive experiments using contextual bandits, providing a practical methodology for deciding when and how to deploy adaptive policies based on offline policy evaluation.
This paper studies KL-regularized contextual bandits and shows that greedy sampling can achieve logarithmic regret without explicit eluder-dimension dependence for both reward and preference feedback.
The article presents COBRA-Skills, a method using contextual bandits to efficiently optimize agent skills, reducing costs by avoiding repeated LLM-based trajectory analysis and skill rewriting.
COBRA-Skills introduces a method for optimizing agent skills using contextual bandits and evolutionary operators, achieving 55-58% lower optimization costs across multiple AI agent benchmarks.
This paper formulates the adaptive routing of prompts to large language model experts as a contextual bandit problem with limited feedback, proposing algorithms that achieve sublinear regret and demonstrate efficient learning of high-quality routing strategies.
This paper proposes High-Probability Constrained UCB for contextual bandits with continuous actions, emphasizing realized-cost constraints over expected-cost to improve safety, and provides theoretical regret bounds and experimental validation.
The paper introduces a failure-aware adversarial retrieval-augmented framework using contextual bandits to improve robustness in natural language understanding, with significant improvements on benchmarks like SNLI, ANLI, and MultiNLI.
Introduces Tree-Coupled A/B Testing (TCAB), an exact feedback-sharing design for comparing multiple adaptive policies with fewer reward queries while preserving each policy's trajectory law.
This paper presents a diagnostic protocol for selecting rewards and policies in delayed-feedback contextual bandits, arguing that standard offline evaluation can mislead and validating the approach on benchmarks and a deployed push system.
This paper proposes new reoptimization algorithms for contextual bandits with knapsack constraints, achieving an average regret bound of O((ln T)^3 / T) and improving existing results.
The paper proposes BC-ICL, a bootstrap-conditioned action selection method that leverages pretrained tabular foundation models with in-context learning for contextual bandits, improving exploration and regret performance under strict online protocols.
This paper introduces cross-domain off-policy evaluation and learning (OPE/L) for contextual bandits, allowing the use of logged data from multiple source domains to improve policy evaluation and learning in target domains with challenging conditions like few-shot data, deterministic logging policies, and new actions.
This paper studies contextual causal bandits with non-manipulable variables, proposing causal variants of Thompson Sampling and Information-Directed Sampling (IDS) that exploit shared causal mechanisms to accelerate decision-making. Theoretical regret bounds and experiments on synthetic tasks show that the proposed methods outperform causal and non-causal baselines.
This paper proposes correlation-aware contextual bandit algorithms that leverage surrogate reward signals from machine learning models for LLM routing, achieving improved accuracy-cost trade-offs and sample efficiency compared to standard baselines.