Tag
A HEC Montréal/MILA paper proposes fair policy optimization for major-minor weakly coupled MDPs, replacing the utilitarian objective with monotone concave fairness functions and introducing a count-proportion-based deep RL approach with a priority-based sampler, validated on machine replacement and NYC taxi pricing/relocation tasks.
The paper develops model-free q-learning algorithms for reinforcement learning in continuous-time jump Markov decision processes, applied to network dynamic pricing, showing superior performance over benchmark methods.
This paper studies decentralized multi-player Q-learning in episodic Markov decision processes under three forms of information asymmetry, proposing algorithms that achieve regret bounds matching the single-agent Q-learning rate up to logarithmic factors.
This paper presents a certificate-carrying sub-quadratic method for computing bisimulation metrics in Markov decision processes using approximate nearest neighbors, with coverage-augmented guarantees and two-sided bounds. Experiments show improved scaling and accurate metric recovery compared to baselines.
This paper studies the sample complexity of robust average-reward Markov decision processes, deriving minimax-optimal learning rates via plug-in reductions under total-variation uncertainty sets.
This paper formalises counterfactual policy optimisation for Markov Decision Processes under probabilistic nondeterministic causal models, which separate latent confounding from inherent stochasticity, and proposes a practical optimisation procedure for deriving robust counterfactual policies. The approach is validated on a sepsis treatment simulator with diabetes as an unobserved global confounder.
This paper introduces a property-driven causal abstraction technique for factored Markov Decision Processes (MDPs), grouping states based on causal relations over state variable predicates to reduce model size while preserving property-relevant behavior. The approach is evaluated on standard benchmarks, yielding small abstractions that support near-optimal policy computation and often generalize to larger MDPs.
This paper proposes efficient online learning algorithms for policy evaluation in MDPs with dynamic utility-based shortfall risk (UBSR) measures under linear function approximation, introducing the UBSR-TD algorithm and demonstrating its convergence and practical effectiveness.
Stanford computer science professor offers a free 83-minute lecture on mastering Markov Decision Processes, covering policy evaluation, value iteration, and convergence limits.
This paper proposes a performance-driven state abstraction method for reinforcement learning that directly optimizes decision quality, using a multi-timescale framework to jointly adapt the policy and a tree-structured abstraction. The algorithm refines or aggregates state space based on Q-value discrepancies, achieving better sample efficiency and faster replanning than baselines.
This paper studies the sample complexity of learning in average-reward weakly-coupled MDPs and restless bandits, establishing finite-sample PAC guarantees with polynomial complexity using a novel Lyapunov-based analysis framework.
This paper introduces Bellman-Taylor Score Decoding, a method to handle state-dependent feasible action sets in Markov decision processes, addressing a key challenge in applying deep reinforcement learning to operations research problems.
This paper formalizes exact unlearning in reinforcement learning, proposing a ρ-TV-stable RL algorithm for tabular MDPs that efficiently removes a user's data influence at a fraction of retraining cost, achieving near-minimax-optimal regret bounds. The work is accepted at ICML and establishes both upper and lower bounds for ρ-TV-stable RL algorithms.
This paper presents an Answer Set Programming (ASP) based implementation of the CARCASS framework for constructing abstractions in reinforcement learning, demonstrating its effectiveness on Blocks World and Minigrid domains.
This paper proposes a quantile Bayesian risk-aware MDP framework for online RL that adaptively balances robustness and exploration over time, providing theoretical regret bounds and demonstrating strong empirical performance.
The article promotes a Stanford lecture on Markov Decision Processes as a valuable resource for understanding the mathematical foundations of systematic trading, claiming it offers more insight than a short-term internship at major financial firms.