finite-time-analysis

This paper provides high-probability guarantees for an unprojected linear TD(0) algorithm with Polyak–Ruppert averaging under Markovian sampling, using a single stepsize schedule that achieves both robust curvature-free and fast curvature-dependent convergence rates.

This paper develops a sign-separated finite-time error analysis for constant step-size Q-learning, decomposing the error into negative and positive parts and providing bounds that reveal an asymmetry related to overestimation.