Paper page - TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videos

Source: https://huggingface.co/papers/2605.01234

Abstract

A novel reconstruction pipeline for table tennis gameplay from monocular broadcast videos achieves high-fidelity 3D ball trajectories and spin estimation through a learned lifting network that operates on unsegmented 2D tracks before time segmentation.

We present TT4D, a large-scale, high-fidelitytable tennis dataset. It provides 140+ hours of reconstructed singles and doubles gameplay frommonocular broadcast videos, featuring multimodal annotations like high-quality camera calibrations, precise3D ball positions,ball spin,time segmentation, and3D human meshesover time. This rich data provides a new foundation for virtual replay, in-depth player analysis, and robot learning. The dataset’s combination of scale and precision is achieved through a novelreconstruction pipeline. Prior methods first partition a game sequence into individual shot segments based on the 2D ball track, and only then attempt reconstruction. However, 2D-basedtime segmentationcollapses under occlusion and varied camera viewpoints, preventing reliable reconstruction. We invert this paradigm by first lifting the entireunsegmented 2D ball trackto 3D through alearned lifting network. This 3D trajectory then allows us to reliably performtime segmentation. Thelearned lifting networkalso infers the ball’s spin, handles unreliable ball detections, and successfully reconstructs the ball trajectory in cases of high occlusion. This lift-first design is necessary, as our pipeline is the only method capable of reconstructing table tennis gameplay from general-view broadcast monocular videos. We demonstrate the dataset’s fidelity through two downstream tasks: estimating the racket’s pose \& velocity at impact, and training a generative model of competitive rallies.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.01234

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.01234 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.01234 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.01234 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.