Volley3D: Unified 3D Full-Sequence Reconstruction from Monocular Broadcasts

Full Monocular 3D Reconstruction: Lifting an uncalibrated 2D broadcast rally into view-free 3D space.

Abstract

3D reconstruction from single-view broadcast sports is a pivotal capability for automated analytics and immersive AR/VR replays, yet notoriously ill-posed due to difficult-to-calibrate moving cameras and severe depth ambiguity. Unlike widely studied racket sports, team sports such as volleyball feature unpredictable multi-contact sequences and high-altitude jumps that invalidate standard planar ground assumptions.

To resolve these limitations without 3D training supervision, we propose a complete, physics-grounded pipeline tailored to monocular broadcast volleyball. First, we correct perspective drift induced by ground-plane projection by coupling temporal jump spotting with ballistic interpolation for accurate 3D player placement. Second, for 3D ball trajectory estimation, we design a dedicated sub-trajectory segmentation architecture alongside a collision-matching algorithm. By anchoring physical equations directly to reconstructed 3D hand vertices, we lift noisy 2D detections into a spatiotemporally coherent 3D scene.

Global Pipeline Overview

Our pipeline is decoupled into two interconnected stages:

Stage 1 (Player Reconstruction): Combines 2D tracking (YOLOX + Deep-EIoU), dynamic court homography estimation, and camera-space SMPL estimation (WHAM). To bypass planar projection breakdown during jumps, a temporal pose-based spotter isolates takeoff and landing frames, interpolating the 3D elevation via Newtonian motion equations.
Stage 2 (Ball 3D Trajectory Estimation): Employs TrackNetV3 for 2D candidate localization, segments discontinuous rallies using our network based on Temporal Convolutional Network (TCN) blocks, identifies contacting hands via our heuristic collision matching algorithm, and solves a physics-governed Boundary Value Problem (BVP) with aerodynamic drag.

1. Full Pipeline 3D Reconstruction

End-to-end reconstructions on broadcast sequences, visualizing synchronized 3D player meshes in global court coordinates alongside aerodynamic ball trajectories.

Rally Reconstruction Example 2

2. Physics-Based 3D Ball Lifting (VolliQ Benchmark)

Quantitative comparisons on the calibrated multi-camera VolliQ benchmark, contrasting our player-anchored aerodynamic solver against volleyball-adapted 2D-reprojection baselines (MonoTrack) and pure ballistics.

2D vs 3D Drift: MonoTrack minimizes 2D pixel reprojection but suffers severe 3D metric drift compared to our hand-anchored solver.

Trajectory Optimization: Evaluation corresponding to Figure 4 in the main paper across multiple camera angles.

Aerodynamic Drag vs Ballistics: Trajectory simulation highlighting the drift caused by neglecting drag forces ($C_d = 0.035$).

BibTeX

@misc{boussouf2026volley3dunified3dfullsequence,
      title={Volley3D: Unified 3D Full-Sequence Reconstruction from Monocular Broadcasts}, 
      author={Noâm Boussouf and Adrien Maglo and Mohamed Chaouch and Hideo Saito},
      year={2026},
      eprint={2610.38358},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.38358}, 
}
}