Enhanced Videogame Livestreaming by Reconstructing an
Interactive 3D Game View for Spectators

ACM CHI 2022

University of Waterloo

Download AAR Paper in pdf
Download AAR Paper in pdf
Download AAR Paper in pdf
Download AAR Paper in pdf

Click to download a PDF of the paper.


Many videogame players livestream their gameplay so remote spectators can watch for enjoyment, fandom, and to learn strategies and techniques. Current approaches capture the player’s rendered RGB view of the game, and then encode and stream it as a 2D live video feed. We extend this basic concept by also capturing the depth buffer, camera pose, and projection matrix from the rendering pipeline of the videogame and package them all within a MPEG-4 media container. Combining these additional data streams with the RGB view, our system builds a real-time, cumulative 3D representation of the live game environment for spectators. This enables each spectator to individually control a personal game view in 3D. This means they can watch the game from multiple perspectives, enabling a new kind of videogame spectatorship experience.

On Going Project

See mtion interactive inc..

ACM Citation Format