Immersive Content Generation from Standard 2D Videos

People

• Kiana Calagari
• Mohamed Hefeeda

Abstract

The aim of this project is to create compelling immersive videos suitable for VR (virtual reality) devices using only standard 2D videos. The focus of the work is on field sports such as soccer, hockey, basketball, etc. Currently the only way to create immersive content is by using multiple cameras and 360 camera rigs. This means that in addition to the already existing standard 2D cameras around the field, an expensive infrastructure should be added and managed in order to shoot and generate immersive content. In this project, however, we propose a more favorable alternative in which we can utilize the content of the already existing standard 2D cameras around the field to generate an immersive video.

field.png

Details

Setup

We assume a setup in which we have at least 3 cameras as follows. Note that such camera setup is a practical setup in capturing and broadcasting field sports and the following cameras usually exist.

1. The main camera, located in the middle of the field. This camera is a rotating camera capturing wide views and following the ball around the field. It is usually the main camera used for broadcasting games, and most of the feed that audience view comes from this camera.
2. A camera on the right side of the field which covers the players on the right that might be missing in the main camera. This camera doesn't necessarily have to be rotating.
3. A camera on the left side of the field which covers the players on the left that might be missing in the main camera. This camera doesn't necessarily have to be rotating.

Process

The main steps for generating an immersive video from 2D cameras around the field can be described as follows:

1. Generating a still panorama using the motion of the main camera.
2. Overlaying the video frames of the main camera on the panorama.
3. Locating and overlaying the missing players using the left and right cameras.