Difference between revisions of "2D to 3D Video Conversion"
| Line 8: | Line 8: | ||
== Details == | == Details == | ||
| + | |||
| + | The figure below shows an overview of our conversion system. We infer depth from a database of synthetically generated high-quality depths, collected | ||
| + | from video games. We then perform the conversion by transferring the depth gradient field from the database and reconstructing depth using Poisson reconstruction. In order to maintain sharp and accurate object boundaries, we create object masks and modify the Poisson equation on object boundaries. Finally, using the 2D frames and their estimated depth the left and right stereo pairs are rendered using a stereo-warping technique. | ||
<center> | <center> | ||
| Line 17: | Line 20: | ||
</center> | </center> | ||
| − | + | The main components of of our depth gradient based conversion technique as shown in the following figure are: | |
<center> | <center> | ||
{| border="0" | {| border="0" | ||
| Line 26: | Line 29: | ||
</center> | </center> | ||
| + | * ''' Visual Search: | ||
| + | |||
| + | * ''' Block-based Matching: ''' | ||
| + | |||
| + | * ''' Poisson Reconstruction: ''' | ||
| + | |||
| + | * ''' Gradient Refinement: ''' | ||
| + | |||
| + | * ''' Object Boundary Cuts: ''' | ||
| + | |||
| + | * ''' Smoothing: ''' | ||
<center> | <center> | ||
Revision as of 14:39, 8 November 2016
People
- Kiana Calagari
- Mohamed Hefeeda
Abstract
A wide spread adoption of 3D displays is hindered by the lack of content that matches the user expectations. Producing 3D videos is far more costly and time-consuming than regular 2D videos, which makes it challenging and thus rarely attempted, especially for live events, such as soccer games. In this project we develop a high-quality automated 2D-to-3D conversion method for soccer videos. Our method is data driven, relying on a reference database of 3D videos. Our key insight is that we use computer generated depth from current computer sports games for creating a synthetic 3D database.
Details
The figure below shows an overview of our conversion system. We infer depth from a database of synthetically generated high-quality depths, collected from video games. We then perform the conversion by transferring the depth gradient field from the database and reconstructing depth using Poisson reconstruction. In order to maintain sharp and accurate object boundaries, we create object masks and modify the Poisson equation on object boundaries. Finally, using the 2D frames and their estimated depth the left and right stereo pairs are rendered using a stereo-warping technique.
 |
| The proposed 2D-to-3D conversion system |
The main components of of our depth gradient based conversion technique as shown in the following figure are:
 |
| The main components of our depth gradient based conversion |
- Visual Search:
- Block-based Matching:
- Poisson Reconstruction:
- Gradient Refinement:
- Object Boundary Cuts:
- Smoothing:
 |
| The effect of each step in our depth estimation technique: (a) Query, (b) A subset of its K candidates, (c) Created matched image, (d) Object boundary cuts, (e) Depth estimation using Poisson reconstruction, (f) Gradient refinement and Poisson reconstruction, (g) Depth with object boundary cuts, (h) Final depth estimation with smoothness, and (i) The zoomed and amplified version of the yellow block in h. |
 |
| Depth estimation for a wide variety of soccer shots using our method |
Publications
K. Calagari, M. Elgharib, P. Didyk, A. Kaspar, W. Matusik, and M. Hefeeda, “Gradient-based 2D-to-3D Conversion for Soccer Videos”, In Proc. of the ACM Multimedia (MM’15), p 331-340, 2015.
