Private:FTV - Revision history

Aah10: /* Software and Data */

2013-09-01T04:16:47Z

Software and Data

Aah10: /* Updates */

2013-08-31T21:14:32Z

Updates

Aah10: /* Updates */

2013-08-31T21:12:37Z

Updates

Aah10: /* libdash */

2013-05-08T00:18:39Z

libdash

Aah10 at 00:23, 16 April 2013

2013-04-16T00:23:18Z

Aah10 at 08:47, 26 March 2013

2013-03-26T08:47:45Z

Aah10 at 08:41, 26 March 2013

2013-03-26T08:41:05Z

Aah10 at 02:34, 12 February 2013

2013-02-12T02:34:21Z

Aah10 at 03:02, 8 February 2013

2013-02-08T03:02:47Z

Aah10 at 21:23, 6 February 2013

2013-02-06T21:23:15Z

@@ Line 254: / Line 254: @@
 ** [http://gpac.wp.mines-telecom.fr/ Homepage]
 ** [http://records.sigmm.ndlab.net/2012/12/open-source-column-gpac/ Overiew in ACM SIGMM Records]
 * [https://github.com/slederer/DASHEncoder DASHEncoder]
@@ Line 264: / Line 266: @@
 ** Example: Encoding MSR video sequences views (can use '''-qp 0''' or '''-crf 0''' to encode a lossless output):
 **:<pre>ffmpeg -i color-cam2-f%03d.bmp -c:v libx264 -preset slow -crf 22 -r 10 -c:a copy cam2.mp4</pre>
 == References ==

@@ Line 6: / Line 6: @@
 * '''[Ahmed:3/24]''' Main problem we are facing now is getting the decoder to start decoding properly at an arbitrary segment. This functionality is necessary during view switching. This requires first getting the initialization segment and then sequentially fetching segments from a certain position in the stream based on the view switching point. The current implementation creates a DASHReceiver object which starts fetching segments immediately after its creation starting from initialization segment. Each segment is fetched in a separate thread. The decoder object LibavDecoder reads from an in-memory buffer and not from a file.
 * '''[Ahmed:2/11]''' Was able to render two frames side-by-side using SDL and OpenGL. Now working on integrating this with the DASH decoder.
-* '''[Ahmed:2/6]''' Managed to build the sample player that comes with ''libdash'' after modifications to build configuration files as well as some parts of the source code. The player works and the video is retrieved from the server, decoded, and displayed. I'm working on modifying it to render two video streams side-by-side. Our initial thought was that we can just use SDL to perform the rendering. However, it seems we might need to use OpenGL after all. Displaying video frames in SDL is achieved using YUV video overlays (SDL_Overlay) and it is not possible to display more than one overlay at the same time. A workaround may be to first merge the two decoded YUV frames side-by-side before drawing them on the overlay. But this must be done on-the-fly. I tried looking into ''libswscale'' to perform this, but I couldn't find anything that can help.
+* '''[Ahmed:2/6]''' Decided not to use the Bino player and work on implementing our own based on libdash. Managed to build the sample player that comes with ''libdash'' after modifications to build configuration files as well as some parts of the source code. The player works and the video is retrieved from the server, decoded, and displayed. I'm working on modifying it to render two video streams side-by-side. Our initial thought was that we can just use SDL to perform the rendering. However, it seems we might need to use OpenGL after all. Displaying video frames in SDL is achieved using YUV video overlays (SDL_Overlay) and it is not possible to display more than one overlay at the same time. A workaround may be to first merge the two decoded YUV frames side-by-side before drawing them on the overlay. But this must be done on-the-fly. I tried looking into ''libswscale'' to perform this, but I couldn't find anything that can help.
 * '''[Meeting on:2/1]'''

@@ Line 3: / Line 3: @@
 == Updates ==
 * '''[Ahmed:3/24]''' Main problem we are facing now is getting the decoder to start decoding properly at an arbitrary segment. This functionality is necessary during view switching. This requires first getting the initialization segment and then sequentially fetching segments from a certain position in the stream based on the view switching point. The current implementation creates a DASHReceiver object which starts fetching segments immediately after its creation starting from initialization segment. Each segment is fetched in a separate thread. The decoder object LibavDecoder reads from an in-memory buffer and not from a file.
 * '''[Ahmed:2/11]''' Was able to render two frames side-by-side using SDL and OpenGL. Now working on integrating this with the DASH decoder.
@@ Line 16: / Line 17: @@
 * '''[Ahmed:1/28]''' Added detailed description of creating proper content for the 3D display.
 == People ==

@@ Line 237: / Line 237: @@
 *:The SDL header files should never try to redefine ''size_t'' on Linux platforms. We should properly install the SDL development package (libsdl1.2-dev) through apt-get/synaptic and edit the ''CMakeLists.txt'' file to point to the header files location on the system. For example:
 *:<pre>include_directories(/usr/include/SDL/)</pre>
-* For the Qt sample player that comes with the latest version of libdash. The CMakeLists.txt file seems to be empty since the developers are using Visual Studio. After installing Qt5, which is required by the code, the following CMakeLists.txt file can be used to build it. However, so far there are issues with linking. Note that the CMAKE_PREFIX_PATH must be properly set to the location of the CMake modules for Qt5 (/opt/Qt5.0.1/5.0.1/gcc_64/lib/cmake/ in the installation that we used).
+* For the Qt sample player that comes with the latest version of libdash. The CMakeLists.txt file seems to be empty since the developers are using Visual Studio. After installing Qt5, which is required by the code, the following CMakeLists.txt file can be used to build it. However, so far there are issues with linking. Note that the CMAKE_PREFIX_PATH must be properly set to the location of the CMake modules for Qt5 (/opt/Qt5.0.1/5.0.1/gcc_64/lib/cmake/ in the installation that we used). More useful CMake variables are listed [http://www.cmake.org/Wiki/CMake_Useful_Variables here].
 *:<pre> cmake_minimum_required(VERSION 2.8.8) &#10; &#10; set(CMAKE_CXX_FLAGS "-D__STDC_CONSTANT_MACROS")  &#10; # Find includes in corresponding build directories &#10; set(CMAKE_INCLUDE_CURRENT_DIR ON) &#10; # Instruct CMake to run moc automatically when needed. &#10; set(CMAKE_AUTOMOC ON) &#10; &#10; # Find the QtWidgets library &#10; find_package(Qt5Widgets) &#10; find_package(Qt5Core) &#10; find_package(Qt5Gui) &#10; find_package(Qt5Multimedia) &#10; find_package(Qt5MultimediaWidgets) &#10; &#10; include_directories(../libdash/include/) &#10; include_directories(./GeneratedFiles/) &#10; &#10; # Tell CMake to create the helloworld executable &#10; add_executable(qtsampleplayer main.cpp) &#10; &#10; # Use the Widgets module from Qt 5. &#10; qt5_use_modules(qtsampleplayer Core Widgets Gui Multimedia MultimediaWidgets)</pre>
 == DASH Options ==

@@ Line 80: / Line 80: @@
 The following script performs the resizing on all the frames of one component. It is assumed that the filename of the frame contains its number and the user specifies the range of frames to be resized as well as the target directory to save the resized frames. We can place the resized frames of both components in the same target directory.
-<pre>
+<source lang="bash">
 #!/bin/bash
@@ Line 95: / Line 95: @@
          convert $F -resize ${WIDTH}x${HEIGHT}\! ${TARGET_DIRECTORY}/$F
 done
-</pre>
+</source>
 Next, to combine the texture frame with the corresponding depth frame, the following script can be used.
-<pre>
+<source lang="bash">
 #!/bin/bash
@@ Line 115: / Line 116: @@
          montage -mode concatenate $COLOR $DEPTH ${TARGET_DIRECTORY}/${SEQ_NAME}_2dz_1920x540_f${i}.bmp
 done
-</pre>
+</source>
 Now we can use WMEncoder or ffmpeg to encode the frames. I personally prefer ffmpeg. We need to generate an uncompressed AVI file from the combined frames. This can be achieved using the following command:
@@ Line 229: / Line 230: @@
 *:<pre>error: ‘Sleep’ was not declared in this scope</pre>
 *:The ''Sleep()'' function is a Windows C++ function. One alternative to this function in Linux is ''usleep()''. This requires adding compiler directive to selectively choose one of the two functions based on the OS.
-*:<pre> #if defined _WIN32 || defined _WIN64 &#10;   Sleep((1 / (double)this->framerate) * 1000); &#10; #else &#10;   usleep((1 / (double)this->framerate) * 1000000); &#10; #endif</pre>
+*:<source lang="cpp"> #if defined _WIN32 || defined _WIN64 &#10;   Sleep((1 / (double)this->framerate) * 1000); &#10; #else &#10;   usleep((1 / (double)this->framerate) * 1000000); &#10; #endif</source>
 * To build the sampleplayer along with libdash, we need to edit the CMakeLists.txt by adding the following line:
 *:<pre>add_subdirectory(sampleplayer)</pre>

← Older revision		Revision as of 08:41, 26 March 2013
Line 235:		Line 235:
	*:The SDL header files should never try to redefine ''size_t'' on Linux platforms. We should properly install the SDL development package (libsdl1.2-dev) through apt-get/synaptic and edit the ''CMakeLists.txt'' file to point to the header files location on the system. For example:		*:The SDL header files should never try to redefine ''size_t'' on Linux platforms. We should properly install the SDL development package (libsdl1.2-dev) through apt-get/synaptic and edit the ''CMakeLists.txt'' file to point to the header files location on the system. For example:
	*:<pre>include_directories(/usr/include/SDL/)</pre>		*:<pre>include_directories(/usr/include/SDL/)</pre>
		+	* For the Qt sample player that comes with the latest version of libdash. The CMakeLists.txt file seems to be empty since the developers are using Visual Studio. After installing Qt5, which is required by the code, the following CMakeLists.txt file can be used to build it. However, so far there are issues with linking. Note that the CMAKE_PREFIX_PATH must be properly set to the location of the CMake modules for Qt5 (/opt/Qt5.0.1/5.0.1/gcc_64/lib/cmake/ in the installation that we used).
		+	*:<pre> cmake_minimum_required(VERSION 2.8.8) set(CMAKE_CXX_FLAGS "-D__STDC_CONSTANT_MACROS") # Find includes in corresponding build directories set(CMAKE_INCLUDE_CURRENT_DIR ON) # Instruct CMake to run moc automatically when needed. set(CMAKE_AUTOMOC ON) # Find the QtWidgets library find_package(Qt5Widgets) find_package(Qt5Core) find_package(Qt5Gui) find_package(Qt5Multimedia) find_package(Qt5MultimediaWidgets) include_directories(../libdash/include/) include_directories(./GeneratedFiles/) # Tell CMake to create the helloworld executable add_executable(qtsampleplayer main.cpp) # Use the Widgets module from Qt 5. qt5_use_modules(qtsampleplayer Core Widgets Gui Multimedia MultimediaWidgets)</pre>

← Older revision		Revision as of 02:34, 12 February 2013
Line 3:		Line 3:

	== Updates ==		== Updates ==
		+	* '''[Ahmed:2/11]''' Was able to render two frames side-by-side using SDL and OpenGL. Now working on integrating this with the DASH decoder.
	* '''[Ahmed:2/6]''' Managed to build the sample player that comes with ''libdash'' after modifications to build configuration files as well as some parts of the source code. The player works and the video is retrieved from the server, decoded, and displayed. I'm working on modifying it to render two video streams side-by-side. Our initial thought was that we can just use SDL to perform the rendering. However, it seems we might need to use OpenGL after all. Displaying video frames in SDL is achieved using YUV video overlays (SDL_Overlay) and it is not possible to display more than one overlay at the same time. A workaround may be to first merge the two decoded YUV frames side-by-side before drawing them on the overlay. But this must be done on-the-fly. I tried looking into ''libswscale'' to perform this, but I couldn't find anything that can help.		* '''[Ahmed:2/6]''' Managed to build the sample player that comes with ''libdash'' after modifications to build configuration files as well as some parts of the source code. The player works and the video is retrieved from the server, decoded, and displayed. I'm working on modifying it to render two video streams side-by-side. Our initial thought was that we can just use SDL to perform the rendering. However, it seems we might need to use OpenGL after all. Displaying video frames in SDL is achieved using YUV video overlays (SDL_Overlay) and it is not possible to display more than one overlay at the same time. A workaround may be to first merge the two decoded YUV frames side-by-side before drawing them on the overlay. But this must be done on-the-fly. I tried looking into ''libswscale'' to perform this, but I couldn't find anything that can help.

← Older revision		Revision as of 03:02, 8 February 2013
Line 30:		Line 30:
	[https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-projects/3DVideo/DASH-FTV https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-projects/3DVideo/DASH-FTV]		[https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-projects/3DVideo/DASH-FTV https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-projects/3DVideo/DASH-FTV]

		+
		+
		+	== System Design ==
		+	The envisioned system will operate as follows:
		+
		+	=== Components: ===
		+	*;DASH Server:
		+	*:The DASH server is a Web (HTTP) server hosting the DASH-encoded views of the free-viewpoint video (FVV) and their associated ''media presentation descriptor'' (MPD). Each view can have multiple representations, e.g., each representation corresponds to a certain bit rate. The server should support partial GET requests and byte ranges.
		+	*;DASH-FTV Client:
		+	*:The FTV client contains the following modules:
		+	: FTV player: is a video player capable of rendering stereoscopic video content using on of the common stereo formats, e.g., side-by-side.
		+	: View Switching Manager: is responsible for handling view switching requests issued by the user and deciding on the video streams to retrieve from the server for the chosen view-point.
		+	: libdash: handles the parsing of the retrieved MPDs and issueing HTTP requests to get the segments.
		+
		+	=== Operation: ===
		+	During the initialization process, the DASH-FTV client retrieves the MPDs of all the views of the FVV from the DASH server. This enables the client to quickly select different segments from different views at any point in time during playback. Based on the desired view-point, the DASH-FTV client selects two corresponding neighboring views that constitute a stereo pair from that view-point. Using libdash, the client then issues HTTP requests to retrieve a range of segments from the server for the chosen view pair with the same segment indices. The segments are passed to two separate DASH decoder instances running within the client on separate threads. Decoded views are then rendered side-by-side on the display. When the display is in 3D-mode, the stereo video will be perceived properly by the viewer.