Summer 2012 (RA)
Course: CMPT894 (Directed Reading) bi-weekly report updated:10/06
Jun 15
- Implemented a m3u8 based dynamic http streaming scheme utilising the rewriting functionality. Conceptually it works and I can play the video which is processed through the rewriting work flow. Following is an example playlist file with two version of a stream:
---bbb.m3u8--- #EXTM3U #EXT-X-STREAM_INF:PROGRAM-ID=1,BANDWIDTH=149300 'http://142.58.185.226:8080/m3u8/bbb_l11.m3u8' #EXT-X-STREAM_INF:PROGRAM-ID=1,BANDWIDTH=137700 'http://142.58.185.226:8080/m3u8/bbb_l10.m3u8' ---bbb_l11.m3u8--- #EXTM3U #EXT-X-TARGETDURATION:5 #EXT-X-MEDIA-SEQUENCE:1 #EXTINF:5 'http://142.58.185.226:8080/AvcRewriter.php?file=bbb_l11_part_01.ts' #EXTINF:5 'http://142.58.185.226:8080/AvcRewriter.php?file=bbb_l11_part_02.ts'
- Configuring C++ applications did not work with either of the three servers(apache, nginx, lighttpd) I tried. Therefore I implemented a basic PHP application with Apache to call the rewriter binaries on server. This solution works and I can play the video on VLC but the calls to each video chunk file take some time to process so the playout is not smooth.
- There are some components of the delay that we did not consider before:
- The SVC substream extraction process has to be added to the rewriter module to make the streaming adaptive. This adds some delay as the extraction process needs to go through the entire file and this cannot be done in conjunction with the rewriting on the fly.
- The second component is the MUX-ing software(ffmpeg) which is currently taking lot of time(probably because it is initializing many resources which are not required for our particular case. One solution to this is to integrate the muxing component into rewriter module as well and have a combined substreamExtractor-Rewriter-Muxer biinary.
- Another challenge I am facing is with the client software. Only some versions of VLC play the stream and I am not able to observe if they are employing any bandwidth adaptation strategy or not.
Jun 10
- I am trying to implement the connection between web server and C++ application which is turning out to be more difficult than expected. I tried with two low footprint servers nginx and lighttpd but the mechanism is not working due to their asynchronous request processing. Currently trying with Apache.
- As per the MPEG-DASH standard the content has to be created before creating the MPD file which rules out the possibility of using MPEG-DASH with on demand avcRewrite. One option is to use the .m3u8 playlists that have simpler format with explicit urls for file chunks. In that case we have to mux the video into TS.
- I did some modifications and obtained the following table for large number of layers. As expected, the storage requirement for AVC becomes much higher than SVC. One interesting observation is that for the top two layers SVC files are actually smaller in size than their AVC versions.
LayerID Resolution Framerate Bitrate SVC(KB) AVC(KB)
============================================
2 352x288 12.0000 28.20 446 483
3 352x288 24.0000 31.10 487 559
4 352x288 3.0000 60.50 938 726
5 352x288 6.0000 65.10 1027 811
6 352x288 12.0000 69.50 1116 903
7 352x288 24.0000 76.00 1204 1004
8 352x288 3.0000 115.80 1954 1269
9 352x288 6.0000 126.00 2149 1428
10 352x288 12.0000 134.40 2330 1580
11 352x288 24.0000 145.80 2499 1728
============================================
10491(4.2x)
May 28
- Created 1min of AVC video chunks and encoded into mp4 files.
- Storage calculations (bytes) for a 1 min CIF sized BigBuckBunny Video split into 5sec chunks:
- Size of the raw SVC files: 2565058
- Size of raw AVC files: 1776821(L3) + 902641(L2) + 438308(L1) = 3117770 (~21% extra compared to SVC)
- Audio and container overhead SVC : 4366459 - 1776821 = 2589638
- Est. size of playable SVC content: 2565058 + 2589638 = 5154696
- Size of playable Simulcast content: 4366459 + 3580554 + 2535836 = 10482849 (~ 100% extra compared to SVC, but this will probably decrease with larger picture size or video length when the video data dominates the size).
- Audio encoded in AAC stereo using NeroAAC for Linux and is same for both the SVC and simulcast versions.
- Video encoded in 3 layer rewritable SVC with layer QPs: 36, 32, 28
- AVC files for simulcast are created from the SVC file by first extracting a substream and then rewriting it.
- ffmpeg and MP4Box is used for muxing the AVC and AAC files into an MP4 file with the following commands(Note: currently this is the only command that is working. Muxing the AVC file with MP4Box tools do not work):
$>ffmpeg -i ../avc/BBB_CIF_24fps_1min_GOP01.h264 -s 352x288 -r 24000/1001 -t 00:00:05 bbb_muxed_01.mp4 $>MP4Box -add ../aac/bbb_01.mp4 bbb_muxed_01.mp4
- Currently Investigating:
- JSVM layer extracter does not work with rewriter. E.g. I have a 3 layer rewritable SVC file. When I rewrite all 3 layers, the our put is playable in VLC. When I extract 2 layers and then rewrite, the output is no longer usable. It does not show any difference in output compare to the 3 layer files in VLC player or ffmpeg. It has severe problems with audio synchronization and drifting.
- Interface between web server and media files. Is may be better idea to integrate a HTTP server into the C++ media adaption/rewriting application than to implement link code between standard http server and the media application through php/CGI. The later is slower.
- Waiting for: Kaushik's update on the DASH setup. Earlier sent him a set of mp4 files for creating DASH content and verifying the setup with it.
- Challenges: The JSVM encoder is very slow. took me 3 days to encode 1 min of CIF video.
May 15
- Currently working on creating chunked content from YUV files and composing the DASH MPD file.
- Tested that video chunks created by splitting YUV files and AVC rewritten can be played back in VLC media player. Also investigated that the rewrite works with layer dropping.
- Storage implications of splitting YUV file instead of splitting compressed file for parallel processing :
Video: bridge_far.yuv, CIF(352x288), 30fps, 900 frames(30s) Encoding: 3layer SVC then rewritten to AVC, GOP size 16, YUV split into 2s chunks Size of single compressed video of 30s = 723191b = 706kb (Note that this contains only one IDR frame) Size of 15 videos of length 2s each = 829674b = 810kb.
Spring 2012 (TA + RA)
Course: CMPT886 (Multicore Systems)
January
Jan 27:
- Conducted a survey of exiting results on SVC to AVC rewriting. While it has been reported that the re-writing process performs about 80% faster than the cascaded trans-coding process on a CPU, there are not many results on the feasibility of streaming implementation of the technique. Sablatschan et.al. have reported that the real time rewriting is not feasible for resolutions more than 480x320 videos using a quad core 3GHz processor. This leads to two directions for exploration: (1)explore the JSVM code to find better ways of doing the rewriting in parallel hardware and (2) using GOP distribution based approach on cloud to explore possibility of improving parallel performance. (1) has been extensively studied in the context of parallel encoders and decoders. (2) has scalability issues because it needs to buffer the GOPs before distributing them.
- One idea here is to distribute the pictures instead of GOPs which will increase the achievable degree of parallelism at the cost of increased picture management overhead.
- Another idea is to use the GOP based parallel rewriting at the server where the entire video is available. Then there will e no need to buffer the GoPs. Need to check whether this has been already done.
- Looked at some possibilities to implement scalable video streaming. Not many streaming server applications support SVC. Will look info DASH related implementations.
Jan 10:
- Started exploring the problem of video trans-coding in cloud
- Started exploring the feasibility of implementing a cloud test-bed using Openstack.
Other
- Courses: CMPT 886 (Multicore Systems)
- TA : CMPT 371 , CMPT 379
Previous work
Spring 2011 (RA)
April
- April 8
- Working on the simulator for hybrid uni/multicast experiments. Taking longer than expected, may miss the MM'11 deadline
March
- Mar 28:
- Updated tech-report on hybrid multicast-unicast link
- Mar 7:
- Tech-report on hybrid multicast-unicast here
- Working on the formulation for mobile patching scheme. Derived an expression for the bandwidth requirement and energy consumption of all-unicast and adaptive-patching schemes. Need to verify correctness analytically.
- Working on numerical examples of the mobile patching scheme.
- Documentation GENI-WiMAX project details.
Feb
- Feb 28:
- Reviewed the existing literature and identified the main challenges in adapting video streaming schemes to wireless networks. Identified four major schemes which seem promising : Skyscraper Broadcasting, Hierarchical Stream Merging, Harmonic Patching and Piggybacking.
- Explored Skyscraper Broadcasting scheme and found it to be unsuitable.
- Documentation of the survey.
- Feb 11: Meeting with Saleh
- Saleh to survey energy efficiency techniques in ad-hoc/sensor network domain and possible adaptation
- Som to survey Internet VoD results and look for possible adaptation in the wireless domain
- Meet on Feb 15 to discuss progress and decide on the first draft of one or two problems
- (If time) Saleh to look at WiMAX model in OPNET to see if it can be used for experiments
- Investigating wireless multimedia streaming in muticast/unicast mixed mode networks
Jan
- Investigating cloud computing for video trans-coding, video mining and mobile video applications
- Survey report on WiMAX/LTE testbed design options here
- Report on current status of DVB-H testbed and design of EPG feature here
Fall 2010 (RA)
- Poster/Demo: Efficient Multiplexing for Mobile Video Streaming (CONNECT'10)
