H.264/SVC Scalable Streams Management System
A complete system for structuring, encoding, adapting, streaming, storing, and transcoding of video streams encoded by the state-of-the-art H.264/SVC coding standards.
Ideas
- We want to create a system and/or APIs for handling SVC streams.
- We want to demonstrate the benefits of the system for the administrators (ease and efficiency) and users (quality)
- Approach: we use open source encoders (e.g., x.264/AVC) and enhance it with SVC features from the open source reference implementation of SVC. We can take/enhance selected parts from SVC. We can also integrate the whole thing into an open source streaming server, e.g., the one from Apple. Of course, our novel parts will be based on solutions of some of the research problems listed below.
- If we have the above SVC management system, we can do many things, e.g., video conference tools, streaming servers (out of the box), SVC-enabled peer-assisted content distribution networks, SVC mobile TV, ...
- Who can benefit from the system? VoD Systems, YouTube, Google Videos, CBC, Nokia, Rogers (mobile TV), ...
Research Problems
- Design an algorithm to compute the best way (how many layers, types of layers, size of each layer, etc) to encode an SVC stream for an expected client distribution. Enhance our previous works in ToM 2008. See also Schwarz and Wiegand, R-D Optimized Multi-Layer Encoder Control for SVC, In Proc. of IEEE ICIP 2007.
- Assume we have a full quality scalable video stream, with temporal, spatial, and quality layers. Given a client with specific resources (screen resolution, estimated BW, CPU, battery), what is the best substream should we stream to that client? E.g., is it better to reduce the frame rate, resolution, or the PSNR quality? How about a mix? This is quite useful and could be a differentiating edge for our technology (good patent). Of course the right mix will depend on the terminal (PDA vs dialup desktop). This research problem will rely on previous works on subjective quality measures and user studies. Then, we design a systematic method for determining the best substream. This problem is also related to the previous one. A recent related work is Wong et.al, Classification-based multidimensional adaptation prediction for scalable video coding using subjective quality evaluation, IEEE Transactions on Circuits and Systems for Video Technology, 15(10), October, 2005.
- SVC peer-assisted Content Distribution Network (pCDN). This will extend the pCDN system to support SVC streams. Research problems include: coordinating multiple senders with different substreams to serve a client. See our previous work on MPEG-4 FGS streams in TOMCCAP 2008. Another problem is analyzing the system stream capacity and average client quality with and without SVC. Analytic and/or simulation analysis is needed. In addition, we may be able to get real data from CBC after they start using our pCDN system.
- Can we use SVC in live streaming, e.g., broadcasting popular sports event to many heterogeneous clients? Do we need multiple multicast trees? Other structures? Very interesting research problem as well.
Links and References
- Open Source SVC coders: Reference Software Implementation
- Schwarz et al., Overview of the Scalable Video Coding Extension of the H.264/AVC Standard, IEEE Transactions on Circuits and Systems for Video Technology, 17(9), Sep 2007
- In the above reference, [40] presents an algorithm to control the bit rates of layers in SVC, [46] describes extracting substreams, [47] describes re-writing SVC streams to AVC streams.
- Other people doing this?
- Important: Industrial needs and interest? Any citations on this need? What are the current practices for managing many video streams?
- Who can benefit from the system? VoD Systems, YouTube, Google Videos, CBC ...
