H.264/SVC Scalable Streams Management System
A complete system for structuring, encoding, adapting, streaming, storing, and transcoding of video streams encoded by the state-of-the-art H.264/SVC coding standards.
Ideas
- We want to create a system and/or APIs for handling SVC streams.
- We want to demonstrate the benefits of the system for the administrators (ease and efficiency) and users (quality)
- Approach: we use open source encoders (e.g., x.264/AVC) and enhance it with SVC features from the open source reference implementation of SVC. We can take/enhance selected parts from SVC. We can also integrate the whole thing into an open source streaming server, e.g., the one from Apple. Of course, our novel parts will be based on solutions of some of the research problems listed below.
- If we have the above SVC management system, we can do many things, e.g., video conference tools, streaming servers (out of the box), SVC-enabled peer-assisted content distribution networks, SVC mobile TV, ...
- Who can benefit from the system? VoD Systems, YouTube, Google Videos, CBC, Nokia, Rogers (mobile TV), ...
Research Problems
- Design an algorithm to compute the best way (how many layers, types of layers, size of each layer, etc) to encode an SVC stream for an expected client distribution. Enhance our previous works in ToM 2008. See also Schwarz and Wiegand, R-D Optimized Multi-Layer Encoder Control for SVC, In Proc. of IEEE ICIP 2007.
- Assume we have a full quality scalable video stream, with temporal, spatial, and quality layers. Given a client with specific resources (screen resolution, estimated BW, CPU, battery), what is the best substream we should stream to that client? E.g., is it better to reduce the frame rate, resolution, or the PSNR quality? How about a mix? Of course the right mix will depend on the terminal (PDA vs dialup desktop). This research problem will rely on previous works on subjective quality measures and user studies. Then, we design a systematic method for determining the best substream. A recent related work is Wong et.al, Classification-based multidimensional adaptation prediction for scalable video coding using subjective quality evaluation, IEEE Transactions on Circuits and Systems for Video Technology, 15(10), October, 2005.
- SVC peer-assisted Content Distribution Network (pCDN). This will extend the pCDN system to support SVC streams. Research problems include: coordinating multiple senders with different substreams to serve a client. See our previous work on MPEG-4 FGS streams in TOMCCAP 2008. Another problem is analyzing the system stream capacity and average client quality with and without SVC. Analytic and/or simulation analysis is needed. In addition, we may be able to get real data from CBC after they start using our pCDN system.
- Can we use SVC in live streaming, e.g., broadcasting popular sports event to many heterogeneous clients? Do we need multiple multicast trees? Other structures? Very interesting research problem as well.
Links and References
- Scalable Video for Peer-to-Peer Streaming A system paper (MS thesis) on streaming scalable video over P2P networks. The author enhanced an existing P2P application called Pulsar. The evaluation is flawed. Nonetheless, it's one of the earliest implementation paper I have seen.
- Client for Adaptive H.264/SVC Reception A project report (done in 2007) discuss how to interface the H.264/SVC reference software with vlc media player. Details on how to wrap the reference software as a vlc plugin are given. The resulting plugin runs on both Windows and Linux platforms. Unfortunately, the source code is not open, nevertheless this report gives a lot implementation details that will speed up reproducing their implementation.
- Open Source SVC coders/decoders:
- JSVM SVC Reference Software Implementation
- Open SVC Decoder. Features of the decoder can be found here.
- The P2P-Next consortium provides its SVC software (encoder/decoder) as open source under LGPL which comprises an optimized version of the JSVM for both encoding and decoding.
- Commercial SVC coders/decoders:
- Global IP Solutions offers a suite of video codecs designed for a variety of environments. These codecs including their H.264 SVC implementation are bundled with GIPS VideoEngine, a voice and video processing solution optimized for softphone applications on PC platforms which provides a high-level API for developers.
- VSofts offers software development kits that offer the ability to integrate VSofts video compression technology into a wide array of applications. This includes an H.264/AVC SDK which also supports SVC coding/decoding for multiple-resolution profiles. They are supported for Windows, Mac OS X and Linux development. VSofts also offers an H.264 Mobile Broadcaster, which is an implementation of the H.264/AVC encoder designed for satellite broadcasting to mobile devices.
- Schwarz et al., Overview of the Scalable Video Coding Extension of the H.264/AVC Standard, IEEE Transactions on Circuits and Systems for Video Technology, 17(9), Sep 2007
- In the above reference, [40] presents an algorithm to control the bit rates of layers in SVC, [46] describes extracting substreams, [47] describes re-writing SVC streams to AVC streams.
- Other people doing this:
- P2P-Next is an integrated project that aims to build a next generation P2P content delivery platform, to be designed, developed, and applied jointly by a consortium consisting of high-profile academic and industrial players
- SCALNET focuses on needed adaptations in core, access and home networks, wired and wireless, fixed and mobile, in order to efficiently deploy the SVC (Scalable Video Coding) technology.
- Important: Industrial needs and interest? Any citations on this need? What are the current practices for managing many video streams?
- Who can benefit from the system? VoD Systems, YouTube, Google Videos, CBC ...
- VQM
