VS:SVC

H.264/SVC Scalable Streams Management System

Back to the main VS:Video Semantics

Overview

A complete system for structuring, encoding, adapting, streaming, storing, and transcoding of video streams encoded by the state-of-the-art H.264/SVC coding standards.

Ideas

• We want to create a system and/or APIs for handling SVC streams.

• We want to demonstrate the benefits of the system for the administrators (ease and efficiency) and users (quality)

• Approach: we use open source encoders (e.g., x.264/AVC) and enhance it with SVC features from the open source reference implementation of SVC. We can take/enhance selected parts from SVC. We can also integrate the whole thing into an open source streaming server, e.g., the one from Apple. Of course, our novel parts will be based on solutions of some of the research problems listed below.

• If we have the above SVC management system, we can do many things, e.g., video conference tools, streaming servers (out of the box), SVC-enabled peer-assisted content distribution networks, SVC mobile TV, ...

• Need to explore market needs and interest. What are the current practices for managing many video streams?

• Who can benefit from the system? VoD Systems, YouTube, Google Videos, CBC, Nokia, Rogers (mobile TV), ...

• Other people working/using/pushing SVC
  • Vidyo: they are doing video conferencing using SVC. They sell complete systems. We would be selling backend software to enable SVC video conferencing, not complete systems. So, our library could be used by companies like Sony, Cisco, and others to enhance their video conference products. At a latter stage, we may do the same as Vidyo and compete with them (on the price, may be).

• • Some chip manufacturers (e.g., Stretch have started producing products to process SVC streams. Implies that SVC is a promising technology.

Research Problems (Papers and/or Patents)

• Design an algorithm to compute the best way (how many layers, types of layers, size of each layer, etc) to encode an SVC stream for an expected client distribution. Enhance our previous works in ToM 2008. See also Schwarz and Wiegand, R-D Optimized Multi-Layer Encoder Control for SVC, In Proc. of IEEE ICIP 2007.

• Assume we have a full quality scalable video stream, with temporal, spatial, and quality layers. Given a client with specific resources (screen resolution, estimated BW, CPU, battery), what is the best substream should we stream to that client? E.g., is it better to reduce the frame rate, resolution, or the PSNR quality? How about a mix? This is quite useful and could be a differentiating edge for our technology (good patent). Of course the right mix will depend on the terminal (PDA vs dialup desktop). This research problem will rely on previous works on subjective quality measures and user studies. Then, we design a systematic method for determining the best substream. This problem is also related to the previous one. I have not yet seen works on this problem. If we come up with a good solution for this problem, it could make a separate product by itself.

• SVC peer-assisted Content Distribution Network (pCDN). This will extend my current pCDN system to support SVC streams. Research problems include: coordinating multiple senders with different substreams to serve a client. See our previous work on MPEG-4 FGS streams in TOMCCAP 2008. Another (academic) problem is analyzing the system stream capacity and average client quality with and without SVC. Analytic and/or simulation analysis is needed. In addition, we may be able to get real data from CBC after they start using our pCDN system, insha'a ALLAH.

• Can we use SVC in live streaming, e.g., broadcasting popular sports event to many heterogeneous clients? Do we need multiple multicast trees? Other structures? Very interesting research problem as well.

Links and References

• x.264 -- Decent Open Source H.264/AVC Encoder

• Open Source SVC coders: Reference Software Implementation

• Schwarz et al., Overview of the Scalable Video Coding Extension of the H.264/AVC Standard, IEEE Transactions on Circuits and Systems for Video Technology, 17(9), Sep 2007
  • In the above reference, [40] presents an algorithm to control the bit rates of layers in SVC, [46] describes extracting substreams, [47] describes re-writing SVC streams to AVC streams.