Scalable Multimedia Streaming

The Internet continues to change many aspects of our everyday life, including the way we communicate, learn, conduct business, and even entertain ourselves. With an increasing fraction of the population having access to high-speed network connections and the abundance of low-cost, powerful, computing devices, more services are continually made possible over the Internet. One such service that has recently been in high demand is multimedia communication in various forms, including on-demand streaming, live broadcasting, video conferencing, and collaborative virtual environments. For example, YouTube was reported to be the fastest growing web site in the Internet history; currently serving 200—300 million video sessions per day.

Participants in multimedia communication systems typically seek the highest possible quality, and the more they get used to multimedia services, the higher the quality they will expect and demand from the system. These participants, however, use quite heterogeneous computing resources. Heterogeneity arises from many sources, including network bandwidth, processing capacity, network delay, display resolution, and energy constraints. Supporting such heterogeneity is a challenging task. Current systems typically use nonscalable coding techniques for video streams. Therefore, they either degrade the quality for all participants to accommodate the least-powerful one, or they offer a very few (2—3) versions of the same stream to provide limited scalability. This is clearly insufficient in increasingly heterogeneous environments. Furthermore, nonscalable coding systems cannot fully utilize client resources, nor can they maximize the quality for all clients.

Unlike the case with nonscalable coders, various representations of the same video stream with different bit rates and qualities can be produced using scalable coders. Scalable coding techniques thus have the potential to support wide range of clients. A number of research challenges, however, need to be addressed to realize this potential. This goal of this project is to analyze and understand scalable coding techniques, and to design several optimization and streaming algorithms to make the best possible use of them in real multimedia systems. This will enable faster and wider adoption of scalable coding techniques in various multimedia communication systems, which will, in turn, yield better quality for users, and more efficient utilization of network and server resources. We consider scalability along several dimensions: quality (PSNR), power consumption, computation complexity, and multi-views (streams). We are also interested in efficient manipulation of scalable streams on special hardware devices (GPUs).

People

• Mohamed Hefeeda

• Cheng-Hsin Hsu (PhD Student)

• Yi Liu (MSc student)

Publications

• C. Hsu and M. Hefeeda, Partitioning of Multiple Fine-Grained Scalable Video Sequences Concurrently Streamed to Heterogeneous Clients, IEEE Transactions on Multimedia, Accepted November 2007.

• Hsu and M. Hefeeda, On the Accuracy and Complexity of Rate-Distortion Models for FGS-encoded Video Sequences, ACM Transactions on Multimedia Computing, Communications, and Applications, Accepted February 2007.

• M. Hefeeda and C. Hsu, Rate-Distortion Optimized Streaming of Fine-Grained Scalable Video Sequences, ACM Transactions on Multimedia Computing, Communications, and Applications, 4(1), Article 2, 28 Pages, January 2008.

• C. Hsu and M. Hefeeda, Optimal Coding of Multi-layer and Multi-version Video Streams, IEEE Transactions on Multimedia, 10(1), pp. 121--131, January 2008.

• C. Hsu and M. Hefeeda, Structuring Multi-Layer Scalable Streams to Maximize Client-Perceived Quality, In Proc. of IEEE International Workshop on Quality of Service (IWQoS'07), pp. 182--187, Chicago, IL, June 2007.

• C. Hsu and M. Hefeeda, Optimal Partitioning of Fine-Grained Scalable Video Streams, In Proc. of ACM International Workshop on Network and Operating Systems Support for Digital Audio & Video (NOSSDAV'07), pp. 63--68, Urbana-Champion, IL, June 2007.

• C. Hsu and M. Hefeeda, Optimal Bit Allocation for Fine-Grained Scalable Video Sequences in Distributed Streaming Environments, In Proc. of 14th ACM/SPIE Multimedia Computing and Networking Conference (MMCN'07), pp. 1--12, San Jose, CA, Jan 2007.

• C. Hsu and M. Hefeeda, Rate-Distortion Models for FGS-encoded Video Sequences, In Proc. of IEEE International Conference on Computer Theory and Applications (ICCTA'06), pp. 334--337, Alexandria, Egypt, September 2006.

Software and Data

We have augmented the MPEG-4 FGS reference software to collect rate-distortion information at encoding time. We have also developed scripts (in perl and/or Matlab) to analyze this information and to generate rate-distortion meta data. Using the rate-distortion meta data, we design several bit-allocation algorithms for streaming applications (see our paper for details). We provide our programs and data in the following tar files:

• Programs: contains the augmented reference software and scripts.

• Data: contains experimental data for three video sequences: Foreman, Mobile, and Bee.

• Howto.txt: contains the instructions of how to use our software.

• Plots_Howto.txt: contains a list of Matlab commands that were used to generate figures from experimental data.