Video Library and Tools

TV Captures

We capture TV programs with the PCI version of ATI HD 650 PCI Digital and Analog TV tuner. We install this card in a Windows XP desktop, and we use the accompanying application called ATI Crystal Media Center to watch and capture TV. One of our goal is to extract the closed captions into timstamped text files. This can be done as follows. We first capture a TV program using the default recording profile Best in the Crystal, which results in an mpg file that contains a video, an audio, and an CC tracks. The resulting mpg file can be played by the VLC player but not by the Windows media player nor QuickTime. We then use an open source software called ccextractor to separate the CC text from this mpg file. ccextractor comes with a Windows GUI, but can also be run in command line as ccextractor -out=srt my.mpg. This command creates an srt file that contains close captions with timestamps.

We have put several sample .mpg and .srt files online at our web server. Each of these capture is about 50 min long, and they were captured from an over-the-air digital TV station.

Video Sequences

HD Sequences from CBC

• Documentary 2MB WMV, Documentary 4MB WMV, Documentary 20MB dvix.
• Jeux 2MB WMV, Jeux 4MB WMV, Jeux 20MB dvix.
• Soap 2MB WMV, Soap 4MB WMV, Soap 20MB dvix.
• Sport2MB WMV, Sport4MB WMV, Sport 20MB dvix.
• News 2MB WMV, News 4MB WMV, News 20MB dvix.

Raw Sequences from the H.264/SVC Group

QCIF

• Bus 7.5 fps, Football 7.5 fps, Foreman 7.5 fps, Mobile 7.5 fps
• Bus 15 fps, Football 15 fps, Foreman 15 fps, Soccer, 15 fps, Crew 15 fps, Ice 15 fps, Mobile 15 fps, City 15 fps, Harbour 15 fps

CIF

• Foreman 15 fps, Harbour 15 fps, Ice 15 fps, Mobile 15 fps, Crew 15 fps, Bus 15 fps, Football 15 fps, City 15 fps, Soccer 15 fps
• Football 30 fps, Harbour 30 fps, Ice 30 fps, Mobile 30 fps, Crew 30 fps, Bus 30 fps, Foreman 30 fps, City 30 fps, Soccer 30 fps

4CIF

• City 30 fps, Crew 30 fps, Soccer 30 fps, Harbour 30 fps, Ice 30 fps
• City 60 fps, Crew 60 fps, Soccer 60 fps, Harbour 60 fps, Ice 60 fps

Raw Sequences from ftp.ldv.e-technik.tu-muenchen.de

720p

• Parkrun Long shot. Man running in a park with an umbrella in his hand. Trees, snow and water in the background. Very detailed and demanding.
• Shields Man with beard and a speckled jacket walking in front of a wall of detailed knight shields. Zoom at the end of the sequence.
• Stockholm Panning view over the Old Town of Stockholm. Detailed houses, water and moving cars.Panning view over the Old Town of Stockholm. Detailed houses, water and moving cars.

1080i

• Mobcal Similar to Mobile and Calendar. Close up. Moving calendar with text and a detailed photo of the ship. Moving train with colorful toys. Background with two types of wallpaper, one brown with details and one yellow with drawn figures. Very detailed and normally demanding.
• Parkrun Long shot. Man running in a park with an umbrella in his hand. Trees, snow and water in the background. Very detailed and demanding.
• Shields Man with beard and a speckled jacket walking in front of a wall of detailed knight shields. Zoom at the end of the sequence.
• Stockholm Panning view over the Old Town of Stockholm. Detailed houses, water and moving cars.Panning view over the Old Town of Stockholm. Detailed houses, water and moving cars.

1080p

• BlueSky Top of two trees against blue sky. High contrast, small color differences in the sky, many details. Camera rotation.
• PedestrianArea Shot of a pedestrian area. Low camera position, people pass by very close to the camera. High depth of field. Static camera.
• Riverbed Riverbed seen through the water. Very hard to code.
• RushHour Rush-hour in Munich city. Many cars moving slowly, high depth of focus. Fixed camera.
• Station View from a bridge to munich station. Evening shot. Long zoom out. Many details, regular structures (tracks).
• Sunflower Sunflower, very detailed shot. One bee at the sunflower, small color differences and very bright yellow. Fixed camera, small global motion.
• Tractor A tractor in a field. Whole sequence contains parts that are very zoomed in and a total view. Camera is following the tractor, chaotic object movement, structure of a harvested field. Very red wheels of the tractor.

Note: Descriptions are extracted from the readme files.

Video Tools

• WebM Project: An Open video format for the web that is freely available to everyone.

• Open SVC Decoder

• Windows YUV Player: executable, source.
  • Allows multiple YUV video files to be played side by side for comparison

• PYUV Player for Linux

• Imagen, Image Sequence Player http://www.gromada.com/imagen.html

• Matlab YUV Toolbox http://sprljan.com/nikola/matlab/yuv.html
  • yuv_compare compares two YUV streams and produces PSNR values

• Tools (Requires Login)
• x264 open source encoder for H.264/AVC

• Google's VP8 open source VP8 encoder and decoder
  • ZIP file includes encoder and decoder pre-built for windows
  • Encoder is vpcenc.exe, type vpcenv -help for instructions
  • Decoder is vpcdec.exe, type vpcdec -help for instructions
  • VP8 Encoder Parameter Guidelines
  • VP8 Overview Slides

• Avisynth version 2.5
  • General purpose scripting tool for reading and processing videos
  • Allows filters to be bound in to compute distortion, add noise, watermarks, etc.

Useful avisynth filters:
* Compare - Compare two videos and Calculate Y-PNSR for each frame
   - This is included as one of the core avisynth filters
     Compare(clip_filtered, clip_original, "channels", "logfile") 
		
* SSIM - Compare two videos Calculate Structure Similarity Metric for each frame
   - http://avisynth.org/warpenterprises/
     SSIM (clip1, clip2, "results.csv", "averageSSIM.txt", lumimask=true)

* RawSource - Read in Raw YUV video files
   - http://avisynth.org/warpenterprises/files/rawsource_25_dll_20060728.zip
     clip = RawSource("SourceFile", Height, Width, "Format")

• ffmpeg
  • a complete, cross-platform solution to record, convert and stream audio and video
  • includes libavcodec - the leading audio/video codec library
  • contains instructions for getting source and windows executable

• ViCoS (Demo Version)
  • Automated codec comparison tool
  • Allows codecs to be bound in and preset parameters to be defined by the user
  • Useful for a quick relative comparison of codecs and different preset parameters
  • Generated files in vicos/perframe_data contains per-frame PNSR/SSIM raw data

• MSU's Codec Comparison Reports
  • Contains short version of annual Codec Comparison Report
  • Provides detailed comparison of the most popular H.264 decoders
  • MSU's 2010 Codec Report - Appendix 8 compares VP8, x264, XViD performance and quality

Links to other Video Libraries

• The Consumer Digital Video Library

• Open-Video Project

• Video Traces Research Group

• The Video Quality Experts Group

• Mobile3DTV Project's 3D Video Database