Private:progress-harvey

Spring 2011 (GF)

• Courses: None

working on: Video Copy Detection using Optical Flow

Feb 23 - March 1

• I have created seversal test cased to examine the motion vectors I extract and validate them.
• Experimenting with motion vectors extracted gave results which did not seem correct. I have been digging into the source code to try and figure what is going on. I have been able to fix one major problem relating whether the prediction was based on the previous frame or the next frame. I am still looking into some others.

Feb 16 - Feb 22

• Using ffmpeg APIs I am now able to do the following:
  • Extract I-frames and save in jpg format for later analysis
  • Extract Motion vectors for B and P frames which I will use to build my histogram

Note: There does not seem to be a way to do this with ffmpeg in the compressed domain. I have to make a call to avcodec_decode_frame() in order to get the motion information and the frame type. There is a workaround for the I-frames. I can use a utility called ffprobe (use the R92 branch) with the -show_frames option from which a list of I-frames can be constructed, but since I need to decode the I-frames to decode subsequent B/P frames, I will just loop through and decode all frames. Theoretically, this can be run in hte compressed domain if one knew enough about the encoding protocal and had lots of time to write their own parser. I think that my approach is enough to show proof of concept for now.

Feb 9 - Feb 15

• Worked on motion vector extraction using ffmpeg APIs

Feb 2 - Feb 8

• Downloaded and compiled ffprobe
• Started coding using ffmpeg libraries to extract I-frames and motion vectors. I have almost got the I-Frame portion worked out, and I will look into the motion vectors next week

Jan 19 - Feb 1

• Clustered SURF pts with both k-means and x-means
• Looking into how to extract I-frames and motion information from video sequences
• Downloaded and compiled source files for ffmpeg
  • I think I can write something using this which will parse the mpegs
  • I am having problems with permissions when i try to install - working with Ahmed and Jason
• Found a Matlab m-file for extracting motion vectors - I am not sure this will be all that usefulMatlab m files
• Also found a reworking of mplayer: modified mplayer (modified for flow)
  • Janez Pers: The modifications are relatively minor, but ugly. The code that draws motion vectors is changed to dump the arrow directions and length into the text file. I cannot offer any support for compiling Mplayer though. The binary-only (windows) version is available here, it has added example video (it is better to start with this): windows binary. If you like the code and will use it in your scientific work, you can check my paper, which uses the same code for the second batch of experiments: Pers's paper.
  • Short instructions:
    • if you have MPEG4 already (I used the mpeg4 encoding as a fast way to get vectors as well), then skip the first step: mencoder original.avi -ovc lavc -lavcopts vcodec=mpeg4 -o mpeg4encoded.avi
    • now extract the motion vectors, without displaying the video (you can display the video as well, if you like, it was just more convenient for me) mplayer mpeg4encoded.avi -benchmark -lavdopts vismv=1
    • Now, the file opticalflow.dat will appear. Do not forget option vismv=1, the extraction is part of the visualisation.
    • The file opticalflow.dat has the following format: framenum,x,y,vx,vy (vx vy being the vectors, x y being the position of the block).
  • Be aware that the data for the I frames will be missing (no flow there). And, in my experience, lower bitrates give better flow than high ones - with high ones the encoder does not need to bother with the motion vectors, since it has enough bandwith already...

Jan 19-25

• TrecVid 2008 final transformation document with examples: Transformation Document
• TrecVid 2008 explanation of how transformations are generated: Transformation Explanation
• 2010 TrecVid Requirements
• Downloaded the TrecVid 2007 and 2009 databases with test cases and testcases for 2010
• Downloaded and investigating x-means experimental software (Licensed to me for research purposes only)
• Experimenting with SURF interext points and x-means clustering

Jan 12-18

• Finished survey of Video Copy Detection methods Survey
• Prepared presentation on Optical Flow My Presentation

Jan 11

• Survey of State of the Art Techniques

Fall 2010 (RA)

• Courses:
  • CMPT-820: Multimedia Systems
• worked on:
  • Video Copy Detection

Summer 2010 (TA)

• Courses:
  • None
• worked on:
  • Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management
• submitted
  • NetGames 2010: Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management (accepted)

Spring 2010 (RA)

• Courses:
  • CMPT-822: Special topics in Database Systems
  • CMPT 884: Computational Vision
• worked on:
  • Energy-Efficient Gaming on Mobile Devices
• submitted
  • Nosdav 2010: Energy-Efficient Gaming on Mobile Devices (not accepted)

Fall 2009 (TA)

• Courses:
  • CMPT-705: Algorithm
  • CMPT-771: Internet Architecture and Protocols