https://nmsl.cs.sfu.ca/api.php?action=feedcontributions&feedformat=atom&user=Feig 2026-06-05T13:42:44Z User contributions MediaWiki 1.35.1 https://nmsl.cs.sfu.ca/index.php?title=News&diff=4746 2012-01-05T06:53:34Z

<p>Feig: </p> <hr /> <div>== 2011 ==<br /> <br /> * December 2011: Fei Gao graduates (MSc) from NSL. Congratulations!<br /> <br /> * September 2011: Cameron Harvey graduates (MSc) from NSL and joins Amazon.com in Seattle, WA. Congratulations! Abdullah and Masum join NSL as MSc students. Welcome aboard. <br /> <br /> * June 2011: Dr. Hefeeda is awarded one of the prestigious [http://www.nserc-crsng.gc.ca/Professors-Professeurs/Grants-Subs/DGAS-SGSA_eng.asp NSERC Discovery Accelerator Supplements (DAS)], which are granted to 123 researchers in all Science and Engineering disciplines in Canada (selected from more than 3,400 applicants in 2011). <br /> <br /> * June 2011: Hamed Neshat graduates (MSc) from NSL and joins Microsoft in Seattle, WA. Congratulations!<br /> <br /> * April 2011: Farid Tabrizi graduates (MSc) from NSL. Congratulations!<br /> <br /> * April 2011: Mohammad Alkurbi graduates (MSc-Project) from NSL. Congratulations!<br /> <br /> <br /> == 2010 == <br /> <br /> * December 2010: Shabnam Mirshokraie graduates (MSc) from NSL and joins UBC as a PhD student. Congratulations!<br /> <br /> * December 2010: Somsubhra Sharangi graduates (MSc) from NSL. Congratulations! <br /> <br /> * October 2010: Our [http://www.nserc-crsng.gc.ca/professors-professeurs/rpp-pp/spg-sps_eng.asp NSERC Strategic Project Grant] application on Mobile Gaming and 3D Video Systems: Next Generation Services for Wireless Networks is accepted. <br /> <br /> * October 2010: Dr. Hefeeda becomes an Associate Editor of the premier [http://tomccap.acm.org/ ACM Transactions on Multimedia Computing, Communications and Applications (TOMCCAP)] journal. <br /> <br /> * September 2010: Dr. Hefeeda becomes program co-chair of the [http://www.icme2011.org/ IEEE International Conference on Multimedia and Expo (ICME 2011)] conference. <br /> <br /> * August 2010: Yuanbin Shen graduates (MSc) from NSL. Congratulations!<br /> <br /> * '''July 2010: Dr. Hefeeda is promoted to the associate professor rank and granted tenure. '''<br /> <br /> * June 2010: Cong Ly graduates (MSc) from NSL and joins Microsoft in Vancouver, BC. Congratulations!<br /> <br /> <br /> <br /> == 2009 == <br /> <br /> * December 2009: Yi Liu graduates (MSc) from NSL and joins Incognito Software Inc. in Vancouver, BC. Congratulations!<br /> <br /> * December 2009: Greg Kowalski graduates (MSc) from NSL Congratulations!<br /> <br /> * December 2009: Dr. Hefeeda becomes the vice chair of the Distributed Multimedia track in the [http://www.ftrg.org/emc2010/index.php Embedded and Multimedia Computing (EMC-10) conference]. <br /> <br /> * '''November 2009: Cheng-Hsin Hsu graduates (PhD) from NSL and joins the Deutsche Telekom R&amp;D Lab in California as a senior researcher. Congratulations Dr. Hsu -- the first PhD graduate from our group. ''' <br /> <br /> * November 2009: Dr. Hefeeda becomes a '''Senior Member of the IEEE'''. <br /> <br /> * November 2009: Dr. Hefeeda becomes the Preservation Editor of the ACM Special Interest Group on Multimedia (SIGMM) Web Magazine.<br /> <br /> * November 2009: Farid and Fei join NSL as MSc students. Welcome aboard. <br /> <br /> * October 2009: Dr. Hefeeda becomes the program chair of [http://nsl.cs.sfu.ca/nossdav10/ NOSSDAV 2010].<br /> <br /> * September 2009: Cameron and Hamed join NSL as MSc students. Welcome aboard. <br /> <br /> * September 2009: Kianoosh Mokhtarian graduates (MSc) from NSL and joins Mobidia in Richmond, BC. Congratulations!<br /> <br /> * July 2009: Our mobile TV research is also featured in the July issue of the [http://technews.acm.org/archives.cfm?fo=2009-07-jul/jul-01-2009.html#418148 ACM Tech News.]<br /> <br /> * June 2009: Our mobile TV research is featured on CTV British Columbia News: [http://www.ctvbc.ctv.ca/servlet/an/local/CTVNews/20090625/bc_mobile_tv_090625/ see article] or [[media:ctv09.pdf | local PDF.]]<br /> <br /> * May 2009: Cheng-Hsin Hsu (NSL PhD student) is featured in [http://www.sfu.ca/sfunews/_pvw9C438741/news/story_05290909.shtml SFU News.] <br /> <br /> * January 2009: Ahmed Hamza joins NSL as PhD student. Mohammad Alkurbi joins NSL as MSc-Project student. Welcome aboard.<br /> <br /> <br /> <br /> == 2008 == <br /> <br /> * December 2008: Our paper on [http://www.cs.sfu.ca/~mhefeeda/Papers/innovations08.pdf Energy Optimization in Mobile TV Broadcast Networks] wins the '''Best Paper Award''' in the IEEE Innovations 2008 Conference. <br /> <br /> * October 2008: Our Mobile TV Testbed wins the [http://www.cs.sfu.ca/~mhefeeda/Papers/mm08DemoAward.pdf Best Technical Demonstration Award] in ACM Multimedia 2008. <br /> <br /> * September 2008: Cong Ly, Yuanbin Shen, and Shabnam Mirshokraie join NSL as MSc students. Welcome aboard. <br /> <br /> * ''' March 2008: This Wiki system has been launched.'''<br /> <br /> * February 2008: Yi Liu joins NSL as MSc student. Welcome aboard Yi!<br /> <br /> <br /> <br /> == 2007 == <br /> <br /> * September 2007: Kianoosh Mokhtarian joins NSL as MSc student. Welcome aboard Kianoosh!<br /> <br /> * December 2007: Behrooz Noorizadeh Graduates (MSc) from NSL joins Eyeball Networks in Vancouver. .<br /> <br /> * July 2007: Hossein Ahmadi Graduates (MSc) from NSL and joins UIUC as PhD Student.<br /> <br /> * April 2007: Majid Bagheri Graduates (MSc) from NSL and takes on a job in Iran.<br /> <br /> <br /> <br /> == 2006 == <br /> <br /> * '''November 2006: Osama Saleh Graduates (MSc) from NSL and joins Eyeball Networks in Vancouver. Congratulations Osama -- the first MSc graduate from our group. '''</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4690 2011-11-16T08:01:11Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Nov 16th ===<br /> * Leveraged Amazon Elastic MapReduce cluster to run the proposed method on Wikipedia dataset, and got great reduction of computing time when comparing with that using 5-node cluster.<br /> * Run the proposed method, Parallel Spectral Clustering and Spectral Clustering on Wikipedia datasets. The results show that the proposed method has better performance than the other two methods.<br /> <br /> === Oct 17th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_17th/reportTemplate.pdf here]<br /> * Experimented with the tuning of the parameter &quot;how many terms to keep&quot;<br /> * Run large dataset on the proposed algorithm with good performance (91.38%).<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4689 2011-11-16T08:00:51Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Nov 10th ===<br /> * Leveraged Amazon Elastic MapReduce cluster to run the proposed method on Wikipedia dataset, and got great reduction of computing time when comparing with that using 5-node cluster.<br /> * Run the proposed method, Parallel Spectral Clustering and Spectral Clustering on Wikipedia datasets. The results show that the proposed method has better performance than the other two methods.<br /> <br /> === Oct 17th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_17th/reportTemplate.pdf here]<br /> * Experimented with the tuning of the parameter &quot;how many terms to keep&quot;<br /> * Run large dataset on the proposed algorithm with good performance (91.38%).<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4687 2011-11-16T06:09:26Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Nov 10th ===<br /> * Leveraged Amazon Elastic MapReduce cluster to run the proposed method on Wikipedia dataset, and got great reduction of computing time when comparing with that using 5-node cluster.<br /> * Run the proposed method, distributed Spectral Clustering and Spectral Clustering on Wikipedia datasets. The results show that the proposed method has better performance than the other two methods.<br /> <br /> === Oct 17th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_17th/reportTemplate.pdf here]<br /> * Experimented with the tuning of the parameter &quot;how many terms to keep&quot;<br /> * Run large dataset on the proposed algorithm with good performance (91.38%).<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4669 2011-10-18T08:43:18Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Oct 17th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_17th/reportTemplate.pdf here]<br /> * Experimented with the tuning of the parameter &quot;how many terms to keep&quot;<br /> * Run large dataset on the proposed algorithm with good performance (91.38%).<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4668 2011-10-18T08:42:27Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Oct 17th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_17th/reportTemplate.pdf here]<br /> * Experimented with the tuning of the parameter &quot;how many terms to keep&quot;<br /> * Run large dataset on the proposed algorithm with good performance.<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4646 2011-10-05T05:56:34Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Oct 4th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Oct_4th/reportTemplate.pdf here]<br /> * Done with stop-word removing and stemming.<br /> * In similarity matrix computation part, in order to reduce feature set, used first-F items in one doc's content.<br /> * Spectral clustering accuracy is 96.6%. Kmeans accuracy is 62.22%.<br /> * Will use the proposed algorithm on the dataset.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4625 2011-09-21T03:19:09Z

<p>Feig: </p> <hr /> <div>= Fall 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === Sep 20th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Sep_20th/reportTemplate.pdf here]<br /> * Successfully crawled 3,550,567 web pages from Wikipedia, forming 739,144 categories.<br /> * Processed the raw HTML files and have it ready for test purpose.<br /> * Now working on stop-word removing and stemming.<br /> * In the following 4-5 days, will have the dataset fully processed and ready. <br /> * And then, one day to do tf-idf on the dataset (since that part of the code is done). <br /> * Following that, first test the dataset on kmeans; and then, try spectral clustering on the data.<br /> <br /> = Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4477 2011-06-01T02:31:12Z

<p>Feig: </p> <hr /> <div>= Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset. Will continue the experiment and collect more data.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4476 2011-06-01T02:25:34Z

<p>Feig: </p> <hr /> <div>= Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 30th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_30th/reportTemplate.pdf here]<br /> * Added Nyström method into the comparison of proposed method and PSC method, in the aspect of accuracy.<br /> * Continued to run the experiment on large dataset.<br /> * Analysed the experiment results in the experiment result part.<br /> * Included more sources to show the legitimacy of the evaluation metrics.<br /> * Refined the structure of figures.<br /> <br /> === May 16th ===<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4446 2011-05-17T03:43:15Z

<p>Feig: </p> <hr /> <div>= Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 16th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_17th/reportTemplate.pdf here]<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). Moreover, the memory usage limitation of Matlab will fail the experiment when using 2^12 data set.<br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4445 2011-05-17T03:35:58Z

<p>Feig: </p> <hr /> <div>= Summer 2011 (RA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 16th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_17th/reportTemplate.pdf here]<br /> * Got Parallel SC method (using MPICH2 in implementation) running on NSL cluster and got the result for comparison purpose. The hardware platform remains unchanged (NSL cluster).<br /> * Examined and ran a few experiments with Nystrom Extension Spectral Clustering method (in matlab) and &quot;Fast approximate spectral clustering&quot; method (in R). However, the problem here is that these two method runs on single machine, which will somehow defeat the comparison purpose (because platform is changed). <br /> * Area affected by update: experiment setup and evaluation parts.<br /> <br /> === May 5th ===<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> = Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4408 2011-05-06T01:56:22Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === May 5th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_May_5th/reportTemplate.pdf here]<br /> * Managed to find two recent works (implementation) for comparison purpose, set up the cluster to run the implementation.<br /> * There are also other similar papers on the same topic, which have been added into the related work part.<br /> * Further clarification of the results (as well as the methodology of our proposed method).<br /> <br /> === Apr 19th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_19th.pdf here]<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4399 2011-04-21T04:48:56Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_19th.pdf here]<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> * TA course assignment five marking, final exam supervision and marking<br /> <br /> === Apr 11th ===<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4398 2011-04-21T04:38:17Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_19th.pdf here]<br /> * TA course assignment five marking, final exam supervision and marking<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18) with guaranteed error, more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> <br /> === Apr 11th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_3rd.pdf here]<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4397 2011-04-21T04:36:55Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 19th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_19th.pdf here]<br /> * TA course assignment five marking, final exam supervision and marking<br /> * Applied larger datasets to the proposed scheme, and get HUGE computing time reduction (the ratio of about 1/18), more can be found in the report.<br /> * Read through related publications, compared our method with the already existing ones, and updated the related work part in the paper.<br /> <br /> === Apr 11th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_3rd.pdf here]<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4360 2011-04-12T00:59:40Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 11th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_3rd.pdf here]<br /> * Corrected the implementation of Dunn Index (which is used for evaluation purpose).<br /> * Solved the problem of memory insufficiency in gram matrix generation (for comparison purpose). The problem is that, in order to compare the proposed method to the original method, we need the raw gram matrix computed beforehand. However, for dataset no less than 2^13, there will be memory insufficiency. The problem is solved using a buffer (instead of keeping all the matrix entries in memory)<br /> * The above mentioned fix will enable us to extend the comparison to larger datasets.<br /> <br /> === Apr 3rd ===<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4329 2011-04-04T06:07:31Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 3rd ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_3rd.pdf here]<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4328 2011-04-04T06:07:13Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Apr 3rd ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Apr_3rd.pdf here]<br /> * Implemented the whole proposed design.<br /> * Implemented evaluation measurement.<br /> * Got the result of running dataset with size from 2^6 to 2^13, more results will be collected.<br /> <br /> === Mar 21st ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_21st.pdf here]<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4314 2011-03-22T07:02:13Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 21st ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_21st.pdf here]<br /> * Corrected bugs in the original Mahout code. The original source code has a &quot;java.lang.RuntimeException: java.lang.ClassNotFoundException&quot; bug, this error appears in several places in the code. <br /> * Implementing evaluation measurement.<br /> <br /> === Mar 14th ===<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4313 2011-03-21T23:35:16Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 14th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_14th.pdf here]<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing opportunity, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4280 2011-03-15T01:36:52Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 14th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_14th.pdf here]<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing chance, named as, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4279 2011-03-15T01:36:37Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 14th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_14th.pdf here]<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing chance, named, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4278 2011-03-15T01:35:55Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 14th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_14th.pdf here]<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing chance, named, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we observe that one Spectral Clustering instance can be initiated right after one bin of data is ready, which is <br /> to say, there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4277 2011-03-15T01:35:39Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 14th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_14th.pdf here]<br /> * Set up Mahout (together with Hadoop) environment and run tests<br /> * Implementing the proposed idea on Mahout<br /> * Found out a nice optimizing chance, named, &quot;Time Multiplexing&quot;. The optimization is based on the observation: In common cases, the Spectral Clustering step should execute only after the preprocessing step. However, we <br /> observe that one Spectral Clustering instance can be initiated right after one bin of data is ready, which is <br /> to say, there can be intersection (time multiplexing) between LSH step and Spectral Clustering step.<br /> <br /> === Mar 7th ===<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4249 2011-03-07T06:05:48Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Mar 7th ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Mar_7th.pdf here]<br /> * Trying to add extra code in the library of Mahout to implement the proposed idea.<br /> * The affected classes: AffinityMatrixInputJob (to perform gram matrix calculation), DistributedRowMatrix (to construct the diagonal matrix), DistributedLanczosSolver (to perform eigen-decomposition)<br /> * Merging the previous reports and making necessary changes<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4223 2011-02-28T05:13:36Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 28 ===<br /> * '''Report:''' [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/reports/Report_Feb_28th.pdf here]<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4222 2011-02-28T05:08:10Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4221 2011-02-28T05:07:48Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance<br /> are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4220 2011-02-28T05:07:37Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 28 ===<br /> * Hadoop's two important design features, which have great influence over the experiment's performance<br /> are examined: data placement policy and task scheduling policy.<br /> * It is noted that, for map tasks, the scheduler uses a locality optimization technique. After selecting a job, the scheduler picks the map task in the job with data closest to the slave, on the same node if possible, otherwise on the same rack, or finally on a remote rack. For reduce tasks, the jobtracker just takes the next in the reduce tasks' list and<br /> assign it to the tasktracker.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4206 2011-02-19T00:23:20Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 14 ===<br /> * Formalized the proposed design. Analyzing why a specific family of function are used, the speedup attained, the evaluation measurement.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4163 2011-02-08T05:42:20Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Feb 7 ===<br /> * Studied LSH and explored different LSH families according to the distance measurement they use.<br /> * Based on how LSH methods work, analysed how these LSH methods can be parallelized in distributed environment.<br /> <br /> === Jan 31 ===<br /> * Examined how the main class clustering algorithms can be parallelized, respectively.<br /> * Explained spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, proposed optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4150 2011-02-01T16:56:31Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 31 ===<br /> * Examining how the main class clustering algorithms can be parallelized, respectively.<br /> * Explaining spectral clustering from theoretical aspect, from graph cut viewpoint. <br /> * Based on the understanding of main clustering algorithms, propose optimizing method for spectral clustering to deal with large data set.<br /> * The proposed method makes use of LSH to do pre-precessing.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4148 2011-02-01T05:15:57Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 31 ===<br /> * Based on the understanding of main clustering algorithms, propose optimizing method for spectral clustering to deal with large data set.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=group_meeting&diff=4039 2011-01-21T00:17:03Z

<p>Feig: </p> <hr /> <div>We hold regular meetings for discussion and for every student to update the group on his/her progress. In some of the meetings, graduate students present talks summarizing their research progress so far. <br /> <br /> The meetings are good opportunities for students to practice their presentation skills and to get constructive feedback from the group on their research. The meetings keep the group members informed about different research problems being addressed in the group. They are also very helpful in finding research topics specially for new students. <br /> <br /> Everybody is welcome to attend. Meeting time: Every Tuesday, 10:00 AM -12:00 PM, room SUR 4010. <br /> <br /> <br /> == Spring 2011==<br /> <br /> * 1 Feb: Ahmed Bu-khamsin, Top Ten Computationally-Complex Problems in Oil and Gas Exploration Filed<br /> <br /> * 25 Jan: Naghmeh, 3D Video Copy Detection<br /> <br /> * 18 Jan: Cameron, [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/harvey/talks/Copy%20Detection%20Using%20Optical%20Flow.pptx Video Copy Detection using Optical Flow]<br /> <br /> * 11 Jan: Mathieu, [[media:3D_VideosOverview.pptx | 3D Media - An Overview]]<br /> <br /> <br /> <br /> Group meetings were held biweekly up to Dec 2010. <br /> <br /> == Fall 2010==<br /> <br /> * 21 Dec 10: Group discussion.<br /> <br /> * 9 Dec 10: Mohammad, [[media:Botnet-Detection-2.0.pptx | Detection of SIP Botnets Based on C&amp;C Communication]]<br /> <br /> * 23 Nov 10: Taher, Approximation algorithms for Web-Scale Kernel Methods<br /> <br /> * 19 Oct 10: Jeff, [https://cs-nsl-svn.cs.surrey.sfu.ca/cssvn/nsl-members/gao/talks/LSH_Cluster.pdf Gram Matrix Approximation Using Locality Sensitive Hashing on Cluster]<br /> <br /> * 7 Oct 10: Dr. Rocky Chang (Hong Kong Polytechnic University),&lt;br/&gt; [[media:Rocky-SFU-7-Oct-2010.pdf | Active Measurement of Data-Path Quality in a Non-cooperative Internet]]<br /> <br /> * 28 Sep 10: Ahmed, [[media:DRS.pdf | Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management]]<br /> <br /> <br /> == Spring/Summer 2010==<br /> <br /> * 31 Aug 10: Hamed, [[media:Predicting_Click_Through_Rate_for_new_ads.pdf | Predicting Click Through Rate for New Ads with Semantically Similarity Measurement]]<br /> <br /> * 9 Aug 10: Yuanbin, [[media:mutisendP2Pstream.pdf | Efficient Algorithms for Multi-Sender Data Transmission in Swarm-based P2P Streaming Systems]]<br /> <br /> * 3 Aug 10: Azin, [[media:CognitiveRadio.ppt | Cognitive Radio Networks]]<br /> <br /> * 20 July 10: Farid, [[media:movid10.pdf | Optimal Scalable Video Multiplexing in Mobile Broadcast Networks]]<br /> <br /> * 17 May 10: Cameron, Reducing Energy Consumption in Online Network Games on Mobile Devices<br /> <br /> * 10 May 10: Cong, Latency Reduction in Online Network Games<br /> <br /> * 19 April 10: Shabnam, [[media:Svc-nc.ppt | Live P2P Streaming with Scalable Video Coding and Network Coding]]<br /> <br /> * 29 March 10: Jeff and Taher, [[media:AppAlgo.ppt | Approximation algorithms for Kernel Methods on Multi-core CPUs and GPUs]]<br /> <br /> * 15 March 10: Som, [http://www.cs.sfu.ca/~ssa121/personal/wimaxSVC.pdf Video Streaming over WiMAX]<br /> <br /> * 1 March 10: Farid, [[media:scalable_video_streaming_for_mobiletv.pptx | Scalable Video Streaming for MobileTV]]<br /> <br /> * 1 Feb 10: Ahmed, Design of pCDN with Scalable Video Coding<br /> <br /> * 18 Jan 10: Shabnam, P2P Streaming with Newtork Coding and Scalable Video Coding<br /> <br /> <br /> == Fall 2009==<br /> <br /> * 8 December 09: Yi, Video Streaming over Cooperative Wireless Networks<br /> <br /> * 10 Nov 09: Cheng, [[media:testbed.ppt | Design of a Mobile TV Testbed]]<br /> <br /> * 27 October 09: Yuanbin, Segment Scheduling in P2P Streaming Systems<br /> <br /> * 13 October 09: Ahmed, [[media:LTE.pdf | Long Term Evolution (LTE) - A Tutorial]]<br /> <br /> * 6 October 09: Cheng, [[media:Mm09.ppt | Statistical Multiplexing of VBR Video Streams]] (ACM MM 09 talk)<br /> <br /> * 22 September 09: Som, Video Streaming over WiMAX Networks<br /> <br /> * 8 September 09: Cong, Minimizing Round-Trip Time in Online Games<br /> <br /> <br /> == Summer 2009==<br /> <br /> * 18 August 09: Mohammad and Cong: 30 min each. Present their Directed Reading projects. <br /> <br /> * 14 July 09: Cheng, [[media:wimaxTV.pptx | Broadcasting Variable-Bit-Rate Videos in 802.16e-Like Mobile Networks]] <br /> <br /> * 7 July 09: Yi<br /> <br /> * 26 June 09: Ahmed <br /> <br /> * 5 June 09: '''Canceled''' (Mohamed attending NOSSDAV'09)<br /> <br /> * 29 May 09: Kianoosh, End-to-End Secure Delivery of Scalable Video Streams <br /> <br /> * 22 May 09: Cong, [[media:wimax.pptx| Multimedia Streaming over WiMAX Networks]]<br /> <br /> * 8 May 09: Kianoosh, Analysis of Authentication Schemes for Nonscalable Video Streams<br /> <br /> <br /> == Spring 2009 ==<br /> <br /> * 17 Apr 09: [[media:infocom09.pptx|Cheng (practice your infocom presentation)]]<br /> <br /> * 27 March 09: Andreas Berger, [[media:Nsl_vancouver.odp | Network-based Detection of SIP Bots]]<br /> <br /> * 27 Feb 09: Shabnam and Yuanbin<br /> <br /> * 23 Jan 09: Cheng (rehearse your PhD proposal) and Kianoosh</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=4027 2011-01-20T05:38:28Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 17 ===<br /> * Understanding Spectral clustering and distributed implementation.<br /> * Mahout experimenting.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=3975 2011-01-11T04:02:09Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout experimenting.<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=3974 2011-01-11T03:56:35Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout code reading (clustering part)<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=3973 2011-01-11T03:56:15Z

<p>Feig: </p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout code reading (clustering part)<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=Private:progress-gao&diff=3972 2011-01-11T03:55:32Z

<p>Feig: New page: = Spring 2011 (TA) = * Courses: **None. === Jan 10 === * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms. * Mahout code reading (clustering ...</p> <hr /> <div>= Spring 2011 (TA) =<br /> * Courses:<br /> **None.<br /> <br /> === Jan 10 ===<br /> * Survey on main clustering algorithms and the distributed map-reduce method of these algorithms.<br /> * Mahout code reading (clustering part)<br /> <br /> == Jan 3 == <br /> * having fun, not really working<br /> <br /> <br /> = Fall 2010 (FELLOWSHIP) = <br /> * Courses:<br /> **CMPT 771: Internet Architecture and Protcols<br /> **CMPT 741: Data Mining<br /> <br /> * Worked on efficient approximation of gram matrix using map-reduce framework, focusing on LSH performance evaluation and network communication measurement.<br /> <br /> <br /> = Summer 2010 (RA) = <br /> * Courses:<br /> **None<br /> <br /> * Worked on Approximation of gram matrices using Locality Sensitive Hashing on Cluster.<br /> <br /> = Spring 2010 (TA+RA) = <br /> * Courses:<br /> **CMPT 886: Special Topics in Operating Systems and Computer Architecture<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.<br /> <br /> = Fall 2009 (TA) = <br /> * Courses:<br /> **CMPT 705: Design and Analysis of Algorithms<br /> **CMPT 726: Machine Learning<br /> <br /> * Worked on Band approximation of gram matrices (large high-dimensional dataset) using Hilbert curve on multicore.</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=group_meeting&diff=3859 2010-10-19T23:46:54Z

<p>Feig: </p> <hr /> <div>We hold regular meetings (mostly bi-weekly) for discussion. In each meeting, a graduate student will present his/her progress on research. This is followed by 15-20 minute discussion. The presenter can also choose a recent paper and present it to the group. The paper must be from the top conferences/journals in our research areas, such as, ACM Multimedia, SIGCOMM, INFOCOM, ICNP, IEEE Transactions on Networking, ACM TOMCCAP, and IEEE Transactions on Multimedia. <br /> <br /> The meetings are good opportunities for students to practice their presentation skills and to get constructive feedback from the group on their research. The meetings keep the group members informed about different research problems being addressed in the group. They are also very helpful in finding research topics specially for new students. <br /> <br /> Everybody is welcome to attend. Meeting time: Every other Tuesday, 11:00 AM -12:00 PM, room SUR 4040. <br /> <br /> <br /> == Fall 2010==<br /> <br /> * 19 Oct 10: Jeff, Gram Matrix Approximation Using Locality Sensitive Hashing on Cluster<br /> <br /> * 7 Oct 10: Dr. Rocky Chang (Hong Kong Polytechnic University),&lt;br/&gt; [[media:Rocky-SFU-7-Oct-2010.pdf | Active Measurement of Data-Path Quality in a Non-cooperative Internet]]<br /> <br /> * 28 Sep 10: Ahmed, [[media:DRS.pdf | Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management]]<br /> <br /> <br /> == Spring/Summer 2010==<br /> <br /> * 31 Aug 10: Hamed, [[media:Predicting_Click_Through_Rate_for_new_ads.pdf | Predicting Click Through Rate for New Ads with Semantically Similarity Measurement]]<br /> <br /> * 9 Aug 10: Yuanbin, [[media:mutisendP2Pstream.pdf | Efficient Algorithms for Multi-Sender Data Transmission in Swarm-based P2P Streaming Systems]]<br /> <br /> * 3 Aug 10: Azin, [[media:CognitiveRadio.ppt | Cognitive Radio Networks]]<br /> <br /> * 20 July 10: Farid, Optimal Scalable Video Multiplexing in Mobile Broadcast Networks<br /> <br /> * 17 May 10: Cameron, Reducing Energy Consumption in Online Network Games on Mobile Devices<br /> <br /> * 10 May 10: Cong, Latency Reduction in Online Network Games<br /> <br /> * 19 April 10: Shabnam, [[media:Svc-nc.ppt | Live P2P Streaming with Scalable Video Coding and Network Coding]]<br /> <br /> * 29 March 10: Jeff and Taher, [[media:AppAlgo.ppt | Approximation algorithms for Kernel Methods on Multi-core CPUs and GPUs]]<br /> <br /> * 15 March 10: Som, [http://www.cs.sfu.ca/~ssa121/personal/wimaxSVC.pdf Video Streaming over WiMAX]<br /> <br /> * 1 March 10: Farid, Mobile Video Streaming <br /> <br /> * 1 Feb 10: Ahmed, Design of pCDN with Scalable Video Coding<br /> <br /> * 18 Jan 10: Shabnam, P2P Streaming with Newtork Coding and Scalable Video Coding<br /> <br /> <br /> == Fall 2009==<br /> <br /> * 8 December 09: Yi, Video Streaming over Cooperative Wireless Networks<br /> <br /> * 10 Nov 09: Cheng, [[media:testbed.ppt | Design of a Mobile TV Testbed]]<br /> <br /> * 27 October 09: Yuanbin, Segment Scheduling in P2P Streaming Systems<br /> <br /> * 13 October 09: Ahmed, [[media:LTE.pdf | Long Term Evolution (LTE) - A Tutorial]]<br /> <br /> * 6 October 09: Cheng, [[media:Mm09.ppt | Statistical Multiplexing of VBR Video Streams]] (ACM MM 09 talk)<br /> <br /> * 22 September 09: Som, Video Streaming over WiMAX Networks<br /> <br /> * 8 September 09: Cong, Minimizing Round-Trip Time in Online Games<br /> <br /> <br /> == Summer 2009==<br /> <br /> * 18 August 09: Mohammad and Cong: 30 min each. Present their Directed Reading projects. <br /> <br /> * 14 July 09: Cheng, [[media:wimaxTV.pptx | Broadcasting Variable-Bit-Rate Videos in 802.16e-Like Mobile Networks]] <br /> <br /> * 7 July 09: Yi<br /> <br /> * 26 June 09: Ahmed <br /> <br /> * 5 June 09: '''Canceled''' (Mohamed attending NOSSDAV'09)<br /> <br /> * 29 May 09: Kianoosh, End-to-End Secure Delivery of Scalable Video Streams <br /> <br /> * 22 May 09: Cong, [[media:wimax.pptx| Multimedia Streaming over WiMAX Networks]]<br /> <br /> * 8 May 09: Kianoosh, Analysis of Authentication Schemes for Nonscalable Video Streams<br /> <br /> <br /> == Spring 2009 ==<br /> <br /> * 17 Apr 09: [[media:infocom09.pptx|Cheng (practice your infocom presentation)]]<br /> <br /> * 27 March 09: Andreas Berger, [[media:Nsl_vancouver.odp | Network-based Detection of SIP Bots]]<br /> <br /> * 27 Feb 09: Shabnam and Yuanbin<br /> <br /> * 23 Jan 09: Cheng (rehearse your PhD proposal) and Kianoosh</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=group_meeting&diff=3858 2010-10-19T19:35:31Z

<p>Feig: </p> <hr /> <div>We hold regular meetings (mostly bi-weekly) for discussion. In each meeting, a graduate student will present his/her progress on research. This is followed by 15-20 minute discussion. The presenter can also choose a recent paper and present it to the group. The paper must be from the top conferences/journals in our research areas, such as, ACM Multimedia, SIGCOMM, INFOCOM, ICNP, IEEE Transactions on Networking, ACM TOMCCAP, and IEEE Transactions on Multimedia. <br /> <br /> The meetings are good opportunities for students to practice their presentation skills and to get constructive feedback from the group on their research. The meetings keep the group members informed about different research problems being addressed in the group. They are also very helpful in finding research topics specially for new students. <br /> <br /> Everybody is welcome to attend. Meeting time: Every other Tuesday, 11:00 AM -12:00 PM, room SUR 4040. <br /> <br /> <br /> == Fall 2010==<br /> <br /> * 19 Oct 10: Jeff, [[media:LSH_Cluster.pdf | Gram Matrix Approximation Using Locality Sensitive Hashing on Cluster]]<br /> <br /> * 7 Oct 10: Dr. Rocky Chang (Hong Kong Polytechnic University),&lt;br/&gt; [[media:Rocky-SFU-7-Oct-2010.pdf | Active Measurement of Data-Path Quality in a Non-cooperative Internet]]<br /> <br /> * 28 Sep 10: Ahmed, [[media:DRS.pdf | Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management]]<br /> <br /> <br /> == Spring/Summer 2010==<br /> <br /> * 31 Aug 10: Hamed, [[media:Predicting_Click_Through_Rate_for_new_ads.pdf | Predicting Click Through Rate for New Ads with Semantically Similarity Measurement]]<br /> <br /> * 9 Aug 10: Yuanbin, [[media:mutisendP2Pstream.pdf | Efficient Algorithms for Multi-Sender Data Transmission in Swarm-based P2P Streaming Systems]]<br /> <br /> * 3 Aug 10: Azin, [[media:CognitiveRadio.ppt | Cognitive Radio Networks]]<br /> <br /> * 20 July 10: Farid, Optimal Scalable Video Multiplexing in Mobile Broadcast Networks<br /> <br /> * 17 May 10: Cameron, Reducing Energy Consumption in Online Network Games on Mobile Devices<br /> <br /> * 10 May 10: Cong, Latency Reduction in Online Network Games<br /> <br /> * 19 April 10: Shabnam, [[media:Svc-nc.ppt | Live P2P Streaming with Scalable Video Coding and Network Coding]]<br /> <br /> * 29 March 10: Jeff and Taher, [[media:AppAlgo.ppt | Approximation algorithms for Kernel Methods on Multi-core CPUs and GPUs]]<br /> <br /> * 15 March 10: Som, [http://www.cs.sfu.ca/~ssa121/personal/wimaxSVC.pdf Video Streaming over WiMAX]<br /> <br /> * 1 March 10: Farid, Mobile Video Streaming <br /> <br /> * 1 Feb 10: Ahmed, Design of pCDN with Scalable Video Coding<br /> <br /> * 18 Jan 10: Shabnam, P2P Streaming with Newtork Coding and Scalable Video Coding<br /> <br /> <br /> == Fall 2009==<br /> <br /> * 8 December 09: Yi, Video Streaming over Cooperative Wireless Networks<br /> <br /> * 10 Nov 09: Cheng, [[media:testbed.ppt | Design of a Mobile TV Testbed]]<br /> <br /> * 27 October 09: Yuanbin, Segment Scheduling in P2P Streaming Systems<br /> <br /> * 13 October 09: Ahmed, [[media:LTE.pdf | Long Term Evolution (LTE) - A Tutorial]]<br /> <br /> * 6 October 09: Cheng, [[media:Mm09.ppt | Statistical Multiplexing of VBR Video Streams]] (ACM MM 09 talk)<br /> <br /> * 22 September 09: Som, Video Streaming over WiMAX Networks<br /> <br /> * 8 September 09: Cong, Minimizing Round-Trip Time in Online Games<br /> <br /> <br /> == Summer 2009==<br /> <br /> * 18 August 09: Mohammad and Cong: 30 min each. Present their Directed Reading projects. <br /> <br /> * 14 July 09: Cheng, [[media:wimaxTV.pptx | Broadcasting Variable-Bit-Rate Videos in 802.16e-Like Mobile Networks]] <br /> <br /> * 7 July 09: Yi<br /> <br /> * 26 June 09: Ahmed <br /> <br /> * 5 June 09: '''Canceled''' (Mohamed attending NOSSDAV'09)<br /> <br /> * 29 May 09: Kianoosh, End-to-End Secure Delivery of Scalable Video Streams <br /> <br /> * 22 May 09: Cong, [[media:wimax.pptx| Multimedia Streaming over WiMAX Networks]]<br /> <br /> * 8 May 09: Kianoosh, Analysis of Authentication Schemes for Nonscalable Video Streams<br /> <br /> <br /> == Spring 2009 ==<br /> <br /> * 17 Apr 09: [[media:infocom09.pptx|Cheng (practice your infocom presentation)]]<br /> <br /> * 27 March 09: Andreas Berger, [[media:Nsl_vancouver.odp | Network-based Detection of SIP Bots]]<br /> <br /> * 27 Feb 09: Shabnam and Yuanbin<br /> <br /> * 23 Jan 09: Cheng (rehearse your PhD proposal) and Kianoosh</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=File:LSH_Cluster.pdf&diff=3857 2010-10-19T19:33:41Z

<p>Feig: Gram Matrix Approximation�Using Locality Sensitive Hashing�on Cluster</p> <hr /> <div>Gram Matrix Approximation�Using Locality Sensitive Hashing�on Cluster</div>

Feig https://nmsl.cs.sfu.ca/index.php?title=group_meeting&diff=3856 2010-10-19T17:15:25Z

<p>Feig: </p> <hr /> <div>We hold regular meetings (mostly bi-weekly) for discussion. In each meeting, a graduate student will present his/her progress on research. This is followed by 15-20 minute discussion. The presenter can also choose a recent paper and present it to the group. The paper must be from the top conferences/journals in our research areas, such as, ACM Multimedia, SIGCOMM, INFOCOM, ICNP, IEEE Transactions on Networking, ACM TOMCCAP, and IEEE Transactions on Multimedia. <br /> <br /> The meetings are good opportunities for students to practice their presentation skills and to get constructive feedback from the group on their research. The meetings keep the group members informed about different research problems being addressed in the group. They are also very helpful in finding research topics specially for new students. <br /> <br /> Everybody is welcome to attend. Meeting time: Every other Tuesday, 11:00 AM -12:00 PM, room SUR 4040. <br /> <br /> <br /> == Fall 2010==<br /> <br /> * 19 Oct 10: Jeff, Gram Matrix Approximation Using Locality Sensitive Hashing on Cluster<br /> <br /> * 7 Oct 10: Dr. Rocky Chang (Hong Kong Polytechnic University),&lt;br/&gt; [[media:Rocky-SFU-7-Oct-2010.pdf | Active Measurement of Data-Path Quality in a Non-cooperative Internet]]<br /> <br /> * 28 Sep 10: Ahmed, [[media:DRS.pdf | Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management]]<br /> <br /> <br /> == Spring/Summer 2010==<br /> <br /> * 31 Aug 10: Hamed, [[media:Predicting_Click_Through_Rate_for_new_ads.pdf | Predicting Click Through Rate for New Ads with Semantically Similarity Measurement]]<br /> <br /> * 9 Aug 10: Yuanbin, [[media:mutisendP2Pstream.pdf | Efficient Algorithms for Multi-Sender Data Transmission in Swarm-based P2P Streaming Systems]]<br /> <br /> * 3 Aug 10: Azin, [[media:CognitiveRadio.ppt | Cognitive Radio Networks]]<br /> <br /> * 20 July 10: Farid, Optimal Scalable Video Multiplexing in Mobile Broadcast Networks<br /> <br /> * 17 May 10: Cameron, Reducing Energy Consumption in Online Network Games on Mobile Devices<br /> <br /> * 10 May 10: Cong, Latency Reduction in Online Network Games<br /> <br /> * 19 April 10: Shabnam, [[media:Svc-nc.ppt | Live P2P Streaming with Scalable Video Coding and Network Coding]]<br /> <br /> * 29 March 10: Jeff and Taher, [[media:AppAlgo.ppt | Approximation algorithms for Kernel Methods on Multi-core CPUs and GPUs]]<br /> <br /> * 15 March 10: Som, [http://www.cs.sfu.ca/~ssa121/personal/wimaxSVC.pdf Video Streaming over WiMAX]<br /> <br /> * 1 March 10: Farid, Mobile Video Streaming <br /> <br /> * 1 Feb 10: Ahmed, Design of pCDN with Scalable Video Coding<br /> <br /> * 18 Jan 10: Shabnam, P2P Streaming with Newtork Coding and Scalable Video Coding<br /> <br /> <br /> == Fall 2009==<br /> <br /> * 8 December 09: Yi, Video Streaming over Cooperative Wireless Networks<br /> <br /> * 10 Nov 09: Cheng, [[media:testbed.ppt | Design of a Mobile TV Testbed]]<br /> <br /> * 27 October 09: Yuanbin, Segment Scheduling in P2P Streaming Systems<br /> <br /> * 13 October 09: Ahmed, [[media:LTE.pdf | Long Term Evolution (LTE) - A Tutorial]]<br /> <br /> * 6 October 09: Cheng, [[media:Mm09.ppt | Statistical Multiplexing of VBR Video Streams]] (ACM MM 09 talk)<br /> <br /> * 22 September 09: Som, Video Streaming over WiMAX Networks<br /> <br /> * 8 September 09: Cong, Minimizing Round-Trip Time in Online Games<br /> <br /> <br /> == Summer 2009==<br /> <br /> * 18 August 09: Mohammad and Cong: 30 min each. Present their Directed Reading projects. <br /> <br /> * 14 July 09: Cheng, [[media:wimaxTV.pptx | Broadcasting Variable-Bit-Rate Videos in 802.16e-Like Mobile Networks]] <br /> <br /> * 7 July 09: Yi<br /> <br /> * 26 June 09: Ahmed <br /> <br /> * 5 June 09: '''Canceled''' (Mohamed attending NOSSDAV'09)<br /> <br /> * 29 May 09: Kianoosh, End-to-End Secure Delivery of Scalable Video Streams <br /> <br /> * 22 May 09: Cong, [[media:wimax.pptx| Multimedia Streaming over WiMAX Networks]]<br /> <br /> * 8 May 09: Kianoosh, Analysis of Authentication Schemes for Nonscalable Video Streams<br /> <br /> <br /> == Spring 2009 ==<br /> <br /> * 17 Apr 09: [[media:infocom09.pptx|Cheng (practice your infocom presentation)]]<br /> <br /> * 27 March 09: Andreas Berger, [[media:Nsl_vancouver.odp | Network-based Detection of SIP Bots]]<br /> <br /> * 27 Feb 09: Shabnam and Yuanbin<br /> <br /> * 23 Jan 09: Cheng (rehearse your PhD proposal) and Kianoosh</div>

Feig