Difference between revisions of "Private:progress-hegazy"

Revision as of 19:08, 12 October 2012

Accomplished (this week)
- Ahmed prepared his seminar presentation for next week.
- I had a meeting with him to give him my feedback/suggestions for the presentation.
- Ahmed reviewed the IPDPS paper and he discussed with me some modifications that need to be made.
Issues
- None
Next Steps
- Ahmed is waiting for his meeting with Dr. Hefeeda this week to discuss next steps towards defense.

Accomplished (this week)
- Abdullah obtained a real I/O log from an ocean simulation application.
- We ran the partial autocorrelation function on it. It shows tendency towards a second-order Markov model.
- We discucssed a scaled down version of the testbed, where he would have HDD and ramdisk storage. Using the testbed, we have to show some improvement if we place the most frequently accessed files in the ramdisk.
Issues
- Abdullah needs a more powerful machine to serve as the scaled-down version of the testbed; his current machine is slow and low on memory.
Next Steps
- Abdullah is to set up his testbed and run some experiments.

Accomplished (this week)
- I finalized the delay experiments and updated/sent the working document.
- I have phone-interviewed the new candidate and sent comments/recommendations to Dr. Hefeeda.
- Ahmed worked on the dual redundancy with "hot swap" experiment.
- I have been doing some research on real Modbus-based processes or how to build one, but I haven't concluded this research yet.
Issues
- Lack of clear visioin for the future of this work.
- None
Next Steps
- Ahmed is to finish the fault-tolerance experiment
- I will continue my research on real processes.

Accomplished (this week)
- Ahmed came up with results on application-level testing using Apache-wget.
- I have finished up writing the IPDPS paper and sent it for Dr. Hefeeda to review.
- I have addressed at least 80% of Dr. Hefeeda's comments and submitted the paper to IPDPS.
Issues
- None
Next Steps
- Ahmed is working on updating his report and writing his thesis.
- I have some leftover comments that I will need to address in the paper.

Accomplished (this week)
- Abdullah came up with I/O logs from the benchmark.
- During a one-on-one, I ran partial auto-correlation function (PACF) on the log, and the result is a first-order Markov Chain.
Issues
- None
Next Steps
- Abdullah will come up with logs with the same format from HPC applications.
- We will start working on defining our state space, action space, and reward/penalty function.

Accomplished (this week)
- Ahmed successfully ran the delay experiments and obtained results.
- I asked for additional experiments for a different scan cycle, which Ahmed ran on the spot!
- The results show great robustness for up to 6.4 seconds of round-trip delay.
Issues
- None
Next Steps
- Ahmed will run an experiment for dual redundancy.
- I am about to decide on the delay estimator to feed Smith Predictor (for variable delay cases).

Accomplished (this week)
- I registered the paper abstract in IPDPS.
- I ported the paper to LaTeX and worked on the abstract, introduction, related work and problem definition.
- Ahmed's code now works with most TCP clients and servers.
Issues
- We are running the experiments now (Friday night) and time is tight for the IPDPS submission.
Next Steps
- We will finish as much experiments as we can Friday and Saturday.
- I should have the paper (almost) ready Saturday night.

Accomplished (this week)
- I had a couple of meetings with Abdullah; he came up with some file access logs from a benchmark
- We ran the partial autocorrelation function on it to get the Markov order.
- I up with a rough state definition based on the requested blocks (or rather "chunks") and storage status. I also came up with a rough definition for the reward/penalty function based on the same info.
Issues
- Progress is a bit slow, but not too bad.
Next Steps
- Abdullah is to get more logs from other real HPC applications
- I will continue to define the Markov Decision Process.

Accomplished (this week)
- I had a couple of meetings with Ahmed where I explained to him in more details what experiments to run.
- Ahmed prepared the testbed based on new cloud instances. He is very close to getting the results.
- I had a Skype call with Ali to explain to him almost the same thing as Ahmed, though Ali is a little behind Ahmed.
Issues
- Progress is a bit too slow; we could have easily obtained results this week, but we didn't.
Next Steps
- Ahmed should come up with the results soon.
- More experiments (this time, on random delay) are to be conducted.
- Ali should be able to run the same experiments soon (just to get familiar; nothing new to accomplish).

Accomplished (this week)
- Ahmed bought and successfully installed the Infiniband cards.
- Benchmark experiments were run for Infiniband.
Issues
- Ahmed's implementation of SDP doesn't support asynchronous calls.
- Ahmed couldn't run PCIe and Infiniband experiments from the same machine (library conflict between his SDP and Infiniband SDP)
- Time is tight for the IPDPS submission.
Next Steps
- Ahmed will implement asynchronous call support in order to run his application-based evaluation and compare it to Infiniband.
- I will register the paper's abstract to IPDPS over the weekend (deadline is Monday, Sept 24th).
- Although the time is tight, Ahmed will try to finish his asynchronous calls support and the application-based evaluation by the end of next week. This gives us a couple of days to polish up the paper.

Accomplished (this week)
- Abdullah tried to obtain HPC I/O logs off of the internet. When that was not possible, he tried a couple of benchmarks to to generate some I/O logs as a placeholder until we obtain logs from real HPC applications.
- During our meeting, we tried to analyze the data, e.g., to verify the Markov chain order using partial autocorrelation functions. I noticed the data was so fine-grained (time in nanoseconds and block size in KB).
Issues
- None
Next Steps
- Abdullah to work on the coarser representation of the log (e.g., 1 second and MBs).
- I will go into one more level of details for the general methodology.

Accomplished (this week)
- I have come up with the delay compensation results and updated the working document.
- I have met with Ahmed AbdelSadek and explained the problem and experiments to him.
- Ali finished his first experiment, which is nothing other than making sure Modbus is working fine for him on LabVIEW.
Issues
- Ahmed was supposed to finish his first task by the end of the week, but he was "discussing some issues with Dr. Mohamed regarding [his] research".
- Communication rate with Ali is a bit slow.
Next Steps
- I may have to work on the delay experiments myself until Ahmed is up to speed.
- Ahmed will try to accomplish the tasks rolled over from this week.
- I will have conference call with Ali to explain to him the next steps.

@@ Line 23: / Line 23: @@
 * '''Accomplished''' (this week)
 ** I finalized the delay experiments and updated/sent the working document.
+** I have phone-interviewed the new candidate and sent comments/recommendations to Dr. Hefeeda.
 ** Ahmed worked on the dual redundancy with "hot swap" experiment.
 ** I have been doing some research on real Modbus-based processes or how to build one, but I haven't concluded this research yet.