Difference between revisions of "Private:progress-hegazy"

From NMSL
 
(51 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
==Mar 2013 to Apr 2013==
 +
===Industrial Automation as a Service===
 +
* Overall progress: 70%
 +
* Physical model building: 100% (Note: I have to fix a broken gearbox of solar collector and bidirectional pump leak)
 +
* Wiring: 95% (Remaining 5%: wiring missing power circuits when done)
 +
* Individual component testing: 100% (Note: I have to re-test the motor with gearbox and the leaking pump when fixed)
 +
* Complete system open-loop test: 100% (Note: I have to re-test the solar positioning process after fixing the gearbox)
 +
* Complete system feedback test: 60% (Remaining 40%: temperature control loops)
 +
* Power circuit design and breadboard testing: 90% (Remaining 10%: tuning the H-bridge circuit for the bidirectional pump and solid state switches for heaters)
 +
* Power circuit soldering on printed circuit board: 40% (Remaining 60%: soldering 1 more H-bridge circuit, 5 Darlington pair power amplifiers, 2 solid-state switches)
 +
* LabVIEW controller implementation and tuning: 40% (Remaining 60%: controllers for the temperature control loops and adding FT components)
 +
 +
 +
Below are the tasks I performed to achieve the progress above. Please note that some of the items were very time consuming due to the nature of the physical labor involved.
 +
* Made about 10 trips to electronics shops in Vancouver, Home Depot, Canadian Tire, and Princess Auto to purchase some tools, and various electronic/electrical components.
 +
* Reorganized the installation of some physical components for better demonstration. For example, I had to craft "spacers" to flip the flowmeters upside down because the bottom of them shows a rotating propeller whose speed indicated the flow. The original installation just shows the solid head of the meter, nothing visual.
 +
* Installed a small heater to emulate solar heat.
 +
* Installed an additional pump for the "steam" cycle; one pump was too weak for the cycle.
 +
* Re-wired the whole system a few times to try to optimize wiring.
 +
* Refined/tested the power circuits for the actuators.
 +
* Soldered them on printed circuit boards. I had to de-solder and solder a couple of times. (very time consuming).
 +
* Crafted (manually) heat sinks for the power transistors; I couldn't find commercially available ones for my design.
 +
* Solved (partially) an issue with the fault tolerance algorithm that has to do with the limited registers in the I/O interface.
 +
 +
 +
==Dec 2012 to Mar 2013==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this period)
 +
** Ahmed addressed comments and defended his thesis.
 +
** Another submission has been made and rejected.
 +
* '''Issues'''
 +
** If work not submitted to another conference/workshop, it will likely never get published.
 +
* '''Next Steps'''
 +
**N/A
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this period)
 +
** Abdullah got a simplified version of the DP algorithm to work.
 +
* '''Issues'''
 +
** Progress is very slow.
 +
* '''Next Steps'''
 +
** Abdullah is currently working on an expanded state space.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this period)
 +
** I performed more emulation experiments.
 +
** I added 3 theorems to the formal arguments.
 +
** We finalized and submitted the SIGCOMM paper.
 +
** I prepared the forms of the architecture and smooth handover patents.
 +
** I allocated a few days to provide the necessary material for the patents.
 +
** As for the physical model, I have wired and configured the I/O interface for some control loops.
 +
** I have faced and resolved a number of issues getting individual loops to work. Without going into details, the most important issues are:
 +
*** Designing power circuits for the pumps
 +
*** Configuring the Modbus port forwarding to work for Dynamic IP connections.
 +
** I got the cloud controllers to control individual control loops (flow and temperature).
 +
** I am also able to get the fault tolerance algorithm to work.
 +
** To get the delay compensator to work, the main issue that is left for me is to find a solution/workaround for the blocking TCP sockets of LabVIEW. Worse come to worst, I may have to write the protocol myself (something I am trying to avoid so far).
 +
* ''Issues''
 +
** None
 +
* '''Next Steps'''
 +
** I will need to come up with a non-blocking implementation of Modbus/TCP protocol.
 +
 +
==Oct 29-Nov 23==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this period)
 +
** Ahmed has updated his thesis multiple times and performed more experiments, especially on CPU utilization.
 +
** I have reviewed Ahmed's thesis multiple times and gave him my feedback.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed will continue improving his thesis.
 +
** Ahmed will defend in 2 weeks.
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this period)
 +
** Abdullah has switched to a different HPC application and evaluated its model order.
 +
** I put together a plan to traget HPDC.
 +
** Abdullah has performed the literature review and delivered a write-up.
 +
* '''Issues'''
 +
** We cannot explain an anomaly whereby SSD is performing slightly worse than a regular HDD. This happened for 2 HPC applications, even when we artificially made them more I/O intensive. This is really holding us back.
 +
** Time is tight for HPDC.
 +
* '''Next Steps'''
 +
** Abdullah is building a fist order Markov Model to use it in our stochastic dynamic programming.
 +
** I will start writing the introduction and related work section.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this period)
 +
** I have been working on experimentation and writing of the "Automation as a Service" paper.
 +
** I have finished the proposed work section according to our last discussion.
 +
** I held several meetings and phone calls with the controls company and we decided on the physical process and I received an unofficial quotation.
 +
* '''Issues'''
 +
** Paper submission is delayed.
 +
* '''Next Steps'''
 +
** I will continue working on finalizing the paper.
 +
** I should be receiving an official quotation from the controls company soon. We should be making the order shortly.
 +
 +
 +
==Oct 22-26==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this week)
 +
** Ahmed is working on writing his thesis and some application-level experimentation.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed will continue the same work.
 +
** I will review any work as soon as it is accomplished
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this week)
 +
** I reviewed Abdullah's report and sent him my comments
 +
** Abdullah used Kiana's machine temporarily, and managed to understand why the code wasn't utilizing all the cores. We will eventually need a machine with high specs, but I would rather wait until he's done experimenting with what he has. Then, we will have a better idea about the most cost-effective specs for his new machine.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Run the example HPC application on SSD and examine the improvement in performance (not accomplished from last week)
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this week)
 +
** I met with a person who owns a workshop who is going to manufacturer the physical process. Quotation is due in about a week.
 +
** I am working on the SIGMETRICS paper.
 +
** Ahmed finished the fault tolerance experiments.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** I will continue working on the SIGMETRICS paper.
 +
 +
 +
==Oct 15-19==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this week)
 +
** Ahmed presented his work at the seminar.
 +
** I had a brief meeting with Ahmed where I explained to him the need for more experiments.
 +
** Ahmed started on addressing Dr. Hefeeda's comments.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed will continue working on improving the IPDPS paper.
 +
** Ahmed will start on getting his SDP implementation to work with DB2 application.
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this week)
 +
** Abdullah has written up a report, which I am currently reviewing.
 +
** Abdullah tried to get the test HPC application to work on other faster machines but he couldn't.
 +
* '''Issues'''
 +
** Abdullah needs a fast Linux machine because we believe that the problem size of the application cannot be increased due to resource limitation.
 +
* '''Next Steps'''
 +
** Run the example HPC application on SSD and examine the improvement in performance.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this week)
 +
** I have researched process control demo systems.
 +
** I met a representative from Davis Controls where he presented to me a number of Modbus solutions and said he would supply the necessary sensors/actuators and the physical process.
 +
** I dug deeper into the material on control theory for computing systems, and I have some comments and suggestions, which I discussed over the conference call.
 +
* '''Issues'''
 +
** I am still looking for a candidate who could help me on this project with both control and computing background.
 +
* '''Next Steps'''
 +
** I will wrap up the work done so far as we discussed and hopefully make it ready for publication.
 +
** I will meet with the workshop engineer who will manufacture the physical industrial process.
 +
 +
 +
==Oct 8-12==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this week)
 +
** Ahmed prepared his seminar presentation for next week.
 +
** I had a meeting with him to give him my feedback/suggestions for the presentation.
 +
** Ahmed reviewed the IPDPS paper and he discussed with me some modifications that need to be made.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed is waiting for his meeting with Dr. Hefeeda this week to discuss next steps towards defense.
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this week)
 +
** Abdullah obtained a real I/O log from an ocean simulation application.
 +
** We ran the partial autocorrelation function on it. It shows tendency towards a second-order Markov model.
 +
** We discucssed a scaled down version of the testbed, where he would have HDD and ramdisk storage. Using the testbed, we have to show some improvement if we place the most frequently accessed files in the ramdisk.
 +
* '''Issues'''
 +
** Abdullah needs a more powerful machine to serve as the scaled-down version of the testbed; his current machine is slow and low on memory.
 +
* '''Next Steps'''
 +
** Abdullah is to set up his testbed and run some experiments.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this week)
 +
** I finalized the delay experiments and updated/sent the working document.
 +
** I phone-interviewed the new candidate and sent comments/recommendations to Dr. Hefeeda.
 +
** Ahmed worked on the dual redundancy with "hot swap" experiment.
 +
** I have been doing some research on real Modbus-based processes or how to build one, but I haven't concluded this research yet.
 +
* '''Issues'''
 +
** Lack of clear visioin for the future of this work.
 +
* '''Next Steps'''
 +
** Ahmed is to finish the fault-tolerance experiment.
 +
** I will continue my research on real processes.
 +
 +
 +
==Oct 1-5==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this week)
 +
** Ahmed came up with results on application-level testing using Apache-wget.
 +
** I have finished up writing the IPDPS paper and sent it for Dr. Hefeeda to review.
 +
** I have addressed at least 80% of Dr. Hefeeda's comments and submitted the paper to IPDPS.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed is working on updating his report and writing his thesis.
 +
** I have some leftover comments that I will need to address in the paper.
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this week)
 +
** Abdullah came up with I/O logs from the benchmark.
 +
** During a one-on-one, I ran partial auto-correlation function (PACF) on the log, and the result is a first-order Markov Chain.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Abdullah will come up with logs with the same format from HPC applications.
 +
** We will start working on defining our state space, action space, and reward/penalty function.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this week)
 +
** Ahmed successfully ran the delay experiments and obtained results.
 +
** I asked for additional experiments for a different scan cycle, which Ahmed ran on the spot!
 +
** The results show great robustness for up to 6.4 seconds of round-trip delay.
 +
* '''Issues'''
 +
** None
 +
* '''Next Steps'''
 +
** Ahmed will run an experiment for dual redundancy.
 +
** I am about to decide on the delay estimator to feed Smith Predictor (for variable delay cases).
 +
 +
 +
==Sep 24-28==
 +
===PCIe Interconnect===
 +
* '''Accomplished''' (this week)
 +
** I registered the paper abstract in IPDPS.
 +
** I ported the paper to LaTeX and worked on the abstract, introduction, related work and problem definition.
 +
** Ahmed's code now works with most TCP clients and servers.
 +
* '''Issues'''
 +
** We are running the experiments now (Friday night) and time is tight for the IPDPS submission.
 +
* '''Next Steps'''
 +
** We will finish as much experiments as we can Friday and Saturday.
 +
** I should have the paper (almost) ready Saturday night.
 +
 +
 +
===HPC Storage===
 +
* '''Accomplished''' (this week)
 +
** I had a couple of meetings with Abdullah; he came up with some file access logs from a benchmark
 +
** We ran the partial autocorrelation function on it to get the Markov order.
 +
** I up with a rough state definition based on the requested blocks (or rather "chunks") and storage status. I also came up with a rough definition for the reward/penalty function based on the same info.
 +
* '''Issues'''
 +
** Progress is a bit slow, but not too bad.
 +
* '''Next Steps'''
 +
** Abdullah is to get more logs from other real HPC applications
 +
** I will continue to define the Markov Decision Process.
 +
 +
 +
===Industrial Automation as a Service===
 +
* '''Accomplished''' (this week)
 +
** I had a couple of meetings with Ahmed where I explained to him in more details what experiments to run.
 +
** Ahmed prepared the testbed based on new cloud instances. He is very close to getting the results.
 +
** I had a Skype call with Ali to explain to him almost the same thing as Ahmed, though Ali is a little behind Ahmed.
 +
* '''Issues'''
 +
** Progress is a bit too slow; we could have easily obtained results this week, but we didn't.
 +
* '''Next Steps'''
 +
** Ahmed should come up with the results soon.
 +
** More experiments (this time, on random delay) are to be conducted.
 +
** Ali should be able to run the same experiments soon (just to get familiar; nothing new to accomplish).
 +
 +
 
==Sep 17-21==
 
==Sep 17-21==
−
===PCIe Project===
+
===PCIe Interconnect===
 
* '''Accomplished''' (this week)
 
* '''Accomplished''' (this week)
 
** Ahmed bought and successfully installed the Infiniband cards.
 
** Ahmed bought and successfully installed the Infiniband cards.
Line 9: Line 287:
 
** Time is tight for the IPDPS submission.
 
** Time is tight for the IPDPS submission.
 
* '''Next Steps'''
 
* '''Next Steps'''
−
* Ahmed will implement asynchronous call support in order to run his application-based evaluation and compare it to Infiniband.
+
** Ahmed will implement asynchronous call support in order to run his application-based evaluation and compare it to Infiniband.
−
* I will register the paper's abstract to IPDPS over the weekend (deadline is Monday, Sept 24th).  
+
** I will register the paper's abstract to IPDPS over the weekend (deadline is Monday, Sept 24th).  
−
* Although the time is tight, Ahmed will try to finish his asynchronous calls support and the application-based evaluation by the end of next week. This gives us a couple of days to polish up the paper.
+
** Although the time is tight, Ahmed will try to finish his asynchronous calls support and the application-based evaluation by the end of next week. This gives us a couple of days to polish up the paper.
 +
 
  
 
===HPC Storage===
 
===HPC Storage===
Line 22: Line 301:
 
** Abdullah to work on the coarser representation of the log (e.g., 1 second and MBs).
 
** Abdullah to work on the coarser representation of the log (e.g., 1 second and MBs).
 
** I will go into one more level of details for the general methodology.
 
** I will go into one more level of details for the general methodology.
 +
  
 
===Industrial Automation as a Service===
 
===Industrial Automation as a Service===
Line 29: Line 309:
 
** Ali finished his first experiment, which is nothing other than making sure Modbus is working fine for him on LabVIEW.
 
** Ali finished his first experiment, which is nothing other than making sure Modbus is working fine for him on LabVIEW.
 
* '''Issues'''
 
* '''Issues'''
−
** Ahmed had was supposed to finish his first task by the end of the week, but he was "discussing some issues with Dr. Mohamed regarding [his] research".
+
** Ahmed was supposed to finish his first task by the end of the week, but he was "discussing some issues with Dr. Mohamed regarding [his] research".
 
** Communication rate with Ali is a bit slow.
 
** Communication rate with Ali is a bit slow.
 
* '''Next Steps'''
 
* '''Next Steps'''

Latest revision as of 23:54, 17 April 2013

Mar 2013 to Apr 2013

Industrial Automation as a Service

  • Overall progress: 70%
  • Physical model building: 100% (Note: I have to fix a broken gearbox of solar collector and bidirectional pump leak)
  • Wiring: 95% (Remaining 5%: wiring missing power circuits when done)
  • Individual component testing: 100% (Note: I have to re-test the motor with gearbox and the leaking pump when fixed)
  • Complete system open-loop test: 100% (Note: I have to re-test the solar positioning process after fixing the gearbox)
  • Complete system feedback test: 60% (Remaining 40%: temperature control loops)
  • Power circuit design and breadboard testing: 90% (Remaining 10%: tuning the H-bridge circuit for the bidirectional pump and solid state switches for heaters)
  • Power circuit soldering on printed circuit board: 40% (Remaining 60%: soldering 1 more H-bridge circuit, 5 Darlington pair power amplifiers, 2 solid-state switches)
  • LabVIEW controller implementation and tuning: 40% (Remaining 60%: controllers for the temperature control loops and adding FT components)


Below are the tasks I performed to achieve the progress above. Please note that some of the items were very time consuming due to the nature of the physical labor involved.

  • Made about 10 trips to electronics shops in Vancouver, Home Depot, Canadian Tire, and Princess Auto to purchase some tools, and various electronic/electrical components.
  • Reorganized the installation of some physical components for better demonstration. For example, I had to craft "spacers" to flip the flowmeters upside down because the bottom of them shows a rotating propeller whose speed indicated the flow. The original installation just shows the solid head of the meter, nothing visual.
  • Installed a small heater to emulate solar heat.
  • Installed an additional pump for the "steam" cycle; one pump was too weak for the cycle.
  • Re-wired the whole system a few times to try to optimize wiring.
  • Refined/tested the power circuits for the actuators.
  • Soldered them on printed circuit boards. I had to de-solder and solder a couple of times. (very time consuming).
  • Crafted (manually) heat sinks for the power transistors; I couldn't find commercially available ones for my design.
  • Solved (partially) an issue with the fault tolerance algorithm that has to do with the limited registers in the I/O interface.


Dec 2012 to Mar 2013

PCIe Interconnect

  • Accomplished (this period)
    • Ahmed addressed comments and defended his thesis.
    • Another submission has been made and rejected.
  • Issues
    • If work not submitted to another conference/workshop, it will likely never get published.
  • Next Steps
    • N/A


HPC Storage

  • Accomplished (this period)
    • Abdullah got a simplified version of the DP algorithm to work.
  • Issues
    • Progress is very slow.
  • Next Steps
    • Abdullah is currently working on an expanded state space.


Industrial Automation as a Service

  • Accomplished (this period)
    • I performed more emulation experiments.
    • I added 3 theorems to the formal arguments.
    • We finalized and submitted the SIGCOMM paper.
    • I prepared the forms of the architecture and smooth handover patents.
    • I allocated a few days to provide the necessary material for the patents.
    • As for the physical model, I have wired and configured the I/O interface for some control loops.
    • I have faced and resolved a number of issues getting individual loops to work. Without going into details, the most important issues are:
      • Designing power circuits for the pumps
      • Configuring the Modbus port forwarding to work for Dynamic IP connections.
    • I got the cloud controllers to control individual control loops (flow and temperature).
    • I am also able to get the fault tolerance algorithm to work.
    • To get the delay compensator to work, the main issue that is left for me is to find a solution/workaround for the blocking TCP sockets of LabVIEW. Worse come to worst, I may have to write the protocol myself (something I am trying to avoid so far).
  • Issues
    • None
  • Next Steps
    • I will need to come up with a non-blocking implementation of Modbus/TCP protocol.

Oct 29-Nov 23

PCIe Interconnect

  • Accomplished (this period)
    • Ahmed has updated his thesis multiple times and performed more experiments, especially on CPU utilization.
    • I have reviewed Ahmed's thesis multiple times and gave him my feedback.
  • Issues
    • None
  • Next Steps
    • Ahmed will continue improving his thesis.
    • Ahmed will defend in 2 weeks.


HPC Storage

  • Accomplished (this period)
    • Abdullah has switched to a different HPC application and evaluated its model order.
    • I put together a plan to traget HPDC.
    • Abdullah has performed the literature review and delivered a write-up.
  • Issues
    • We cannot explain an anomaly whereby SSD is performing slightly worse than a regular HDD. This happened for 2 HPC applications, even when we artificially made them more I/O intensive. This is really holding us back.
    • Time is tight for HPDC.
  • Next Steps
    • Abdullah is building a fist order Markov Model to use it in our stochastic dynamic programming.
    • I will start writing the introduction and related work section.


Industrial Automation as a Service

  • Accomplished (this period)
    • I have been working on experimentation and writing of the "Automation as a Service" paper.
    • I have finished the proposed work section according to our last discussion.
    • I held several meetings and phone calls with the controls company and we decided on the physical process and I received an unofficial quotation.
  • Issues
    • Paper submission is delayed.
  • Next Steps
    • I will continue working on finalizing the paper.
    • I should be receiving an official quotation from the controls company soon. We should be making the order shortly.


Oct 22-26

PCIe Interconnect

  • Accomplished (this week)
    • Ahmed is working on writing his thesis and some application-level experimentation.
  • Issues
    • None
  • Next Steps
    • Ahmed will continue the same work.
    • I will review any work as soon as it is accomplished


HPC Storage

  • Accomplished (this week)
    • I reviewed Abdullah's report and sent him my comments
    • Abdullah used Kiana's machine temporarily, and managed to understand why the code wasn't utilizing all the cores. We will eventually need a machine with high specs, but I would rather wait until he's done experimenting with what he has. Then, we will have a better idea about the most cost-effective specs for his new machine.
  • Issues
    • None
  • Next Steps
    • Run the example HPC application on SSD and examine the improvement in performance (not accomplished from last week)


Industrial Automation as a Service

  • Accomplished (this week)
    • I met with a person who owns a workshop who is going to manufacturer the physical process. Quotation is due in about a week.
    • I am working on the SIGMETRICS paper.
    • Ahmed finished the fault tolerance experiments.
  • Issues
    • None
  • Next Steps
    • I will continue working on the SIGMETRICS paper.


Oct 15-19

PCIe Interconnect

  • Accomplished (this week)
    • Ahmed presented his work at the seminar.
    • I had a brief meeting with Ahmed where I explained to him the need for more experiments.
    • Ahmed started on addressing Dr. Hefeeda's comments.
  • Issues
    • None
  • Next Steps
    • Ahmed will continue working on improving the IPDPS paper.
    • Ahmed will start on getting his SDP implementation to work with DB2 application.


HPC Storage

  • Accomplished (this week)
    • Abdullah has written up a report, which I am currently reviewing.
    • Abdullah tried to get the test HPC application to work on other faster machines but he couldn't.
  • Issues
    • Abdullah needs a fast Linux machine because we believe that the problem size of the application cannot be increased due to resource limitation.
  • Next Steps
    • Run the example HPC application on SSD and examine the improvement in performance.


Industrial Automation as a Service

  • Accomplished (this week)
    • I have researched process control demo systems.
    • I met a representative from Davis Controls where he presented to me a number of Modbus solutions and said he would supply the necessary sensors/actuators and the physical process.
    • I dug deeper into the material on control theory for computing systems, and I have some comments and suggestions, which I discussed over the conference call.
  • Issues
    • I am still looking for a candidate who could help me on this project with both control and computing background.
  • Next Steps
    • I will wrap up the work done so far as we discussed and hopefully make it ready for publication.
    • I will meet with the workshop engineer who will manufacture the physical industrial process.


Oct 8-12

PCIe Interconnect

  • Accomplished (this week)
    • Ahmed prepared his seminar presentation for next week.
    • I had a meeting with him to give him my feedback/suggestions for the presentation.
    • Ahmed reviewed the IPDPS paper and he discussed with me some modifications that need to be made.
  • Issues
    • None
  • Next Steps
    • Ahmed is waiting for his meeting with Dr. Hefeeda this week to discuss next steps towards defense.


HPC Storage

  • Accomplished (this week)
    • Abdullah obtained a real I/O log from an ocean simulation application.
    • We ran the partial autocorrelation function on it. It shows tendency towards a second-order Markov model.
    • We discucssed a scaled down version of the testbed, where he would have HDD and ramdisk storage. Using the testbed, we have to show some improvement if we place the most frequently accessed files in the ramdisk.
  • Issues
    • Abdullah needs a more powerful machine to serve as the scaled-down version of the testbed; his current machine is slow and low on memory.
  • Next Steps
    • Abdullah is to set up his testbed and run some experiments.


Industrial Automation as a Service

  • Accomplished (this week)
    • I finalized the delay experiments and updated/sent the working document.
    • I phone-interviewed the new candidate and sent comments/recommendations to Dr. Hefeeda.
    • Ahmed worked on the dual redundancy with "hot swap" experiment.
    • I have been doing some research on real Modbus-based processes or how to build one, but I haven't concluded this research yet.
  • Issues
    • Lack of clear visioin for the future of this work.
  • Next Steps
    • Ahmed is to finish the fault-tolerance experiment.
    • I will continue my research on real processes.


Oct 1-5

PCIe Interconnect

  • Accomplished (this week)
    • Ahmed came up with results on application-level testing using Apache-wget.
    • I have finished up writing the IPDPS paper and sent it for Dr. Hefeeda to review.
    • I have addressed at least 80% of Dr. Hefeeda's comments and submitted the paper to IPDPS.
  • Issues
    • None
  • Next Steps
    • Ahmed is working on updating his report and writing his thesis.
    • I have some leftover comments that I will need to address in the paper.


HPC Storage

  • Accomplished (this week)
    • Abdullah came up with I/O logs from the benchmark.
    • During a one-on-one, I ran partial auto-correlation function (PACF) on the log, and the result is a first-order Markov Chain.
  • Issues
    • None
  • Next Steps
    • Abdullah will come up with logs with the same format from HPC applications.
    • We will start working on defining our state space, action space, and reward/penalty function.


Industrial Automation as a Service

  • Accomplished (this week)
    • Ahmed successfully ran the delay experiments and obtained results.
    • I asked for additional experiments for a different scan cycle, which Ahmed ran on the spot!
    • The results show great robustness for up to 6.4 seconds of round-trip delay.
  • Issues
    • None
  • Next Steps
    • Ahmed will run an experiment for dual redundancy.
    • I am about to decide on the delay estimator to feed Smith Predictor (for variable delay cases).


Sep 24-28

PCIe Interconnect

  • Accomplished (this week)
    • I registered the paper abstract in IPDPS.
    • I ported the paper to LaTeX and worked on the abstract, introduction, related work and problem definition.
    • Ahmed's code now works with most TCP clients and servers.
  • Issues
    • We are running the experiments now (Friday night) and time is tight for the IPDPS submission.
  • Next Steps
    • We will finish as much experiments as we can Friday and Saturday.
    • I should have the paper (almost) ready Saturday night.


HPC Storage

  • Accomplished (this week)
    • I had a couple of meetings with Abdullah; he came up with some file access logs from a benchmark
    • We ran the partial autocorrelation function on it to get the Markov order.
    • I up with a rough state definition based on the requested blocks (or rather "chunks") and storage status. I also came up with a rough definition for the reward/penalty function based on the same info.
  • Issues
    • Progress is a bit slow, but not too bad.
  • Next Steps
    • Abdullah is to get more logs from other real HPC applications
    • I will continue to define the Markov Decision Process.


Industrial Automation as a Service

  • Accomplished (this week)
    • I had a couple of meetings with Ahmed where I explained to him in more details what experiments to run.
    • Ahmed prepared the testbed based on new cloud instances. He is very close to getting the results.
    • I had a Skype call with Ali to explain to him almost the same thing as Ahmed, though Ali is a little behind Ahmed.
  • Issues
    • Progress is a bit too slow; we could have easily obtained results this week, but we didn't.
  • Next Steps
    • Ahmed should come up with the results soon.
    • More experiments (this time, on random delay) are to be conducted.
    • Ali should be able to run the same experiments soon (just to get familiar; nothing new to accomplish).


Sep 17-21

PCIe Interconnect

  • Accomplished (this week)
    • Ahmed bought and successfully installed the Infiniband cards.
    • Benchmark experiments were run for Infiniband.
  • Issues
    • Ahmed's implementation of SDP doesn't support asynchronous calls.
    • Ahmed couldn't run PCIe and Infiniband experiments from the same machine (library conflict between his SDP and Infiniband SDP)
    • Time is tight for the IPDPS submission.
  • Next Steps
    • Ahmed will implement asynchronous call support in order to run his application-based evaluation and compare it to Infiniband.
    • I will register the paper's abstract to IPDPS over the weekend (deadline is Monday, Sept 24th).
    • Although the time is tight, Ahmed will try to finish his asynchronous calls support and the application-based evaluation by the end of next week. This gives us a couple of days to polish up the paper.


HPC Storage

  • Accomplished (this week)
    • Abdullah tried to obtain HPC I/O logs off of the internet. When that was not possible, he tried a couple of benchmarks to to generate some I/O logs as a placeholder until we obtain logs from real HPC applications.
    • During our meeting, we tried to analyze the data, e.g., to verify the Markov chain order using partial autocorrelation functions. I noticed the data was so fine-grained (time in nanoseconds and block size in KB).
  • Issues
    • None
  • Next Steps
    • Abdullah to work on the coarser representation of the log (e.g., 1 second and MBs).
    • I will go into one more level of details for the general methodology.


Industrial Automation as a Service

  • Accomplished (this week)
    • I have come up with the delay compensation results and updated the working document.
    • I have met with Ahmed AbdelSadek and explained the problem and experiments to him.
    • Ali finished his first experiment, which is nothing other than making sure Modbus is working fine for him on LabVIEW.
  • Issues
    • Ahmed was supposed to finish his first task by the end of the week, but he was "discussing some issues with Dr. Mohamed regarding [his] research".
    • Communication rate with Ali is a bit slow.
  • Next Steps
    • I may have to work on the delay experiments myself until Ahmed is up to speed.
    • Ahmed will try to accomplish the tasks rolled over from this week.
    • I will have conference call with Ali to explain to him the next steps.
Retrieved from "https://nmsl.cs.sfu.ca/index.php?title=Private:progress-hegazy&oldid=5366"
Powered by MediaWiki