Private:progress-bu-khamsin
2012
Points to descuss
- I noticed that DMA transfer is not implemented on the driver downloaded from the chips maker website. There is a possibility that the function is implemented on the card maker driver which is shipped with the hardware. I tired to open a support account with the chip maker website but my request was rejected because I am using a university email! I have registered again with my company email and haven't received a response yet.
- Every thing else is fine, I am about to finish with the driver API while the SDP module need more work.
February
The way PLX_API in the user space interacting with the driver by opening the driver and sending ioctl requests. So to provide similar functionality to kernel space, I have written similar function to the ioctl handler and exported it to the kernel. This function requires an instance of device_object structure as one of its parameter. To make this structure available in the kernel space, I have written and exported another function that get this structure from the driver and makes it available to the kernel.
After finished the API, I have started working on a test module that uses the API to transfer data between two machines. However, I am facing some problems with the hardware register and memory mapping to the user space. As the module is running purely on the kernel space, I couldn't find a way to allocate user space address to be used for the mapping. And even if I managed to do so, all the kernel calls I found require file descriptor as a parameter which can only be obtained when the driver is opened from the user space. I looked for a solution for this problem and found some posts that advice to refere to Infiniband driver. So that is what I am doing right now. I have studied the driver before but didn't go in depth in the memory management part.
January 30
I found out that the PLX API available on the SDK is only accessible by application written for user space and cannot be used for kernel space modules. So I am working right now on writing similar functions for kernel space which requires modification to PLX drivers.
January 27
I find it hard to replace RDMA calls with DMA calls as each API is written with different level of abstraction. So I find my self in a need to have deeper understanding of memory organization and how it is divided. Also I need to know which part of memory is accessible via DMA and the kind of addressing required and how to translate virtual to physical and virtual to PCI address. After I finished reading few chapters from the books below, I am now going through the code again trying to apply what I have learned. I also find the documentation of the RDMA and DMA API not very detailed and doesn't describe the affect of calling the functions, so I need either to go through the implementation files of the API or playing with the sample applications after I receive the hardware.
January 26
- Reading chapters from "LINUX DEVICE DRIVERS" THIRD EDITION Jonathan Corbet, Alessandro, Rubini, and Greg Kroah-Hartman
January 25
- Reading chapters from "Understanding the Linux® Virtual Memory Manager".
January 22
- Successfully isolated and compiled the Infiniband SDP driver:
Finally, I was able to compile the driver without modifying the kernel headers. I installed Centos 5.7 and installed and compiled kernel 2.6.18. Then applied some patches included with the source code of the driver that matches this version of the kernel. After that I moved all the dependencies to the driver directory and modified some Makefiles and some includes in the header files of the driver.
- Studying PLX NTB DMA Drivers
- Reading about DMA programming
January 3
- Study the differences between DMA and RDMA
- Discover how Infiniband SDP is exchanging local memory addresses
January 1
- Trying to prepare the development environment and find the right version of kernel:
Before I start modifying the code of the infiniband SDP implementation, I need to find a way to build the source code smoothly. Building Kernel modules under 2.6 Linux kernel requires downloading and compiling the kernel first. I have tried different linux distributions and install different kernels but the SDP code keeps giving me errors during compilation time. I managed to compile the driver after I modified few headers in the kernel and in the driver code but I don't think this is the right way. I am still trying..
- Identify driver dependencies and find PCIe equivalent.
Fall 2011
- Courses:
- CMPT 880 Programming Parallel and Distributed Systems
December 27
- Isolating interested code from OFED Infiniband software stack.
- Socket switch modul can be used with no change
December 26
- Reading chapters from "Understanding Linux Network Internals"
- Going through the source code of SDP implementation for Infiniband
December 12
- Exploring PLX SDK
- Studying Infiniband implementation of SDP and how to bypass TCP while using standard socket API
- Researching how to create devices in Linux
- Working on course project
November 30
- Collected all the information I need about NTB and SDP.
- Registered to PLX website to download and experiment with their SDK and development kit.
November 14
- Learning how to write kernel modules
- Learning about PCIe protocol layers
- Researching the different types of PCIe bridges and adapters.
September 19
- Here is my progress report [1]
September 12
- Working on my research progress report
Summer 2011
- Courses:
- CMPT 777 Formal Verification
June 17
Studying for the final
June 6
No update
May 8
Beside my course work, I am planning to contunue work on my project. Check the latest report. [2]
Spring 2011
- Courses:
- CMPT 771 Internet Arch and Protocols
- CMPT 886 Special Topics Operating Syst
April 8
Plotted the initial result of an experiment that supports the claim that process network traffic is correlated to execution times which makes network traffic a good metric for performance degradation due to interconnect contention. What is left is to run the experiment two more times to confirm the results.
Mar 14
- Work on Course Projects.
- Experiment with a tool that measure network traffic per process on HPC systems to use it on my project.
Mar7
- Work on Course Projects.
- Read about power consumption effects on multi-core system performance.
Mar 1
- Work on Course Projects.
- Prepare description for my new research project.
Jan21
- I have been working on "Top Ten Computationally-Complex Problems in Oil and Gas Exploration Filed" survey and trying to enrich my overall knowledge about the subject. I am focusing right now on applications based on seismic data and have written an introduction about it in the report [3]
- I have also found a very interesting book "Soft Computing and Intelligent Data Analysis in Oil Exploration" by M. Nikravesh, L.A. Zadeh, Fred Aminzadeh [4]. It is mainly about solving petroleum engineering problems using artificial intelligent techniques, which I think can lead me to find interesting research topic.
Fall 2010
- Courses:
- CMPT 705 Design/Analysis Algorithms
- CMPT 741 Data Mining
