Difference between revisions of "pcie sdp"

From NMSL
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Socekt Direct Protocol (SDP) over PCI Express Project Page=  
+
= Socket Direct Protocol (SDP) over PCI Express Project Page=  
  
 
In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.
 
In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.
 +
  
 
== Installation ==
 
== Installation ==
 +
 
== Installing PCIe Drivers ==
 
== Installing PCIe Drivers ==
1. Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
+
# Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
 
+
# Unzip the package  
2. Unzip the package  
+
#:<pre> # Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip</pre>
# Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip
+
# Make sure that you are at the same level as PlxSdk and apply the patch:
 
+
#:<pre> #ls&#10; Documentation  PlxSdk  PlxSdk.tar </pre>
3. Make sure that you are at the same level as PlxSdk and apply the patch:
+
#:<pre> # patch -p1 < ../patch.txt &#10; patching file PlxSdk/Include/PlxIoctl.h &#10; patching file PlxSdk/Include/Plx_sysdep.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back &#10; patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod &#10; patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c &#10; patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c &#10; patching file PlxSdk/Linux/Samples/NT_myTest/Makefile &#10; patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c &#10; patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c &#10; patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c &#10; patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c</pre>
#ls
+
# Make sure that Plx main directory is defined in you environment:
Documentation  PlxSdk  PlxSdk.tar
+
#:<pre> # export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/</pre>
 
+
# Change directory to the drivers folder:
# patch -p1 < ../patch.txt  
+
#:<pre> cd Linux/Driver/ </pre>
patching file PlxSdk/Include/PlxIoctl.h
+
# Run build builddriver:
patching file PlxSdk/Include/Plx_sysdep.h
+
#:<pre> # ./builddriver 8000 &#10; # ./builddriver Dma</pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c
+
# Load the drivers:
patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h
+
#:<pre> # ../../Bin/Plx_load 8000 &#10; # ../../Bin/Plx_load Dma</pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c
 
patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back
 
patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h
 
patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back
 
patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c
 
patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c
 
patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c
 
patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back
 
patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h
 
patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back
 
patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod
 
patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c
 
patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c
 
patching file PlxSdk/Linux/Samples/NT_myTest/Makefile
 
patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c
 
patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c
 
patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c
 
patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c
 
 
 
4. Make sure that Plx main directory is defined in you environment:
 
# export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/
 
  
5. Change directory to the drivers folder:
 
cd Linux/Driver/
 
6. Run build builddriver:
 
# ./builddriver 8000
 
# ./builddriver Dma
 
7. Load the drivers:
 
# ../../Bin/Plx_load 8000
 
# ../../Bin/Plx_load Dma
 
  
 
== Installing libsdp Library ==
 
== Installing libsdp Library ==
1. Download the library http://nsl.cs.sfu.ca/wiki/upload/8/86/Libsdp-1.1.108.tar.gz
+
# Download the library [http://nsl.cs.sfu.ca/projects/pcie-sdp/libsdp-1.1.108.tar.gz libsdp-1.1.108.tar.gz]
2. Extract and install using "./configure", "make", "make install"
+
# Extract and install using "./configure", "make", "make install"
  
 
== Installing SDP Kernel Module ==
 
== Installing SDP Kernel Module ==
1. Download the source code of SDP kernel module http://nsl.cs.sfu.ca/wiki/upload/a/a7/Pcie_sdp.tar.gz
+
# Download the source code of SDP kernel module [http://nsl.cs.sfu.ca/projects/pcie-sdp/pcie_sdp.tar.gz pcie_sdp.tar.gz]
2. Extract and install
+
# Extract and install  
# tar zxvf Pcie_sdp.tar.gz
+
#:<pre> # tar zxvf Pcie_sdp.tar.gz&#10; # cd Pcie_sdp&#10; # make</pre>
# cd Pcie_sdp
+
# Load the module to the kernel  
# make
+
#:<pre> insmod ib_sdp.ko</pre>
3. Load the module to the kernel
 
insmod ib_sdp.ko
 
  
 
== Testing the SDP over PCIe ==
 
== Testing the SDP over PCIe ==
 
To test the module you can run any TCP based client server application in one of two ways.
 
To test the module you can run any TCP based client server application in one of two ways.
1. Specifying the location of libsdp
+
* Specifying the location of libsdp
LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE
+
*:<pre> LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE</pre>
2. By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf
+
* By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf

Latest revision as of 09:44, 22 September 2017

Socket Direct Protocol (SDP) over PCI Express Project Page

In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.


Installation

Installing PCIe Drivers

  1. Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
  2. Unzip the package
     # Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip
  3. Make sure that you are at the same level as PlxSdk and apply the patch:
     #ls
     Documentation  PlxSdk  PlxSdk.tar 
     # patch -p1 < ../patch.txt 
     patching file PlxSdk/Include/PlxIoctl.h 
     patching file PlxSdk/Include/Plx_sysdep.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back 
     patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod 
     patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c 
     patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c 
     patching file PlxSdk/Linux/Samples/NT_myTest/Makefile 
     patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c 
     patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c 
     patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c 
     patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c
  4. Make sure that Plx main directory is defined in you environment:
     # export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/
  5. Change directory to the drivers folder:
     cd Linux/Driver/ 
  6. Run build builddriver:
     # ./builddriver 8000 
     # ./builddriver Dma
  7. Load the drivers:
     # ../../Bin/Plx_load 8000 
     # ../../Bin/Plx_load Dma


Installing libsdp Library

  1. Download the library libsdp-1.1.108.tar.gz
  2. Extract and install using "./configure", "make", "make install"

Installing SDP Kernel Module

  1. Download the source code of SDP kernel module pcie_sdp.tar.gz
  2. Extract and install
     # tar zxvf Pcie_sdp.tar.gz
     # cd Pcie_sdp
     # make
  3. Load the module to the kernel
     insmod ib_sdp.ko

Testing the SDP over PCIe

To test the module you can run any TCP based client server application in one of two ways.

  • Specifying the location of libsdp
     LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE
  • By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf