Difference between revisions of "pcie sdp"

From NMSL
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Socekt Direct Protocol (SDP) over PCI Express Project Page=  
+
= Socket Direct Protocol (SDP) over PCI Express Project Page=  
  
 
In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.
 
In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.
 +
  
 
== Installation ==
 
== Installation ==
Install:
 
1. Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
 
2. # Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip
 
3. Make sure that you are at the same level as PlxSdk and apply the patch:
 
#ls
 
Documentation  PlxSdk  PlxSdk.tar
 
  
  # patch -p1 < ../patch.txt  
+
== Installing PCIe Drivers ==
patching file PlxSdk/Include/PlxIoctl.h
+
# Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
patching file PlxSdk/Include/Plx_sysdep.h
+
# Unzip the package
patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c
+
#:<pre> # Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip</pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h
+
# Make sure that you are at the same level as PlxSdk and apply the patch:
patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c
+
#:<pre> #ls&#10; Documentation PlxSdk  PlxSdk.tar </pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back
+
#:<pre> # patch -p1 < ../patch.txt &#10; patching file PlxSdk/Include/PlxIoctl.h &#10; patching file PlxSdk/Include/Plx_sysdep.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c &#10; patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h &#10; patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back &#10; patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod &#10; patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c &#10; patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c &#10; patching file PlxSdk/Linux/Samples/NT_myTest/Makefile &#10; patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c &#10; patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c &#10; patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c &#10; patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c</pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h
+
# Make sure that Plx main directory is defined in you environment:
patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back
+
#:<pre> # export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/</pre>
patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c
+
# Change directory to the drivers folder:
patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c
+
#:<pre> cd Linux/Driver/ </pre>
patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c
+
# Run build builddriver:
patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back
+
#:<pre> # ./builddriver 8000 &#10; # ./builddriver Dma</pre>
patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h
+
# Load the drivers:
patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back
+
#:<pre> # ../../Bin/Plx_load 8000 &#10; # ../../Bin/Plx_load Dma</pre>
patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod
 
patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c
 
patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c
 
patching file PlxSdk/Linux/Samples/NT_myTest/Makefile
 
patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c
 
patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c
 
patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c
 
patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c
 
  
4. Make sure that Plx main directory is defined in you environment:
 
# export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/
 
  
5. Change directory to the drivers folder:
+
== Installing libsdp Library ==
cd Linux/Driver/
+
# Download the library [http://nsl.cs.sfu.ca/projects/pcie-sdp/libsdp-1.1.108.tar.gz libsdp-1.1.108.tar.gz]
6. Run build builddriver:
+
# Extract and install using "./configure", "make", "make install"
# ./builddriver 8000
 
# ./builddriver Dma
 
7. Load the drivers:
 
# ../../Bin/Plx_load 8000
 
# ../../Bin/Plx_load Dma
 
  
 +
== Installing SDP Kernel Module ==
 +
# Download the source code of SDP kernel module [http://nsl.cs.sfu.ca/projects/pcie-sdp/pcie_sdp.tar.gz pcie_sdp.tar.gz]
 +
# Extract and install
 +
#:<pre> # tar zxvf Pcie_sdp.tar.gz&#10; # cd Pcie_sdp&#10; # make</pre>
 +
# Load the module to the kernel
 +
#:<pre> insmod ib_sdp.ko</pre>
  
To be continued...
+
== Testing the SDP over PCIe ==
 +
To test the module you can run any TCP based client server application in one of two ways.
 +
* Specifying the location of libsdp
 +
*:<pre> LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE</pre>
 +
* By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf

Latest revision as of 09:44, 22 September 2017

Socket Direct Protocol (SDP) over PCI Express Project Page

In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.


Installation

Installing PCIe Drivers

  1. Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
  2. Unzip the package
     # Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip
  3. Make sure that you are at the same level as PlxSdk and apply the patch:
     #ls
     Documentation  PlxSdk  PlxSdk.tar 
     # patch -p1 < ../patch.txt 
     patching file PlxSdk/Include/PlxIoctl.h 
     patching file PlxSdk/Include/Plx_sysdep.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c 
     patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h 
     patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back 
     patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod 
     patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c 
     patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c 
     patching file PlxSdk/Linux/Samples/NT_myTest/Makefile 
     patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c 
     patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c 
     patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c 
     patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c
  4. Make sure that Plx main directory is defined in you environment:
     # export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/
  5. Change directory to the drivers folder:
     cd Linux/Driver/ 
  6. Run build builddriver:
     # ./builddriver 8000 
     # ./builddriver Dma
  7. Load the drivers:
     # ../../Bin/Plx_load 8000 
     # ../../Bin/Plx_load Dma


Installing libsdp Library

  1. Download the library libsdp-1.1.108.tar.gz
  2. Extract and install using "./configure", "make", "make install"

Installing SDP Kernel Module

  1. Download the source code of SDP kernel module pcie_sdp.tar.gz
  2. Extract and install
     # tar zxvf Pcie_sdp.tar.gz
     # cd Pcie_sdp
     # make
  3. Load the module to the kernel
     insmod ib_sdp.ko

Testing the SDP over PCIe

To test the module you can run any TCP based client server application in one of two ways.

  • Specifying the location of libsdp
     LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE
  • By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf