Socekt Direct Protocol (SDP) over PCI Express Project Page
In this project we designed and implemented the Socket Direct Protocol (SDP) for PCI Express based fabric. SDP is an efficient data transfer protocol alternative to the TCP that utilizes the advanced features available in high performance interconnects such as direct memory access. Our SDP implementation is designed for PLX PCIe switches and supports only two nodes for now. SDP is compatible with Socket/TCP applications yet provides much better performance. It is developed as a Linux kernel module and has been tested under Redhat Linux 5.7.
Installation
Installing PCIe Drivers
1. Download the drivers from http://www.plxtech.com/download/file/899/217277 after registrations
2. Unzip the package
# Unzip PLX_SDK_Linux_v6_50_Final_2011-09-30.zip
3. Make sure that you are at the same level as PlxSdk and apply the patch:
#ls Documentation PlxSdk PlxSdk.tar
# patch -p1 < ../patch.txt patching file PlxSdk/Include/PlxIoctl.h patching file PlxSdk/Include/Plx_sysdep.h patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.c patching file PlxSdk/Linux/Driver/Source.Plx8000/ApiFunc.h patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c patching file PlxSdk/Linux/Driver/Source.Plx8000/Dispatch.c_back patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h patching file PlxSdk/Linux/Driver/Source.Plx8000/DrvDefs.h_back patching file PlxSdk/Linux/Driver/Source.Plx8000/PlxInterrupt.c patching file PlxSdk/Linux/Driver/Source.Plx8000/SuppFunc.c patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c patching file PlxSdk/Linux/Driver/Source.PlxDma/Dispatch.c_back patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h patching file PlxSdk/Linux/Driver/Source.PlxDma/DrvDefs.h_back patching file PlxSdk/Linux/Driver/.tmp_versions/PlxDma_dbg.mod patching file PlxSdk/Linux/Samples/ApiTest/ApiTest.c patching file PlxSdk/Linux/Samples/NT_DmaTest/NT_DmaTest.c patching file PlxSdk/Linux/Samples/NT_myTest/Makefile patching file PlxSdk/Linux/Samples/NT_myTest/NT_DmaTest.c patching file PlxSdk/Linux/Samples/NT_myTest/PlxDmaPerf.c patching file PlxSdk/Linux/Samples/PlxDma/PlxDma.c patching file PlxSdk/Linux/Samples/PlxNotification/PlxNotification.c
4. Make sure that Plx main directory is defined in you environment:
# export PLX_SDK_DIR=/home/ambu50/trunk/Plx_new_download/PlxSdk/
5. Change directory to the drivers folder:
cd Linux/Driver/
6. Run build builddriver:
# ./builddriver 8000 # ./builddriver Dma
7. Load the drivers:
# ../../Bin/Plx_load 8000 # ../../Bin/Plx_load Dma
Installing libsdp Library
1. Download the library http://nsl.cs.sfu.ca/wiki/upload/8/86/Libsdp-1.1.108.tar.gz 2. Extract and install using "./configure", "make", "make install"
Installing SDP Kernel Module
1. Download the source code of SDP kernel module http://nsl.cs.sfu.ca/wiki/upload/a/a7/Pcie_sdp.tar.gz 2. Extract and install
# tar zxvf Pcie_sdp.tar.gz # cd Pcie_sdp # make
3. Load the module to the kernel
insmod ib_sdp.ko
Testing the SDP over PCIe
To test the module you can run any TCP based client server application in one of two ways. 1. Specifying the location of libsdp
LD_LIBRARY_PATH=/usr/local/lib LD_PRELOAD=libsdp.so YOUR_SERVER_EXECUTABLE
2. By configuring the process name and port number in libsdp.conf that can be found under /usr/local/etc/libsdp.conf or /etc/libsdp.conf
