Difference between revisions of "Private:psim"

From NMSL
 
(42 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
This page documents the development of a discrete event simulator for P2P video streaming applications. This simulator captures important features of data-driven video streaming systems. In particular, it is designed to evaluate: (i) the performance of various segment scheduling algorithms; (ii) the potential of network coding in multi-layer P2P video streaming systems.
 
This page documents the development of a discrete event simulator for P2P video streaming applications. This simulator captures important features of data-driven video streaming systems. In particular, it is designed to evaluate: (i) the performance of various segment scheduling algorithms; (ii) the potential of network coding in multi-layer P2P video streaming systems.
 
   
 
   
−
= Class Diagrams =
+
== Todo List ==
  
−
* use long for time/offset in msec, which has a rollover time 24.85 days
+
We keep a short todo list here. For each todo item, we also list the expected workload, the current assignee, the tentative due date, and the completion time.
  
−
<pre>
+
{| class="wikitable"
 +
|-
 +
! Description
 +
! Workload (ppl-day)
 +
! Assignee
 +
! Tentative Duedate
 +
! Completion time
 +
|-
 +
| Reread and implement Meng Zhang's algorithm
 +
| 1.5
 +
| Cheng
 +
| March 18, 2009
 +
| March 18, 2009
 +
|-
 +
| Finalize the simulator engine
 +
| 1
 +
| Cheng
 +
| March 17, 2009
 +
| March 17, 2009
 +
|-
 +
| Generalize the scheduling algorithm for multiple layer problem
 +
| 3
 +
| N/A
 +
|
 +
|
 +
|-
 +
| Reorganize the evaluation setup subsection
 +
| 2
 +
| Yuanbin
 +
| March 18, 2009
 +
| March 18, 2009
 +
|}
  
−
class Layer { // video layer, we divide each layer into fixed size blocks, we keep track of
+
= High-Level Design =
−
                    // block size and no blocks here
 
−
    int layer; // layer number it belongs to, set to zero for nonscalable frame
 
−
    int blockSize; // blocks are fixed size, in bytes
 
−
    int noBlocks; // how many block here
 
−
    int dataSize; // aggregate size of all blocks
 
−
}
 
  
−
class Frame { // video frame, read from trace file
+
We present the high level design of the pSim. Please find a coarse class diagram below, and the source file can be downloaded [[media:psim.dia|here]].
−
    int no; // serial number
 
−
    long deadlineOffset; // deadline offset compared to the start of the video, in msec
 
−
    int totalSize; // total frame size in bytes, including all layers
 
−
    Hashtable<Integer, Layer> layers; // reference to layers of this frame, key is the layer id
 
−
}
 
  
−
class Segment { // packetized frames
+
[[Image:psim.png|center|640px]]
−
    int no; // serial number
 
−
    Vector<Frame> frames; // reference to included frame
 
−
    int totalSize; // aggregate size in bytes
 
−
}
 
  
−
class Video { // a media file, shared by a group of peers
 
−
    String filename; // video trace file path and name
 
−
    Hashtable<int, Frame> frameTrace; // frames read from the trace file
 
−
    Hashtable<int, Segment> segmentTrace; // segments generated by the prepareSegments(...)
 
−
    void prepareSegments(int noFrame); // packetize frames into segment,
 
−
        // noFrame indicates how many frames should we put in one segment;
 
−
        // we might implement other packtization schemes later.
 
−
}
 
  
−
class Neighbor { // keep track of what my neighbor has done
 
−
    Peer peer; // peer instance, for accessing availability info
 
−
    int estimateRate; // estimated rate (maybe historical)
 
−
    Connection conn; // e2e connection between us and another neighboring peer
 
−
}
 
  
 +
Some misc notes:
 +
* We use long for time/offset in msec, which has a rollover time 24.85 days
  
−
class Tracker { // holds a set of groups/videos
+
=== System Architecture ===
−
    Vector<Group> groups; // all the video groups
+
The core of the simulator is a simulation engine that is abstracted as the
−
}
+
'''Simulator''' class. Since pSim is a discrete event simulator, the
 +
'''Simulator''' class keeps track of a moving simulation time '''simTime''',
 +
which is different from the wall clock (real-time). '''Simulator''' consists of
 +
an '''EventQueue''' that provides the interface of push and pop an event from a
 +
priority queue of the pending events. The pending events are sorted on the
 +
event time in the ascending order. '''Simulator''' works as the follows, in
 +
'''dispatchEvents''' method, the next event is popped from '''EventQueue'''.
 +
'''Simulator''' then update '''simTime''' and calls '''dispatch''' method to
 +
process that event using method '''dispatch''', and then drop the processed
 +
event away. '''Simulator''' then popped the next event from '''EventQueue'''
 +
and update '''simTime'''. That is, the method '''dispatchEvent''' is a forever
 +
loop, thus the caller must not expect '''dispatchEvent''' to return.
  
−
class Peer { // represent a running peer
+
Other components in pSim add new events to '''Simulator''' by calling its
−
    int ingressBW; // incoming bandwidth in bps
+
'''push''' method. The new events are pushed through '''EventQueue''' to
−
    int egressBW; // outgoing bandwidth in bps
+
'''CalendarQueue'''. '''CalendarQueue''' is an efficient implementation of
−
    Vector<Neighbor> neighbors; // peers that we may send requests to  
+
priority queue, which achieves constant time complexity. We implement the
−
    BufferedMatrix matrix; // matrix of coefficients and encoded values up to now
+
Dynamic Calendar Queue (DCQ) based on a Berkeley implementation, which sort the
−
    NoEncodedBlocks noCodedBl; // keep track of No. of encoded blocks in each segment
+
pending events on their '''time'''. We notice that we keep '''EventQueue'''
−
    Connection tracker; // control connection to the tracker
+
between '''Simulator''' and '''CalendarQueue''', in case we need to switch to
−
    Hashtable<Video, VideoBufMap> vBufMaps; // points to the video-level buffer map
+
other priority queue implementation such as standard '''TreeMap'''.
−
}
 
  
−
class Group { // peers that have downloaded or want to download a Video
+
In addition to '''EventQueue''', '''Simulator''' also consists of: (i)
−
    Video video; // media shared among peers
+
'''Logger''' that is responsible for filtering and saving logging messages,
−
    Vector<Peer> peers; // peers in this group
+
(ii) '''RndGen''' that is the system-wide random number generator, and (iii)
−
    Vector<Peer> join(Peer peer); // adding a new peer into this group, and return a list of senders
+
'''SysParam''' that has all system parameters.
−
    void leave(Peer peer); // removing a peer from this group
 
−
}
 
  
−
class Connection { // end to end network link between two peers
+
=== Event and Event Handler ===
−
    Peer peer1; // one end
+
Since '''Simulator''' class is driven by its priority queue of events, every
−
    Peer peer2; // the other
+
'''Event''' must consist of enough information on how to process that
−
    long delay; // transmission delay in msec
+
'''Event'''. The main fields of '''Event''' include: '''type''' that is the
−
    int e2eBW; // end-to-end bandwidth in bps
+
event type, and '''time''' that is the event time. Moreover, the '''Event'''
−
}
+
class maintains a vector of '''EventHandler''' interface, which specifies how
 +
to process that '''Event''''. There is only an abstract method defined in
 +
'''EventHandler''', which is its '''handle''' function. The '''handle
 +
function''' returns a log entry in '''String''' format. We note that this
 +
String must use pipe as the field delimiter, and should not contain any CR nor
 +
LF. See existing code for other convention of the log entries. Notice that,
 +
each '''Event''' instance has no '''EventHandler''' initially: the class who
 +
creates an '''Event''' must provide one or more '''EventHandler''' by calling
 +
the method '''addHandler'''. Last, we mention that the '''dispatch''' method of
 +
'''Simulator''' retrieves all '''EventHandler''' by calling method
 +
'''getHandlers'''.
  
−
class VideoBufMap {
+
'''Event''' has two subclasses: '''PeerEvent''' is for events that is local to
−
    Hashtable<Integer, SegBufMap> sBufMaps; // points to segment bufmap
+
a peer, e.g., a peer joins a video group is a '''PeerEvent''';
−
}
+
'''NetworkEvent''' is for sending messages between two peers, e.g., a receiver
 +
sends a '''CONNECTMSG''' to a sender. '''PeerEvent''' has two fields:
 +
'''Peer''' and '''Video'''. '''NetworkEvent''' has four fields: (i) '''conn'''
 +
that captures the connection status between two peers, (ii) '''length''' that
 +
describes the length of this message, (iii) '''direction''' that indicates
 +
whether the message is sent from receiver to sender or not, and (iv)
 +
'''arrived''' shows whether the message has arrived at the destination. More
 +
precisely, a '''NetworkEvent''' is created with '''arrived''' value set to off,
 +
which means that event just enter the connection '''conn'''.  Following the
 +
bandwidth defined in '''conn''', we can compute when will the event arrives at
 +
the destination. This computation is done in '''tHandler''' that stands for
 +
transmission handler. The transmission handler processes a '''NetworkEvent'''
 +
as follows. First, it computes and updates the event time to be the time that
 +
event arrives at the destination. Second, it sets '''arrived''' to be true.
 +
Third, it pushes the same '''NetworkEvent''' back to '''EventQueue'''.
  
−
class SegBufMap {
 
−
    Hashtable<Integer, LayerBufMap> lBufMaps; // points to layer bufmap
 
−
}
 
  
−
class LayerBufMap {
+
=== Random Number Generator and Profiles ===
−
    boolean avail; // availability bit for this (video, segment, layer) tuple
 
−
    NCBuf ncBuf; // network coding structure
 
−
}
 
  
−
class NCBuf {
+
TODO
−
    boolean servCapibility; // Serving capability of each segment
 
−
                                      // show whether this segment is capable of serving other peers(applicable in NC)
 
−
    int noCodedBl; // growable 2 dimensional array corresponding to segments and layers
 
−
                          // each element in this vector keep the No. of encoded blocks in the 
 
−
                          // corresponding segment
 
−
    BufferedMatrix marix; // coefficients and their corresponding encoded blocks up to now
 
−
}
 
  
−
class BufferedMatrix{ // save coefficients and their corresponding encoded blocks up to now
 
−
    Vector<Vector<Integer>> coef; // coefficients of the corresponding encoded blocks
 
−
    Vector <Integer>encodedValue; //encoded blocks
 
−
}
 
  
 +
= Performance metrics for scheduling =
  
−
// XXX TODO XXX
+
Here are some performance metrics:
 +
# Average delivery ratio: number of on-time scheduled segments over total number of segments to be scheduled
 +
# Load balance among senders
 +
# Initial buffering time
 +
# Time and space complexity of the scheduling algorithm
  
−
class SlidingWindow{ // the sliding window which contains all the up-to-date segments at the peer
 
−
                                  // set it to a fixed size, for example 1 minute of videos
 
−
  int startOffset;        //the offset of the starting frame compared with the start of the video
 
−
  int endOffset;          // the offset of the ending frame compared with the start of the video
 
−
  double slideSpeed;  // the speed of sliding. The window goes forward continuously (or periodically?) at
 
−
                                  // the speed of the streaming rate.
 
−
  void getSlidingWindowSize(); //get the size of the sliding window
 
−
}
 
  
−
class exchangeWindow{ // the exchange window is the front part of the sliding window.
+
= Performance metrics for network coding =
−
                                      //The unavailable segments in exchange window will be requested by the peer
 
−
                                      //usually we set it to a fixed size, for example, 10 seconds of video
 
−
                                      // the field of this class is similar to that of the SlidingWindow class
 
−
  int startOffset;
 
−
  int end Offset;
 
−
  double slideSpeed;
 
−
  void getSlidingWindowSize()
 
−
}
 
  
−
interface Scheduler{ // any class wants to do scheduling should implement this interface
+
Here are some performance metrics:
−
  void schedule() throws scheduleException; // the scheduling period is specified in the specific scheduling algorithm class
 
−
}
 
  
−
class scheduleException{
+
# Playback quality (PSNR)
 +
# Resilience to peer dynamics (ability of maintaning good streaming quality)
 +
# Required server capacity
  
−
}
 
  
−
class SchedulingResult{                // this class is used to record scheduling results and do some statistics,
+
= How to install uml and svn tools in eclipse3.2.x =
−
                                                      // for example, calculate the average delivery rate
 
−
  Tuple<int, int, double> log; //log file of the results. Each item of the tuple specifies the peerID, segmentID and
 
−
                                            // the starting time for the peer to transmit the segment
 
−
  double getDeliveryRatio();  //calculate the average delivery ratio
 
−
}
 
  
−
class SEFF implements Scheduler{ // the Serial Earliest Finish-time First scheduling algorithm
+
* Install emul2
−
  int schedulingPeriod;                    // scheduling period, for example, 2 seconds
+
*# Go to "http://www.soyatec.com/euml2/installation/", download its free edition. (Select the right version according to your eclipse)
−
  void schedule();                            //implement the Scheduler interface
+
*# Unpack the zip file
−
  SchedulingResult result;                // the scheduling results stored here
+
*# Open your eclipse, and go to Help -> Software Updates -> Find and install ... -> Search for new features to install -> New local site then find the unpakced file to install it.
 +
*# How to use: after you creating a new java project, go to File -> New -> Other, select the "UML2 Class Diagram" under the eUML directory, then you can create a class diagram for your project .
  
−
}
 
  
−
class LRF implements Scheduler{ // the Local Rarest First algorithm
+
* Install subclipse
−
  int schedulingPeriod;                  // scheduling period, for example, 2 seconds
+
*# Open your eclipse, and go to Help -> Software Updates -> Find and install ... -> Search for new features to install -> New  Remote Site
−
  void schedule();                          //implement the Scheduler interface
+
*# Add "http://subclipse.tigris.org/update_1.4.x" to the URL field and add an arbitrary name to the Name field
−
  SchedulingResult result;              // the scheduling results stored here
+
*# Follow the instruction to install it.
−
}
+
*# If you encounter an error like "Subclipse Integration for Mylyn 3.x (Optional) (3.0.0) requires plug-in "org.eclipse.mylyn.tasks.core (3.0.0)" don't worry, just deselect the "Integrations" item and continue to install.
 +
*# Go to Window -> Preferences -> Team. If you can see SVN under the Team tab, congratulations, your installation is done.
 +
*# How to use: right click the project you want to commit, select Tean -> Share project -> SVN, input the URL of your svn server, then you can import your project to it.
 +
*# For more details on how to use subclipse, please refer to http://www.ibm.com/developerworks/opensource/library/os-ecl-subversion/
  
  
 +
= The project directory =
  
−
class RndGen { // common random number/event generators, to deal with
+
I've put the psim project to "https://cs-svn.cs.surrey.sfu.ca/svn/nsl/schedule/psim". You can check it out in eclipse with subclipse.
−
                      // 1. event: add new receiver for a group/video
 
−
                      // 2. event: remove a receiver from a group/video
 
−
                      // 3. number: peer's bandwidth
 
−
}
 
−
 
 
−
class SimException {
 
−
 
 
−
}
 
−
 
 
−
Interface EventHandler {
 
−
    void handle() throws SimException; // callback function
 
−
}
 
−
 
 
−
class Event {
 
−
    long time; // when this event happens
 
−
    Peer src; // which peer generates this event
 
−
    Peer dst; // whom is destinate of this event
 
−
    Vector<EventHandler> handlers; //process an event and update states accordingly
 
−
    int type; // enum, see below
 
−
    // 1. connect: peer 1 makes a connection to peer 2, and add each other into neighbors
 
−
    // 2. requestSent: a receiver sends a high-priority request message to a sender
 
−
    // 3. requestArrived: a request message gets to a sender
 
−
    // 4. dataSent: a sender sends a data segment
 
−
    // 5. dataArrived: a data segment arrives to a receiver
 
−
    // 6. joinSent: a join message to the tracker
 
−
    // 7. joinArrived: a join message reaches the tracker
 
−
    // 8. responseSent: a response message sent by the tracker
 
−
    // 9. responseArrived: a response message reaches the receiver
 
−
 
 
−
    //10. scheduleStarted: a scheduling period started.
 
−
    // 11. scheduleEnded: a scheduling period ended.
 
−
 
 
−
    //commet: how should we do when a scheduling period expires?
 
−
    //There are two situations: 1. all scheduled segments arrived; 2. some segments are still in transmission.
 
−
 
 
−
    void addHandler(Interface EventHandler); // add an additional event handler
 
−
    void rmvHandler(Interface EventHandler); // remove an additional event handler
 
−
}
 
−
 
 
−
class EventQueue {
 
−
    SortedMap<Long, Event> queue; // events sorted on its time
 
−
    void dispatch(Event event); // sequentially invoke even handlers
 
−
}
 
−
 
 
−
class Simulator {
 
−
    long simTime; // elapsed simulation time in msec
 
−
    EventQueue events; // pending events that will be processed
 
−
    void dispatchEvent(); // forever loop to process the next event
 
−
}
 
−
 
 
−
 
 
−
 
 
−
Some comments:
 
−
  1. At the beginning, we should have a peer publish a video (or a trace file).
 
−
  2. The join() and leave() methods in the Group class may be moved to the Peer class and each peer
 
−
    should has a bitmap field.
 
−
  3. We may add a Tracker class:
 
−
    Class Tracker
 
−
    {
 
−
      URL url; //peers use this url to find the tracker
 
−
      Vector<Peer> peers;        //maintain a peer list to be quired by peers
 
−
      Vector<InetAddress> SelectPeers();  //when a peer joins a group at the first time, it quires other peers from the
 
−
                                                                  //tracker,  then the tracker invokes SelectPeers() and returns
 
−
                                                                //a list of neighbors to that peer
 
−
      void recordPeer(); //record information of a peer when it first connects the tracker or update it when some old
 
−
                                    //information is already stored
 
−
                                  //the information exchanged between a tracker and a peer may be <peerID, videoName>
 
−
                                  // or <peerID, videoName, ip_Address, port> if we want to do simulation on different machines
 
−
    }
 
−
  4. If we really want to send video files between peers, we may need a Storage class to operate with files
 
−
  5. We may design a superclass of all peers, trackers, and neighbors, as a typical networking server to deal with
 
−
    network connections
 
−
  6. In case of Network Coding when a peer joins a group it will receive segments that are 'n' seconds after the current 
 
−
    playback point. ('n' is a tunable parameter)
 
−
 
 
−
 
 
−
</pre>
 

Latest revision as of 22:19, 24 March 2009

This page documents the development of a discrete event simulator for P2P video streaming applications. This simulator captures important features of data-driven video streaming systems. In particular, it is designed to evaluate: (i) the performance of various segment scheduling algorithms; (ii) the potential of network coding in multi-layer P2P video streaming systems.

Todo List

We keep a short todo list here. For each todo item, we also list the expected workload, the current assignee, the tentative due date, and the completion time.

Description Workload (ppl-day) Assignee Tentative Duedate Completion time
Reread and implement Meng Zhang's algorithm 1.5 Cheng March 18, 2009 March 18, 2009
Finalize the simulator engine 1 Cheng March 17, 2009 March 17, 2009
Generalize the scheduling algorithm for multiple layer problem 3 N/A
Reorganize the evaluation setup subsection 2 Yuanbin March 18, 2009 March 18, 2009

High-Level Design

We present the high level design of the pSim. Please find a coarse class diagram below, and the source file can be downloaded here.

640px


Some misc notes:

  • We use long for time/offset in msec, which has a rollover time 24.85 days

System Architecture

The core of the simulator is a simulation engine that is abstracted as the Simulator class. Since pSim is a discrete event simulator, the Simulator class keeps track of a moving simulation time simTime, which is different from the wall clock (real-time). Simulator consists of an EventQueue that provides the interface of push and pop an event from a priority queue of the pending events. The pending events are sorted on the event time in the ascending order. Simulator works as the follows, in dispatchEvents method, the next event is popped from EventQueue. Simulator then update simTime and calls dispatch method to process that event using method dispatch, and then drop the processed event away. Simulator then popped the next event from EventQueue and update simTime. That is, the method dispatchEvent is a forever loop, thus the caller must not expect dispatchEvent to return.

Other components in pSim add new events to Simulator by calling its push method. The new events are pushed through EventQueue to CalendarQueue. CalendarQueue is an efficient implementation of priority queue, which achieves constant time complexity. We implement the Dynamic Calendar Queue (DCQ) based on a Berkeley implementation, which sort the pending events on their time. We notice that we keep EventQueue between Simulator and CalendarQueue, in case we need to switch to other priority queue implementation such as standard TreeMap.

In addition to EventQueue, Simulator also consists of: (i) Logger that is responsible for filtering and saving logging messages, (ii) RndGen that is the system-wide random number generator, and (iii) SysParam that has all system parameters.

Event and Event Handler

Since Simulator class is driven by its priority queue of events, every Event must consist of enough information on how to process that Event. The main fields of Event include: type that is the event type, and time that is the event time. Moreover, the Event class maintains a vector of EventHandler interface, which specifies how to process that Event'. There is only an abstract method defined in EventHandler, which is its handle function. The handle function returns a log entry in String format. We note that this String must use pipe as the field delimiter, and should not contain any CR nor LF. See existing code for other convention of the log entries. Notice that, each Event instance has no EventHandler initially: the class who creates an Event must provide one or more EventHandler by calling the method addHandler. Last, we mention that the dispatch method of Simulator retrieves all EventHandler by calling method getHandlers.

Event has two subclasses: PeerEvent is for events that is local to a peer, e.g., a peer joins a video group is a PeerEvent; NetworkEvent is for sending messages between two peers, e.g., a receiver sends a CONNECTMSG to a sender. PeerEvent has two fields: Peer and Video. NetworkEvent has four fields: (i) conn that captures the connection status between two peers, (ii) length that describes the length of this message, (iii) direction that indicates whether the message is sent from receiver to sender or not, and (iv) arrived shows whether the message has arrived at the destination. More precisely, a NetworkEvent is created with arrived value set to off, which means that event just enter the connection conn. Following the bandwidth defined in conn, we can compute when will the event arrives at the destination. This computation is done in tHandler that stands for transmission handler. The transmission handler processes a NetworkEvent as follows. First, it computes and updates the event time to be the time that event arrives at the destination. Second, it sets arrived to be true. Third, it pushes the same NetworkEvent back to EventQueue.


Random Number Generator and Profiles

TODO


Performance metrics for scheduling

Here are some performance metrics:

  1. Average delivery ratio: number of on-time scheduled segments over total number of segments to be scheduled
  2. Load balance among senders
  3. Initial buffering time
  4. Time and space complexity of the scheduling algorithm


Performance metrics for network coding

Here are some performance metrics:

  1. Playback quality (PSNR)
  2. Resilience to peer dynamics (ability of maintaning good streaming quality)
  3. Required server capacity


How to install uml and svn tools in eclipse3.2.x

  • Install emul2
    1. Go to "http://www.soyatec.com/euml2/installation/", download its free edition. (Select the right version according to your eclipse)
    2. Unpack the zip file
    3. Open your eclipse, and go to Help -> Software Updates -> Find and install ... -> Search for new features to install -> New local site then find the unpakced file to install it.
    4. How to use: after you creating a new java project, go to File -> New -> Other, select the "UML2 Class Diagram" under the eUML directory, then you can create a class diagram for your project .


  • Install subclipse
    1. Open your eclipse, and go to Help -> Software Updates -> Find and install ... -> Search for new features to install -> New Remote Site
    2. Add "http://subclipse.tigris.org/update_1.4.x" to the URL field and add an arbitrary name to the Name field
    3. Follow the instruction to install it.
    4. If you encounter an error like "Subclipse Integration for Mylyn 3.x (Optional) (3.0.0) requires plug-in "org.eclipse.mylyn.tasks.core (3.0.0)" don't worry, just deselect the "Integrations" item and continue to install.
    5. Go to Window -> Preferences -> Team. If you can see SVN under the Team tab, congratulations, your installation is done.
    6. How to use: right click the project you want to commit, select Tean -> Share project -> SVN, input the URL of your svn server, then you can import your project to it.
    7. For more details on how to use subclipse, please refer to http://www.ibm.com/developerworks/opensource/library/os-ecl-subversion/


The project directory

I've put the psim project to "https://cs-svn.cs.surrey.sfu.ca/svn/nsl/schedule/psim". You can check it out in eclipse with subclipse.

Retrieved from "https://nmsl.cs.sfu.ca/index.php?title=Private:psim&oldid=2660"
Powered by MediaWiki