Datacenter and ISP Networks
XXX Datacenter and ISP Networking ... brief intro
Our current problems include task scheduling, datacenter multicast, ISP multicast, and load balancing.
Recent large-scale Internet applications have introduced a renewed interest in scalable multicast services. Examples of such applications include live Internet broadcast (e.g., Facebook Live), stock market applications, cloud-based enterprise applications, IPTV, webinars and video conferencing, and massive multiplayer games. The scale of these applications is unprecedented. For instance, Facebook Live aims to stream millions of live sessions to millions of concurrent users. Traditional multicast approaches do not scale, because of their substantial state and communication overheads. In addition, most traditional multicast approaches do not enable controlling the network paths chosen for the multicast sessions (usually referred to as traffic engineering), nor do they offer efficient methods to direct multicast traffic through various network functions (e.g., firewalls, IDS, video transcoding), which is known as service chaining.
In this project, we design scalable multicast systems for general ISP networks and datacenter networks.
People
- Khaled Diab
- Parham Yassini
- Saeed Zangeneh
- Erfan Mehraban
- Mohamed Hefeeda
Horus: In-Network Task Scheduling for Datacenters
Introducing Horus, an in-network task scheduler for datacenters that efficiently assigns short-lived tasks across distributed workers. It minimizes state and communication overhead, dynamically adapts to workloads, and does not buffer tasks, enabling high throughput and reduced tail response times. Compared to traditional rack-confined schedulers, Horus' distributed approach leverages parallel scheduling at line rate across switches. Testing shows Horus scales superbly in large datacenters, outperforming current in-network schedulers in both speed and capacity.
Stateless Multicast For ISP Networks
Current multicast forwarding systems suffer from large state requirements at routers and high communication overhead. In addition, these systems do not support generalized multicast forwarding, where traffic passes through traffic-engineered paths or requires service chaining. We propose a new system that completely eliminates the state at routers and reduces communication overhead.
Scalable Multicast for Datacenter Networks
Group communications appear in many recent datacenter applications. These applications, however, do not benefit from multicast due to the absence of efficient systems that support large-scale multicast sessions, minimize state at switches and reduce bandwidth overheads. We propose a new architecture that addresses the challenges of multicast in datacenter networks. The proposed approach carefully divides the state and tasks of the data plane among switches and servers in order to realize efficient multicast services in datacenters, by partially offloading the management of multicast sessions to servers.
Oktopus: Service Chaining for Multicast Traffic
Multicast service chaining refers to the orchestration of network services for multicast traffic. Paths of a multicast session that span the source, destinations and required services form a complex structure that we refer to as the multicast distribution graph. We propose a new path-based algorithm, called Oktopus, that runs at the control plane of the ISP network to calculate the multicast distribution graph for a given session. Oktopus aims at minimizing the routing cost for each multicast session while satisfying all service chaining requirements. Oktopus consists of two steps. The first one generates a set of segments from the given ISP network topology, and the second step uses these segments to efficiently calculate the multicast distribution graph. Oktopus has a fine-grained control over the selection of links in the distribution graphs that leads to significant improvements. The code of Oktopus is open source.
Traffic-Engineered Distribution of Adaptive Multimedia Content in Telco-CDNs
Telco-CDNs refer to content distribution networks deployed and managed by Internet Service Providers (ISPs). They are getting popular among major ISPs because they offer new revenue streams and have the potential of providing better performance compared to traditional CDNs. Managing telco-CDNs is, however, a complex problem, because it requires jointly managing the network resources (links and switches) and the caching resources (processing and storage capacities), while supporting the adaptive nature and skewed popularity of multimedia content. To address this problem, we present a new algorithm called CAD (Cooperative Active Distribution). CAD strives to serve as much as possible of the requested multimedia objects within the ISP while carefully engineering the traffic paths through the network. This is achieved by enabling the cooperation among caches within the ISP not only to serve various representations of multimedia objects, but also to create them on demand using the often under-utilized processing capacity of caches.
Code and Datasets
Publications
- K. Diab, P. Yassini, and M. Hefeeda, Orca: Server-assisted Multicast for Datacenter Networks, In Proc. of USENIX Networked Systems Designed and Implementation (NSDI'22), Renton, WA, April 2022.
- K. Diab and M. Hefeeda, Yeti: Stateless and Generalized Multicast Forwarding, In Proc. of USENIX Networked Systems Designed and Implementation (NSDI'22), Renton, WA, April 2022.
- K. Diab, C. Lee, and M. Hefeeda, Oktopus: Service Chaining for Multicast Traffic, IEEE International Conference on Network Protocols (ICNP'20), Madrid, Spain, October 2020.
- K. Diab and M. Hefeeda, Joint Content Distribution and Traffic Engineering for Adaptive Videos in Telco-CDNs, In Proc. of IEEE INFOCOM’19, Paris, France, April 2019.
