Current Internet is composed of heterogeneous multi-domain multi-AS (autonomous system) networks (wireless, optical networks etc.) as shown in Fig. 1. Wherein, 5G and optical passive networking (PON) have been recognized as the building blocks for next-generation access networks, while elastic optical networking (EON) is emerging as one of most promising technologies for future backbone networks due to its fine-grained and agile spectrum allocation schemes. With the rapid development of datacenter networks and the explosion of cloud-driven applications, future Internet is expected to be able to support high-capacity and quality-of-transmission aware end-to-end services across multiple domains. Due to the heterogeneity and autonomy of these domains, how to design a multi-domain network control and management system that can realize efficient service provisioning in the Internet with cross-stratum resource optimization is critical.
Fig. 1. Internet Infrastructure .
The service provisioning in multi-domain SD-EONs can rely on either flat (the domain managers work collaboratively to exchange intra-domain information and set up inter-domain lightpaths in a distributed manner) or hierarchical (a higher-level multi-domain orchestrator coordinates the operations of different domains and the inter-domain lightpaths are set up in a semi-centralized way) control plane designs. While the flat design suffers from the issues of low resource efficiency and prolonged service latency, the hierarchical design violates the autonomy of each administrative domain and also has the survivability issue due to having a single decision-maker in the multi-domain SD-EON. Therefore, we propose to address these issues with a multi-broker based design  as shown in Fig. 1. Specifically, we allow multiple brokers residing in the broker plane, each of which interacts with domain managers using certain inter-domain communication protocol to collect domain topology and resource abstractions and calculate inter-domain service schemes with them. Each domain manager can subscribe to multiple brokers for inter-domain services and submit different intra-domain information to them based on the signed service level agreements (SLAs). Here, the brokers generally are managed by third-party entities and participate in the multi-domain service provisioning due to market-driven incentives (e.g., reputation, revenue gain etc.).
1. The autonomy of each administrative domain should be respected. This implies that a fully centralized architecture with a multi-domain software-defined networking (SDN) controller managing the holistic multi-domain network is infeasible. 2. The global multi-domain traffic engineering information is not accessible. Domain managers tend to protect the privacy of their domains by exposing only limited information, and how to realize efficient cross-domain resource allocation with such limited intra-domain information is critical, especially when the multi-domain network is highly dynamic and evolves quickly. 3. The multi-domain network control and management system should be scalable and survivable, i.e., being feasible for large scale networks, easy to upgrade and resilient to data and control plane failures.
1. Network Architectural Design: This project aims to design a powerful multi-domain network control and management framework that can address the aforementioned challenges. Fig. 2 shows the designed multi-broker based multi-domain architecture [2-5]. Specifically, we allow multiple brokers residing in the broker plane, each of which interacts with domain managers using certain inter-domain communication protocol to collect domain topology and resource abstractions and calculate inter-domain service schemes with them. Each domain manager can subscribe to multiple brokers for inter-domain services and submit different intra-domain information to them based on the signed service level agreements (SLAs). Here, the brokers generally are managed by third-party entities and participate in the multi-domain service provisioning due to market-driven incentives (e.g., reputation, revenue gain etc.). To address the challenges of limited intra-domain information and dynamic network scenario, we bring in big data analytics to enable brokers cognizing in depth the network behaviors and performing knowledge-based autonomous service provisioning. The database of brokers store all the traffic engineering and performance monitoring data received from DMs and each of the autonomous service provisioning applications is developed based on a certain data analytics scheme for a specific management function. For instance, with the bit error rate (BER) and optical signal to noise ratio (OSNR) monitoring data collected from large numbers of inter-domain lightpaths, brokers can estimate the QoT of unestablished lightpaths and conduct anomaly detection. While accurate QoT estimation assists brokers in provisioning lightpaths with guaranteed QoT and reduced margins, anomaly detection is able to generate alarms to inform DMs of lightpath reconfigurations when anomalous behaviors (e.g., QoT degradation induced by component failures or physical layer attacks) are identified. Consequently, the proposed architecture enables the broker plane (as well as DMs) to operate according to the observe-analyze-act cycle, i.e., observing the network status using advanced data monitoring and abstraction technologies, analyzing the observed data to learn the knowledge about the network behaviors and acting based on the obtained knowledge, such that it can adapt to the network changes and autonomously (re)optimize the multi-domain service provisioning decisions.
Fig.2 Design multi-broker based multi-domain architecture. Left: system function design, right: resource (optical spectrum) allocation principle.
2. Service Provisioning Policy Design: The key problem of designing inter-domain service provisioning policies based on the proposed architecture lies in 1) how to make domain managers interact with the incentive-driven brokers (i.e., which brokers to rely on for building the inter-domain lightpaths, how much intra-domain information to expose to the brokers etc.), 2) how should the brokers behave in the multi-domain service provisioning market (whether a broker should compete or cooperate with other brokers, how should be brokers bid for their services etc.) [3-5] and 3) how to monitor the multi-domain network and learn its behaviors to realize knowledge-based service provisioning. We are investigating both game-theoretic and machine learning approaches for the problem. For instance, Fig. 3 shows our design on multi-domain traffic estimator which can assist broker make intelligent decisions in avoiding future congestions . Fig. 4 shows the performance of the proposed method.
Fig. 3. Design of multi-domain traffic estimator.
Fig. 4. Performances of the traffic estimator based service provisioning approach.
3. System design and implementation: This project also aims to design and implement a practical SDN-based control plane system for the proposed multi-broker based service provisioning paradigm. We are working on the designing of the system function modules, work-flow, OpenFlow agent, OpenFlow controller, OpenFlow protocol extension (for supporting the EON parameters) and unified APIs (i.e., the YANG model, for interoperability between heterogeneous systems and network layers) ,. For instance, we have experimentally demonstrated software-defined heterogeneous wireline-wireless-optical multi-domain networks connecting UC Davis Campus, USTC, California OpenFlow Testbed Network (COTN) and Energy Sciences Network (ESNet)  as shown in Fig. 5.
Fig. 5. Our field test experimental testbed.
 https://esdn.upc.edu/en  "Multi-domain Cognitive Optical Software Defined Networks with Market-Driven Brokers", S. J. B. Yoo, European Conference and Exhibition on Optical Communication (ECOC), 2014.  "Incentive-Driven Bidding Strategy for Brokers to Compete for Service Provisioning Tasks in Multi-Domain SD-EONs", Xiaoliang Chen, Zuqing Zhu, Lu Sun, Jie Yin, Shilin Zhu, Alberto Castro, S. J. Ben Yoo, IEEE JLT, 2016.  "Broker-based Cooperative Game in Multi-Domain SD-EONs: Nash Bargaining for Agreement on Market-Share Partition", Lu Sun, Xiaoliang Chen, Shilin Zhu, Zuqing Zhu, Alberto Castro, S. J. Ben Yoo, European Conference on Optical Communications (ECOC), 2016.  "On Efficient Incentive-Driven VNF Service Chain Provisioning with Mixed-Strategy Gaming in Broker-based EO-IDCNs", Xiaoliang Chen, Lu Sun, Zuqing Zhu, Hongbo Lu and S. J. Ben Yoo, Optical Fiber Communication Conference (OFC), 2017.  "Leveraging Deep Learning to Achieve Knowledge-based Autonomous Service Provisioning in Broker-based Multi-Domain SD-EONs with Proactive and Intelligent Predictions of Multi-Domain Traffic", X. Chen, J. Guo, Z. Zhu, A. Castro, R. Proietti, H. Lu, M. Shamsabardeh and S. J. B. Yoo, European Conference on Optical Communications (ECOC), 2017.  "Brokered Orchestration for End-to-End Service Provisioning across Heterogeneous Multi-Operator (Multi-AS) Optical Networks", Alberto Castro, Luis Velasco, Lluis Gifre, Cen Chen, Jie Yin, Zuqing Zhu, Roberto Proietti, and S. J. Ben Yoo, IEEE JLT, 2016.  "Field trial of broker-based multi-domain software-defined heterogeneous wireline-wireless-optical networks", Lei Liu, Zuqing Zhu, Xiong Wang, Guanghua Song, Cen Chen, Xiaoliang Chen, Shoujiang Ma, Xiaotao Feng, Roberto Proietti, and S. J. B. Yoo, Optical Fiber Communication Conference (OFC), 2015