PCE Working Group H. Chen Internet-Draft Huawei Technologies Intended status: Standards Track M. Toy Expires: November 5, 2017 Verizon L. Liu Fujitsu V. Liu China Mobile May 4, 2017 PCE Hierarchical SDNs draft-chen-pce-h-sdns-02 Abstract This document presents extensions to the Path Computation Element Communication Protocol (PCEP) for supporting a hierarchical SDN control system, which comprises multiple SDN controllers controlling a network with a number of domains. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 5, 2017. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect Chen, et al. Expires November 5, 2017 [Page 1] Internet-Draft PCE-H-SDNs May 2017 to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Conventions Used in This Document . . . . . . . . . . . . . . 6 4. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. Overview of Hierarchical SDN Control System . . . . . . . . . 6 6. Extensions to PCEP . . . . . . . . . . . . . . . . . . . . . . 9 6.1. Capability Discovery . . . . . . . . . . . . . . . . . . . 9 6.2. New Messages for Hierarchical SDN Control System . . . . . 10 6.2.1. Contents of Messages . . . . . . . . . . . . . . . . . 12 6.2.2. Individual Encoding of Messages . . . . . . . . . . . 24 6.2.3. Group Encoding of Messages . . . . . . . . . . . . . . 25 6.2.4. Embedded Encoding of Messages . . . . . . . . . . . . 26 6.2.5. Mixed Encoding of Messages . . . . . . . . . . . . . . 27 6.3. Controller Relation Discovery . . . . . . . . . . . . . . 27 6.3.1. Using Open Message . . . . . . . . . . . . . . . . . . 27 6.3.2. Using Discovery Message . . . . . . . . . . . . . . . 29 6.4. Connections and Accesses Advertisement . . . . . . . . . . 30 6.5. Tunnel Creation . . . . . . . . . . . . . . . . . . . . . 30 6.5.1. Computing Path in Two Rounds . . . . . . . . . . . . . 31 6.5.2. Computing Path in One Round . . . . . . . . . . . . . 32 6.5.3. Creating Tunnel along Path . . . . . . . . . . . . . . 34 6.6. Objects and TLVs . . . . . . . . . . . . . . . . . . . . . 36 6.6.1. CRP Objects . . . . . . . . . . . . . . . . . . . . . 36 6.6.2. LOCAL-CONTROLLER Object . . . . . . . . . . . . . . . 37 6.6.3. REMOTE-CONTROLLER Object . . . . . . . . . . . . . . . 38 6.6.4. CONNECTION and ACCESS Object . . . . . . . . . . . . . 40 6.6.5. NODE Object . . . . . . . . . . . . . . . . . . . . . 47 6.6.6. TUNNEL Object . . . . . . . . . . . . . . . . . . . . 53 6.6.7. STATUS Object . . . . . . . . . . . . . . . . . . . . 54 6.6.8. LABEL Object . . . . . . . . . . . . . . . . . . . . . 54 6.6.9. INTERFACE Object . . . . . . . . . . . . . . . . . . . 55 7. Security Considerations . . . . . . . . . . . . . . . . . . . 56 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 56 9. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 56 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 56 10.1. Normative References . . . . . . . . . . . . . . . . . . . 56 10.2. Informative References . . . . . . . . . . . . . . . . . . 57 Appendix A. Details on Embedded Encoding of Messages . . . . . . 58 A.1. Message for Controller Relation Discovery . . . . . . . . 58 A.2. Message for Connections and Accesses Advertisement . . . . 60 A.3. Request for Computing Path Segments . . . . . . . . . . . 60 Chen, et al. Expires November 5, 2017 [Page 2] Internet-Draft PCE-H-SDNs May 2017 A.4. Reply for Computing Path Segments . . . . . . . . . . . . 61 A.5. Request for Removing Path Segments . . . . . . . . . . . . 61 A.6. Reply for Removing Path Segments . . . . . . . . . . . . . 62 A.7. Request for Keeping Path Segments . . . . . . . . . . . . 62 A.8. Reply for Keeping Path Segments . . . . . . . . . . . . . 63 A.9. Request for Creating Tunnel Segment . . . . . . . . . . . 63 A.10. Reply for Creating Tunnel Segment . . . . . . . . . . . . 64 A.11. Request for Removing Tunnel Segment . . . . . . . . . . . 64 A.12. Reply for Removing Tunnel Segment . . . . . . . . . . . . 65 Chen, et al. Expires November 5, 2017 [Page 3] Internet-Draft PCE-H-SDNs May 2017 1. Introduction A domain is a collection of network elements within a common sphere of address management or routing procedure which are operated by a single organization or administrative authority. Examples of such domains include IGP (OSPF or IS-IS) areas and Autonomous Systems. For scalability, security, interoperability and manageability, a big network is organized as a number of domains. For example, a big network running OSPF as routing protocol is organized as a number of OSPF areas. A network running BGP is organized as multiple Autonomous Systems, each of which has a number of IGP areas. The concepts of Software Defined Networks (SDN) have been shown to reduce the overall network CapEx and OpEx, whilst facilitating the deployment of services and enabling new features. The core principles of SDN include: centralized control to allow optimized usage of network resources and provisioning of network elements across domains. For a network with a number of domains, it is natural to have multiple SDN controllers, each of which controls a domain in the network. To achieve a centralized control on the network, a hierarchical architecture of controllers is a good fit. At top level of the hierarchy, it is a parent controller that is not a child controller. The parent controller controls a number of child controllers. Some of these child controllers are not parent controllers. Each of them controls a domain. Some other child controllers are also parent controllers, each of which controls multiple child controllers, and so on. This document presents extensions to the Path Computation Element Communication Protocol (PCEP) for supporting a hierarchical SDN control system, which comprises multiple SDN controllers controlling a network with a number of domains. 2. Terminology The following terminology is used in this document. ABR: Area Border Router. Router used to connect two IGP areas (Areas in OSPF or levels in IS-IS). ASBR: Autonomous System Border Router. Router used to connect together ASes of the same or different service providers via one or more inter-AS links. Chen, et al. Expires November 5, 2017 [Page 4] Internet-Draft PCE-H-SDNs May 2017 BN: Boundary Node. A boundary node is either an ABR in the context of inter-area Traffic Engineering or an ASBR in the context of inter-AS Traffic Engineering. A Boundary Node is also called an Edge Node. Entry BN of domain(n): a BN connecting domain(n-1) to domain(n) along the path found from the source node to the BN, where domain(n-1) is the previous hop (or upstream) domain of domain(n). An Entry BN is also called an in-BN or in-edge node. Exit BN of domain(n): a BN connecting domain(n) to domain(n+1) along the path found from the source node to the BN, where domain(n+1) is the next hop (or downstream) domain of domain(n). An Exit BN is also called a out-BN or out-edge node. Source Domain: For a tunnel from a source to a destination, the domain containing the source is the source domain for the tunnel. Destination Domain: For a tunnel from a source to a destination, the domain containing the destination is the destination domain for the tunnel. Source Controller: A controller controlling the source domain. Destination Controller: A controller controlling the destination domain. Parent Controller: A parent controller is a controller that communicates with a number of child controllers and controls a network with multiple domains through the child controllers. A PCE can be enhanced to be a parent controller. Child Controller: A child controller is a controller that communicates with one parent controller and controls a domain in a network. A PCE can be enhanced to be a child controller. Exception list: An exception list for a domain contains the nodes in the domain and its adjacent domains that are on the shortest path tree (SPT) that the parent controller is building. GTID: Global Tunnel Identifier. It is used to identify a tunnel in a network. PID: Path Identifier. It is used to identify a path for a tunnel in a network. Chen, et al. Expires November 5, 2017 [Page 5] Internet-Draft PCE-H-SDNs May 2017 Inter-area TE LSP: a TE LSP that crosses an IGP area boundary. Inter-AS TE LSP: a TE LSP that crosses an AS boundary. LSP: Label Switched Path LSR: Label Switching Router PCC: Path Computation Client. Any client application requesting a path computation to be performed by a Path Computation Element. PCE: Path Computation Element. An entity (component, application, or network node) that is capable of computing a network path or route based on a network graph and applying computational constraints. PCE(i): a PCE with the scope of domain(i). TED: Traffic Engineering Database. This document uses terminology defined in [RFC5440]. 3. Conventions Used in This Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 4. Requirements This section summarizes the requirements for Hierarchical SDN Control System (need more text here). 5. Overview of Hierarchical SDN Control System The Figure below illustrates a hierarchical SDN control system. There is one Parent Controller and four Child Controllers: Child Controller 1, Child Controller 2, Child Controller 3 and Child Controller 4. Chen, et al. Expires November 5, 2017 [Page 6] Internet-Draft PCE-H-SDNs May 2017 +-------------------+ | Parent Controller | +--+---------+----+-+ _/| \ \____ _/ | \ \____ _/ | \ \__ __/ | +---------+---------+ \ __/ | |Child Controller 3 | | / | +-------------------+ | +---------+---------+ | / \ | |Child Controller 1 | | .---. .---,\ | +-------------------+ | ( ' ') | / \ | ( Domain 3 ) | .---. .---,\ | ( ) +---------+---------+ ( ' ') | '-o-.--o) |Child Controller 4 | ( Domain 1 ) | | +-------------------+ ( ) | | / \____ '-o-.---) +--------+----------+ \ / \ \____ | |Child Controller 2 | \ /\ .---. .---+ \ | +-------------------+ \ | \( ' |'.---. | | / \____ \_ |---\ Domain 4 | '+, \ / \ \____ (o \ | | ) \ /\ .---. .---+ \ ( | | o) \ | \( ' |'.---. | ( | | ) \ |---\ Domain 2 | '+. ( o o .-' \____(o \ | | ) ' ) ( | | o)-------o---._.-.-----) ( | | ) ( o o .-' ' ) '---._.-.-----) The parent controller communicates with these four child controllers and controls them, each of which controls (or is responsible for) a domain. Child controller 1 controls domain 1, Child controller 2 controls domain 2, Child controller 3 controls domain 3, and Child controller 4 controls domain 4. One level of hierarchy of controllers is illustrated in the figure above. There is one parent controller at top level, which is not a child controller. Under the parent controller, there are four child controllers, which are not parent controllers. In a general case, at top level there is one parent controller that is not a child controller, there are some controllers that are both parent controllers and child controllers, and there are a number of child controllers that are not parent controllers. This is a system Chen, et al. Expires November 5, 2017 [Page 7] Internet-Draft PCE-H-SDNs May 2017 of multiple levels of hierarchies, in which one parent controller controls or communicates with a first number of child controllers, some of which are also parent controllers, each of which controls or communicates with a second number of child controllers, and so on. Considering one parent controller and its child controllers, each of the child controllers controls a domain and has the topology information on the domain, the parent controller does not have the topology information on any domain controlled by a child controller normally. This is called parent without domain topology. In some special cases, the parent controller has the topology information on a region consisting of the domains controlled by its child controllers. In other words, the parent controller has the topology information on the domains controlled by its child controllers and the topology/inter-connections among these domains. This is called parent with domain topology. The parent controller receives requests for creating end to end tunnels from users or applications. For each request, the parent controller is responsible for obtaining a path for the tunnel and creating the tunnel along the path. For parent without domain topology, the parent controller asks each of its related child controllers to compute path segments from an entry boundary node to exit boundary nodes in the domain it controls or path segments from an exit boundary node in its domain to entry boundary nodes of other adjacent domains just using the inter-domain links attached to the exit boundary node. The details of the segments are hidden from the parent, which sees each of the segments as a link from a boundary node to another boundary node with a cost. The parent controller builds a shortest path tree (SPT) using the path segments computed as links to get the end to end path and then creates the tunnel along the path by asking its related child controllers. The end to end path does not have any details from the parent's point of view. It can be considered as a sequence of domains containing the shortest path. Along this sequence of domains, the details of the end to end path can be obtained. And then the tunnel along the path with details can be created. For parent with domain topology, the parent controller computes a path for the tunnel using the topology information on the domains controlled by its child controllers. And then it creates the tunnel along the path computed through asking its related child controllers. Chen, et al. Expires November 5, 2017 [Page 8] Internet-Draft PCE-H-SDNs May 2017 6. Extensions to PCEP This section describes the extensions to PCEP for a Hierarchical SDN Control System (HSCS). The extensions include the definition of a new flag in the RP object, a global tunnel identifier (GTID), a path identifier (PID), a list of path segments and an exception list in the PCReq and PCRep message. 6.1. Capability Discovery During a PCEP session establishment between two PCEP speakers (PCE or PCC), each of them advertises its capabilities for HSCS through the Open Message with the Open Object containing a new TLV to indicate its capabilities for HSCS. This new TLV is called HSCS capability TLV. It has the following format. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = TBD1 | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Capability Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | (Optional Sub-TLVs) | ~ ~ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The type of the TLV is TBD1. It has a length of 4 octets plus the size of optional Sub-TLVs. The value of the TLV comprises a capability flags field of 32 bits, which are numbered from the most significant as bit zero. Each bit represents a capability. o PC (Parent Controller - 1 bit): Bit 0 is used as PC flag. It is set to 1 indicating a parent controller. o CC (Child Controller - 1 bit): Bit 1 is used as PC flag. It is set to 1 indicating a child controller. o PS (Path Segments - 1 bit): Bit 2 is used as PS flag. It is set to 1 indicating support for computing path segments for HSCS o TS (Tunnel Segment - 1 bit): Bit 3 is used as TS flag. It is set to 1 indicating support for creating tunnel segment for HSCS Chen, et al. Expires November 5, 2017 [Page 9] Internet-Draft PCE-H-SDNs May 2017 o ET (End to end Tunnel - 1 bit): Bit 4 is used as ET flag. It is set to 1 indicating support for creation and maintenance of end to end LSP tunnels 6.2. New Messages for Hierarchical SDN Control System This section describes the contents and semantics of the new messages, and presents a few of different encodings for the messages. There are a number of new messages for supporting HSCS. These new messages can be encoded in a few of ways as follows: o To use a new type at top level for each of the new messages. This is called individual encoding. o To use a new type at top level for each group of the new messages and a option/operation/sub-type value for every message in the group. This is called group encoding. o To use/re-use existing messages and a value of options/operations for each new message in an existing message. This is called embedded encoding. o To combine the ways above. This is called mixed encoding. Various types of messages for supporting HSCS are listed below. Note that many new messages may not be needed for some procedures/options. For example, four messages Request and Reply for Removing Path Segments and Request and Reply for Keeping Path Segments are not needed if path segments computed are not stored/remembered by a child controller. But in this case, the path segment in each domain along the end to end path computed needs to be re-computed when a tunnel along the path is set up. Message for Controller Relation Discovery: It is a message exchanged between a parent controller and a child controller for discovering their parent-child relation. Message for Connections and Accesses Advertisement: It is a message that a child controller sends its parent controller to describe the connections from the domain it controls to its adjacent domains and the access points in the domain to be accessible outside of the domain. Request for Computing Path Segments: It is a message that a parent controller sends a child controller to request the child controller for computing path segments in the domain the child controller controls. Chen, et al. Expires November 5, 2017 [Page 10] Internet-Draft PCE-H-SDNs May 2017 Reply for Computing Path Segments: It is a message that a child controller sends a parent controller to reply the parent controller for a request message for computing path segments after receiving the request message from the parent controller for computing path segments and computing path segments as requested, which normally contains the path segments computed. Request for Removing Path Segments: It is a message that a parent controller sends a child controller to request the child controller for removing the path segments computed by the child controller and stored in the child controller. Reply for Removing Path Segments: It is a message that a child controller sends a parent controller to reply the parent controller for a request message for removing a set of path segments after receiving the request message from the parent controller for removing path segments and removing the path segments as requested, which normally contains a status of removing path segments. Request for Keeping Path Segments: It is a message that a parent controller sends a child controller to request the child controller for keeping a set of path segments computed by the child controller and stored in the child controller. Reply for Keeping Path Segments: It is a message that a child controller sends a parent controller to reply the parent controller for a request message for keeping path segments after receiving the request message from the parent controller for keeping path segments and keeping the path segments as requested, which normally contains a status of keeping path segments. Request for Creating Tunnel Segment: It is a message that a parent controller sends a child controller to request the child controller for creating tunnel segments related to the domain the child controller controls. Reply for Creating Tunnel Segment: It is a message that a child controller sends a parent controller to reply the parent controller for a request message for creating tunnel segment after receiving the request message from the parent controller for creating tunnel segment and creating tunnel segment as requested, which normally contains a status of creating tunnel segment and a label and an interface. Chen, et al. Expires November 5, 2017 [Page 11] Internet-Draft PCE-H-SDNs May 2017 Request for Removing Tunnel Segment: It is a message that a parent controller sends a child controller to request the child controller for removing the tunnel segment created by the child controller. Reply for Removing Tunnel Segment: It is a message that a child controller sends a parent controller to reply the parent controller for a request message for removing tunnel segment after receiving the request message from the parent controller for removing tunnel segment and removing the tunnel segment as requested, which normally contains a status of removing tunnel segment. 6.2.1. Contents of Messages This section describes the contents in each of the messages and gives the format of each of messages in individual encoding, which is the same as in group encoding. Some of the objects in the messages are defined in the following sections. 6.2.1.1. Message for Controller Relation Discovery A message for controller relation discovery is exchanged between a parent controller and a child controller for discovering their parent-child relation. A message for controller relation discovery (CRDis message for short) sent from a local controller to a remote controller comprises: o Local controller attributes o Remote controller attributes after the local controller receives the remote controller attributes from a remote end and determines that the relation between the local controller and the remote controller can be formed. The format of the CRDis message is as follows: ::= [] where CRP (Controller Request Parameters) object is defined in section Objects and TLVs. Chen, et al. Expires November 5, 2017 [Page 12] Internet-Draft PCE-H-SDNs May 2017 6.2.1.2. Message for Connections and Accesses Advertisement After a child controller discovers its parent controller, it sends its parent controller a message for connections and accesses advertisement. A message for connections and accesses advertisement (CAAdv message for short) from a child controller comprises: o Inter-domain links from the domain the child controller controls to its adjacent domains. o The addresses in the domain to be accessible to the outside of the domain. o Attributes of each of the boundary nodes of the domain. The format of the CAAdv message is as follows: ::= [] where: ::= [] ::= [] 6.2.1.3. Request for Computing Path Segments After receiving a request for creating an end to end tunnel from source A to destination Z for a given set of constraints, a parent controller allocates a global tunnel identifier (GTID) for the end to end tunnel crossing domains and a path identifier (PID) for an end to end path to be computed for the tunnel. The parent controller sends a request message to each of its related child controllers for computing a set of path segments in the domain the child controller controls in a special order. The parent controller builds a shortest path tree (SPT) using these path segments and obtains a shortest path from source A to destination Z that satisfies the constraints. Note: The details of the path segments are hidden from the parent, which sees each of the segments as a link from one (boundary) node to another (boundary) node with a cost. The end to end path does not have any details from the parent's point of view, which may be considered as a domain path. Chen, et al. Expires November 5, 2017 [Page 13] Internet-Draft PCE-H-SDNs May 2017 A request message for computing path segments (PSReq message for short) from a parent controller to a child controller comprises: o The address or identifier of the start-node (saying X) in the domain controlled by the child controller. From this node, a number of path segments are to be computed. o The global tunnel identifier (GTID) and the path identifier (PID). For the path of the tunnel, a number of path segments are to be computed. o An exception list containing the nodes that are on the SPT and in the domain controlled by the child controller or its adjacent domains. o The constraints for the path such as bandwidth constraints and color constraints. o A destination node Z. If Z is in the domain controlled by the child controller, the child controller computes a shortest path segment satisfying the constraints from node X to node Z within the domain. o Options for computing path segments: E: E set to 1 indicating computing a shortest path segment satisfying the constraints from node X to each of the edge nodes of the domain controlled by the child controller except for the nodes in the exception list. E is set to 1 if there is not any previous hop of node X in the domain. After receiving the request message, the child controller computes a shortest path segment satisfying the constraints from node X to each of the edge nodes of the domain controlled by the child controller except for the nodes in the exception list if E is 1. In addition, it computes a shortest path segment satisfying the constraints from node X to each of the edge nodes of the adjacent domains except for the edge nodes in the exception list just using the inter-domain links attached to node X if node X is an edge node of the domain and an end point of an inter-domain link. The format of the PSReq message is as follows: Chen, et al. Expires November 5, 2017 [Page 14] Internet-Draft PCE-H-SDNs May 2017 ::= [] where: ::=[] ::= [] ::= [] [] [] [] [] [[]] [] [] 6.2.1.4. Reply for Computing Path Segments After receiving a request message from a parent controller for computing path segments, a child controller computes the path segments as requested in the message and sends the parent controller a reply message to reply the request message, which contains the path segments computed. The details of the path segments are hidden from the parent, which sees each of the path segments as a link with a cost. A reply message for computing path segments (PSRep message for short) comprises: o The global tunnel identifier (GTID) and the path identifier (PID). For the path of the tunnel, the path segments are computed. o The address or identifier of the start-node (saying X) in the domain controlled by the child controller. From this node, the path segments are computed. o For each shortest path segment from node X to node Y computed, the address or identifier of node Y and the cost of the shortest path segment from node X to node Y. The child controller stores the details about every shortest path segment computed under the global tunnel identifier (GTID) and the path identifier (PID) when it sends the reply message containing the path segments to the parent controller. Chen, et al. Expires November 5, 2017 [Page 15] Internet-Draft PCE-H-SDNs May 2017 The child controller may delete the path segments computed for the global tunnel identifier (GTID) and the path identifier (PID) if it does not receive any request for keeping them from the parent controller for a given period of time. The format of the PSRep message is as follows: ::= where: ::= [] ::= [ | ] [] 6.2.1.5. Request for Removing Path Segments After a shortest path satisfying a set of constraints from source A to destination Z is computed, a parent controller may delete the path segments computed and stored in the related child controllers, which are not any part of the shortest path. A parent controller may send a child controller a request message for removing the path segments computed by the child controller and stored in the child controller. 1). A request message for removing path segments (RPSReq message for short) comprises: o The global tunnel identifier (GTID). All the path segments stored under GTID in the child controller are to be removed. 2). A request message for removing path segments comprises: o The global tunnel identifier (GTID) and the path identifier (PID). All the path segments stored under GTID and PID in the child controller are to be removed. 3). A request message for removing path segments comprises: Chen, et al. Expires November 5, 2017 [Page 16] Internet-Draft PCE-H-SDNs May 2017 o The global tunnel identifier (GTID) and the path identifier (PID) o A list of start point (or node) addresses or identifiers. All the path segments stored in the child controller under GTID and PID and with a start point or node from the list of start point (or node) addresses or identifiers are to be removed. 4). A request message for removing path segments comprises: o The global tunnel identifier (GTID) and the path identifier (PID) o A list of start point (or node) addresses or identifiers o A list of pairs (start point, a list of end points), which identifies the path segments from start point of each pair to each of the end points in the list of the pairs. In addition to the path segments as described in the previous message, the path segments stored in the child controller under GTID and PID and identified by the list of pairs are to be removed. The format of the RPSReq message is as follows: ::= where: :: = [] ::= [] [] [] ::= [] ::= [] ::= ::= [] Chen, et al. Expires November 5, 2017 [Page 17] Internet-Draft PCE-H-SDNs May 2017 6.2.1.6. Reply for Removing Path Segments After removing the path segments as requested by a request message for removing path segments from a parent controller, a child controller sends the parent controller a reply message for removing path segments. A reply message for removing path segments (RPSRep message for short) comprises: o The global tunnel identifier (GTID) and the path identifier (PID) o Status of the path segments removal: Success: The path segments requested for removal are removed successfully. Fail: The path segments requested for removal can not be removed. o Error code and reasons for failure if the status is Fail. The format of the RPSRep message is as follows: ::= where: ::= [] ::= [] [] 6.2.1.7. Request for Keeping Path Segments After a shortest path satisfying a set of constraints from source A to destination Z is computed, a parent controller may send a request message for keeping path segments to each of the related child controllers to keep the path segments on the shortest path. A request message for keeping path segments (KPSReq message for short) comprises: Chen, et al. Expires November 5, 2017 [Page 18] Internet-Draft PCE-H-SDNs May 2017 o The global tunnel identifier (GTID) and the path identifier (PID). o A list of pairs (start point, end point), each of which identifies the path segment from the start point of the pair to the end point of the pair. The child controller will keep the path segments given by the list of pairs (start point, end point) stored under GTID and PID. It will remove all the other path segments stored under GTID and PID. The format of the KPSReq message is as follows: ::= where: :: = [] ::= ::= [] ::= 6.2.1.8. Reply for Keeping Path Segments After keeping path segments as requested by a request message for keeping path segments from a parent controller, a child controller sends the parent controller a reply message for keeping path segments. A reply message for keeping path segments (KPSRep message for short) comprises: o The global tunnel identifier (GTID) and the path identifier (PID). o Status of the path segment retention: Success: The path segments requested for retention are retained successfully. Chen, et al. Expires November 5, 2017 [Page 19] Internet-Draft PCE-H-SDNs May 2017 Fail: The path segments requested for retention can not be retained. o Error code and reasons for failure if the status is Fail. The format of the KPSRep message is as follows: ::= where: ::= [] ::= [] 6.2.1.9. Request for Creating Tunnel Segment After obtaining the end to end shortest point to point (P2P) path, a parent controller creates a tunnel along the path crossing multiple domains through sending a request message for creating tunnel segment to each of the child controllers along the path in a reverse direction to create a tunnel segment. A request message for creating tunnel segment (CTSReq message for short) comprises: o The global tunnel identifier (GTID) and the path identifier (PID). o A path segment from a start point to an end point for parent without domain topology or a path segment details/ERO for parent with domain topology. o A label and an interface if the domain controlled by the child control is not a destination domain. For parent without domain topology, the child controller allocates and reserves link bandwidth along the path segment identified by the start point and end point, assigns labels along the path segment, and writes cross connects on each of the nodes along the path segment. For parent with domain topology, the child controller assigns labels along the path segment ERO and writes cross connects on each of the Chen, et al. Expires November 5, 2017 [Page 20] Internet-Draft PCE-H-SDNs May 2017 nodes along the path segment. The link bandwidth along the path segment is allocated and reserved by the parent controller. For the non destination domain, the child controller writes the cross connect on the edge node to the downstream domain using the label and the interface from the downstream domain in the message. For the non source domain, the child controller will include a label and an interface in a message to be sent to the parent controller. The interface connects the edge node of the upstream domain along the path. The label is allocated for the interface on the node that is the next hop of the edge node. The format of the CTSReq message is as follows: ::= where: ::= [] ::= [