DetNet H. Wang Internet Draft P. Wang Intended status: Standards Track C. Zhang Expires: December 15, 2017 Y. Yang Chongqing University of Posts and Telecommunications June 13, 2017 Joint Scheduling Architecture for Deterministic Industrial Field/Backhaul Networks draft-wang-detnet-backhaul-architecture-01 Abstract Joint scheduling of industrial field network and backhaul network is significant for end-to-end deterministic delay requirements of data flows in factories. This document describes a joint scheduling architecture for deterministic industrial field and backhaul networks. Taking WIA-PA wireless field network and IPv6-based backhaul network as an example, this document shows how the joint scheduling architecture works. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on December 15, 2017. Wang, et al. Expires December 15, 2017 [Page 1] Internet-Draft DetNet Backhaul Architecture June 2017 Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction ................................................ 2 2. Joint Scheduling Architecture................................ 4 2.1. Distributed Architecture................................ 4 2.2. Centralized Architecture................................ 5 2.3. Joint Scheduling Architecture........................... 6 3. Joint Scheduling Scheme...................................... 9 3.1. WIA-PA Network Joint Scheduling ....................... 10 3.2. Protocol Conversion.................................... 10 3.3. Industrial Backhaul Network Scheduling ................ 12 4. Security Considerations..................................... 14 5. IANA Considerations ........................................ 14 6. References ................................................. 14 6.1. Normative References................................... 14 6.2. Informative References................................. 14 1. Introduction Deterministic network is an essential element of the industrial network. Using deterministic network in the industrial field can enhance the network performance and greatly reduce the network packet loss. Thus, it is the future development direction of industrial network technology to use deterministic networks in the whole industrial network. Deterministic networks in industrial networks are mainly concentrated on the industrial field networks, such as ISA100.11a[IEC62734], WirelessHART[IEC62591] and WIA- PA[IEC62601]. At the same time, in order to solve the transmission problem among industrial field networks as well as the transmission problem between industrial field networks and wide area networks, a nondeterministic industrial backhaul network is often deployed in Wang, et al. Expires December 15, 2017 [Page 2] Internet-Draft DetNet Backhaul Architecture June 2017 the factory. However, there is little joint scheduling scheme that can be applied to industrial networks. In the Internet, the emergence of SDN technology brings a new choice to solve this problem. SDN is a new type of network architecture that has arisen in recent years. This network has separated the network control plane from the data forwarding plane in physics, which has brought a revolution for the network domain. Through the separation of control plane and data forwarding plane and open communication protocol, SDN has broken the seal of traditional network device. In addition, open interfaces and free programmability also make network management simpler, more dynamic and more flexible. Nowadays, in the use case document[draft-bas-usecase-detnet] and architecture document[draft-finn-detnet-architecture] submitted by the IETF DetNet working group, a deterministic network based on Ethernet has already been researched. The document proposes a network architecture based on SDN technology, which can accurately control the transmission of data streams. However, the document does not consider the characteristics of the industrial backhaul networks and the actual situation of other industrial field deterministic networks. First of all, the data flow of industrial backhaul network is highly sensitive to the uncertainty of time. Therefore, it is very important that how to apply the deterministic networks based on Ethernet to industrial backhaul networks. Secondly, the existing deterministic networks in the industrial field have been widely deployed in the factory, and Deterministic network technology is already very mature, and direct replacement will consume a lot of manpower and material resources. Based on existing work in the architecture document[draft-finn- detnet-architecture], this document proposes a joint scheduling architecture for deterministic industrial field networks. This framework will firstly replace the industrial backhaul networks and other non-deterministic networks of industrial networks into deterministic Ethernet-based network, and then on the basis of SDN technology, this document proposes a joint scheduler, which can be used for joint scheduling on other deterministic networks in deterministic Ethernet-based network and industrial field network. Through deploying the deterministic network throughout the industrial network based on the joint scheduling architecture, it can realize the end-to-end deterministic scheduling between different industrial field networks, and ensure data stream indicators as well as save manpower and material resources. Wang, et al. Expires December 15, 2017 [Page 3] Internet-Draft DetNet Backhaul Architecture June 2017 2. Joint Scheduling Architecture For industrial networks, there are many network controllers in the network, which together constitute the control plane for the whole industrial network. The control plane is very important in the entire network, especially when it comes to cross domain transfer of time-sensitive data. So the control plane architecture will greatly affect the performance of the network, therefore it is becoming a research hotspot on how to give full play to the performance of their respective networks when the multiple controllers are in the joint cooperation. However, there is not a unified standard of joint architecture of multiple controllers in the industry at present. The main frameworks are the following two kinds: the distributed architecture and the centralized architecture. The WIA-PA network, which is the typical of WSNs standards which has become an international standard for industrial field networks approved by IEC, is used as an example to illustrate these architectures. 2.1. Distributed Architecture Distributed architecture is also known as East-West architecture. In the architecture, the status of all network controller is equal, these controllers are connected to each other to form an unstructured network, and achieve cross domain transfer task deployment through the mutual transmission of information, as shown in Figure 1. In the distributed architecture, the controller can exchange different network topologies and the accessibility of information through the east-west interface, and each controller can build a global network topology. In the access to the global network topology, since each controller is equal, it can serve as a server role at the same time, as well as has the service capacity of starting deterministic cross-network transmission. Wang, et al. Expires December 15, 2017 [Page 4] Internet-Draft DetNet Backhaul Architecture June 2017 +-------------------------------------------------------------+ | | Application | +--------+ +--------+ +-------+ | Plane | | APP | | APP | | APP | | | +----+---+ +----+---+ +---+---+ | | | | | | +----------+------------------+------------------+------------+ | | | --------------------------------------------------------------------------------- | | | +----------+------------------+------------------+------------+ | | | | | Control | +----+-----+ +----+-----+ +----+-----+ | Plane | |Controller|------>|Controller|------>|Controller| | | | |<------| |<------| | | | +----------+ +----------+ +----------+ | | | | | | +----------+------------------+------------------+------------+ | | | --------------------------------------------------------------------------------- | | | +----+-----+ +----+-----+ +----+-----+ Forwarding | WIA-PA |------>| backhaul |------>| WIA-PA | Plane | network |<------| network |<------| network | +----------+ +----------+ +----------+ Figure 1. Distributed Architecture 2.2. Centralized Architecture Centralized architecture is also known as vertical multi-level architecture. In this architecture, the control plane is divided into two parts, one is the basic control plane composed of a variety of network controllers; another part is a network controller composed of the main controller, which is responsible for controlling the basic control plane, as shown in Figure 2. Wang, et al. Expires December 15, 2017 [Page 5] Internet-Draft DetNet Backhaul Architecture June 2017 +-------------------------------------------------------------+ | | Application | +--------+ +--------+ +-------+ | Plane | | APP | | APP | | APP | | | +--------+ +----+---+ +-------+ | | | | +-----------------------------+-------------------------------+ | -------------------------------------------------------------------------------- | +-----------------------------+-------------------------------+ | | | | +------+-----+ | | +-----------| Main |------------+ | | | | Controller | | | Control | | +------+-----+ | | Plane | | | | | | +----+-----+ +----+-----+ +----+-----+ | | |Controller| |Controller| |Controller| | | +----+-----+ +----+-----+ +----+-----+ | | | | | | +----------+------------------+------------------+------------+ | | | -------------------------------------------------------------------------------- | | | +----+-----+ +----+-----+ +----+-----+ Forwarding | WIA-PA |------>| backhaul |------>| WIA-PA | Plane | network |<------| network |<------| network | +----------+ +----------+ +----------+ Figure 2. Centralized Architecture The centralized architecture needn't to expand the east-west interface. It only needs to establish a connection with the basic controllers through the southbound interface. After the connection is established, the main controller obtains the every domain network topology through the API interface provided by the basic controllers, and storages global network topology on its own. It can also assign tasks to basic controllers through the API interface. 2.3. Joint Scheduling Architecture In the practical application, distributed architecture not only needs to extend the east-west interface, but also maintains a global Wang, et al. Expires December 15, 2017 [Page 6] Internet-Draft DetNet Backhaul Architecture June 2017 network topology in each controller. Only each controller maintains such a global network topology, it can ensure the deterministic control of the control plane for the whole network. Though the centralized architecture does not have the above requirements, for the deterministic industrial network, the scale of the network is not very large, in the industrial backhaul network, a single SDN controller is sufficient to meet the control demands of industrial backhaul network. If centralized architecture is directly applied to an industrial network, it will not only be unable to give full play to the advantages of the architecture in multi controllers collaboration, but also cause meaningless information interaction between the controllers, which will waste network resource. In view of the problems existing in these two architectures, this document takes the WIA-PA network as an example and proposes a joint scheduling architecture based on the architecture document[draft- finn-detnet-architecture]. The architecture is optimized according to the characteristics of deterministic industrial network, so that a single SDN controller can unite the WIA-PA network systems manager to manage the entire industrial network, and provide support for the deterministic scheduling of data streams across network transmission through industrial backhaul network located in different domains of WIA-PA network. Wang, et al. Expires December 15, 2017 [Page 7] Internet-Draft DetNet Backhaul Architecture June 2017 +-------------------------------------------------------------+ | | Application | +--------+ +--------+ +-------+ | Plane | | APP | | APP | | APP | | | +--------+ +----+---+ +-------+ | | | | +-----------------------------+-------------------------------+ | --------------------------------------------------------------------------------------- | +-----------------------------+-------------------------------+ | | | Control | +--------------+ +----+-----+ +--------------+ | Plane | | WIA-PA |------| SDN |------| WIA-PA | | | |System Manager| |Controller| |System Manager| | | +------+-------+ +----+-----+ +-------+------+ | | | | | | +---------+-------------------+--------------------+----------+ | | | --------------------------------------------------------------------------------------- | | | +----+-----+ +----+-----+ +-----+----+ Forwarding | WIA-PA |------->| backhaul |------->| WIA-PA | Plane | network |<-------| network |<-------| network | +----------+ +----------+ +----------+ Figure 3. Joint scheduling architecture As shown in Figure 3, joint scheduling architecture can be mainly classified into three planes: o Forwarding plane: this plane contains various types of network equipment in different networks. It is the physical entities of the network transmission. In general, to achieve the desired network functions for the network manager, these devices are specific factors of management control operation, which makes their own resources abstract for their own control elements to manage and configure. o Control plane: this plane is formed by the WIA-PA System Manager and the SDN controller. Joint scheduler is integrated into the SDN controller in the form of plugin, and other WIA-PA System Managers accept joint management scheduler by establishing a connection with the SDN controller. Meanwhile, inside the SDN controller, joint scheduler achieves the management of industrial backhaul network by directly calling the corresponding module of SDN controller. Wang, et al. Expires December 15, 2017 [Page 8] Internet-Draft DetNet Backhaul Architecture June 2017 o Application plane: this plane provides users with a unified interface about a variety of resources for the whole network. At the same time, it also provides users with an intuitive, user- friendly interface, which can shield the complex network information of the original. When the application plane triggers a joint scheduling task, the SDN controller calculates path information and resource allocation information based on the task request from the application plane. Upon completion of the calculation, the SDN controller sends them through the unified joint scheduling interface to the corresponding network manager, and the network manager sends them to the industrial field network. So far, the SDN controller has completed a joint scheduling task. Joint Scheduling Architecture defines an architecture that when industrial networks contain other deterministic networks, these deterministic networks and deterministic Ethernet-based networks are jointly scheduling. On the basis of this architecture, control and scheduling for the entire industrial network can be realized by joint scheduler, so as to provide a real-time protection for each data stream. 3. Joint Scheduling Scheme Taking WIA-PA wireless field network and IPv6-based backhaul network as an example, this section shows how the joint scheduling architecture works. Existing WIA-PA scheduling scheme only applies to WIA-PA field network. Scheduling scheme will fail once the data is transferred to backhaul networks. Joint scheduling scheme is innovation and expansion of WIA-PA scheduling scheme. Firstly, scheduling scheme based on SDN in industry backhaul network is added to the original scheduling scheme, so that data can flow in the industrial backhaul network, and the data can be identified and assigned existing backhaul network resource according to their requirements for the network resources. Secondly, conducting an optimization for original WIA-PA scheduling scheme enables scheduling scheme based on WIA-PA networks plays together joint scheduler, and scheduling scheme can simultaneously apply to two non-adjacent domains so that it can be adapt to the cross-border joint operation based on SDN. Thirdly, due to the specificity of cross-border transmission services, the joint scheduling scheme for WIA-PA network VCR_ID and Route ID is reclassified. Wang, et al. Expires December 15, 2017 [Page 9] Internet-Draft DetNet Backhaul Architecture June 2017 Finally, since the system manager allocates a short address to the field device on the basis of the network address information about its own domain in WIA-PA networks. Thus resulting in the entire network short address field device is uncertain. In order to identify the field device on different network domains and domain, the network identifier (PAN_ID) is applied to the joint scheduling scheme to identify WIA-PA network. After the SDN controller initiates joint scheduling module, WIA-PA system manager will actively establish a connection with the united scheduler. After the scheduler receives a cross-border transmission request, joint scheduler will send a request for obtaining topology information and node information to WIA-PA System Manager. Then, the scheduler will assign paths and network resources according to this information by pre-defined scheduling algorithm. After the routing and network resources have been calculated, joint scheduler will configure and deploy networks by the corresponding network controller. 3.1. WIA-PA Network Joint Scheduling In the united scheduling process, path deployment and resource allocation for WIA-PA network are performed by calling the WIA-PA network system manager API interface. System manager will query the corresponding information of the field device in the network upon receiving the acquisition command of joint operation for the network information, and then return the received information to the united scheduler. The system manager will configure communication resources for the corresponding gateway device, routing equipment and field equipment if the system manager receives configuration commands from joint scheduler. After receiving a successful response, it will send a successful reply to the united scheduler. 3.2. Protocol Conversion In the process of cross-border transmission, since industrial backhaul network is different from WIA-PA network, which is not an IP-based Ethernet. Protocol conversion of gateway for WIA-PA packet is needed when the data of WIA-PA network needs to transmit to another network through cross-border industrial backhaul. Meanwhile, according to the joint scheduling scheme, SDN controller is able to identify the WIA-PA Ethernet data stream, and allocate resources according to the data stream type and level of the data stream. Therefore, in the protocol conversion process of gateway, scheduling and control of WIA-PA data flow can be realized by SDN controller Wang, et al. Expires December 15, 2017 [Page 10] Internet-Draft DetNet Backhaul Architecture June 2017 unless the VCR of WIA-PA data stream and the priority are filled in the IPv6 header. +-------+ | Start | +-------+ | +-------------+ | Receiving | |data packets | +-------------+ | /-------------\ +--------------+ /Whether is the \ | Forwarded to | | management |-- Yes -->| the system | \ data / | manager | \-------------/ +--------------+ | No | +-----------+ | Resolution| | Packet | +-----------+ | /------------\ /------------\ +-------------+ / Find the \ / Find the \ | Encapsulate | |corresponding |-- Yes -->| corresponding |-- Yes-->| and sent | \ VCR / \ IPv6 address / | IPv6 packet| \------------/ \------------/ +-------------+ | | | No No | | | | +--------+ | | | End |<---------------------+------------------------+ +--------+ Figure 4. The conversion process of gateway protocol As shown in Figure 4, according to the above section, the gateway will receive the address mapping of joint scheduler configure when configuration WIA-PA network. After that, VCR tables and IPv6 address-mapping tables will be formed according to this information. When the gateway receives WIA-PA packets, it will firstly parse out Route ID, Object ID and Instance ID, and find corresponding VCR from VCR tables. Meanwhile, the gateway finds the corresponding IPv6 address according to Route ID in IPv6 address mapping table. Then, Wang, et al. Expires December 15, 2017 [Page 11] Internet-Draft DetNet Backhaul Architecture June 2017 the gateway begins to encapsulate WIA-PA packets based on IPv6 format, fill VCR_ID in IPv6 header flow label field, and fill the priority of WIA-PA packet in communication category of IPv6 header fields, zero is used to fill up insufficient bytes. Then, the protocol conversion for WIA-PA data is completed. When the gateway receives IPv6 packets from the industrial backhaul networks, the gateway will make out VCR_ID from IPv6 packet header, and find packets VCR in the domain WIA-PA network according to the VCR ID in its own maintenance VCR table, and replace it with the information of original packet. Then, the protocol conversion for IPv6 packet is completed. 3.3. Industrial Backhaul Network Scheduling In deterministic network based on SDN, joint scheduler can recognize WIA-PA data stream through matching on IPv6 flow label field. According to priority of IPv6 and VCR_ID type, joint scheduling can allocate the necessary resources to communication, and ensure that the key data flow is not affected when adding new data flow in the existing network. It can also monitor the real-time data flow of the network. To protect critical data flows from affected, switching paths is also considered when necessary. The scheduling process of industrial backhaul network is shown in Figure 5. Wang, et al. Expires December 15, 2017 [Page 12] Internet-Draft DetNet Backhaul Architecture June 2017 +-------+ | Start | +-------+ | +--------------+ |Obtain network| | topology | +--------------+ | +-------------------+ | Calculate the | | path and allocates|<---------------+ | resources | | +-------------------+ | | | +----------+ | |Query path| | +----------+ | | | /---------------\ +----------------+ /whether the path \ | Calculate the | |meets the resource|- No ->| weight and | \ requirements / | adjustment path| \---------------/ +----------------+ | Yes | +------------+ | Deployment | | flow table | +------------+ | +-----+ | End | +-----+ Figure 5. The scheduling process of Industrial backhaul network After receiving the request for service, the joint scheduler will calculate the route information and network resource allocation. Once the path information and resource allocation are determined, joint dispatcher will confirm whether the resource path is capable of meeting business requirements through the inside module of SDN controller. If it meets business requirements, then the flow table is deployed by SDN controller. Otherwise, the path information and resource allocation are recalculated to choose the other paths to transmit data flow. Wang, et al. Expires December 15, 2017 [Page 13] Internet-Draft DetNet Backhaul Architecture June 2017 3.4. Bandwidth guarantee method Bandwidth guarantee method is implemented on the basis of joint scheduling mechanism, in order to solve the problem that industrial backhaul networks can not fine identify data transmission across the network. By filling the priority information in the WIA-PA network with the RouteID into the IPv6 header, the SDN controller can not only identify cross network transmission of the WIA-PA data stream, but also obtain priority information for the WIA-PA data stream. According to these information, the SDN controller can schedule different priority network transmission data stream to the corresponding switch port queue. In this way, the data stream can obtain corresponding bandwidth guarantee. 4. Security Considerations 5. IANA Considerations This memo includes no request to IANA. 6. References 6.1. Normative References 6.2. Informative References [IEC62734] ISA/IEC, "ISA100.11a, Wireless Systems for Automation, also IEC 62734", 2011, . [IEC62591] IEC, "Industrial Communication Networks - Wireless Communication Network and Communication Profiles - WirelessHART - IEC 62591", 2010, [IEC62601] IEC, "Industrial networks - Wireless communication network and communication profiles - WIA-PA ?IEC 62601", 2015, < https://webstore.iec.ch/preview/info_iec62601%7Bed2.0%7Db.pdf> Wang, et al. Expires December 15, 2017 [Page 14] Internet-Draft DetNet Backhaul Architecture June 2017 [I-D.finn-detnet-problem-statement] Finn, N. and P. Thubert, "Deterministic Networking Problem Statement", draft-finn-detnet-problem-statement-04 (work in progress), October 2015. [I-D.finn-detnet-architecture] Finn, N., Thubert, P., and M. Teener, "Deterministic Networking Architecture", draft-finn-detnetarchitecture-03 (work in progress), March 2016. [I-D.bas-usecase-detnet] Kaneko, Y., Toshiba and Das, S, "Building Automation Use Cases and Requirements for Deterministic Networking", draft- bas-usecase-detnet-00 (work in progress), April 2016. Wang, et al. Expires December 15, 2017 [Page 15] Internet-Draft DetNet Backhaul Architecture June 2017 Authors' Addresses Heng Wang Chongqing University of Posts and Telecommunications 2 Chongwen Road Chongqing, 400065 China Phone: (86)-23-6248-7845 Email: wangheng@cqupt.edu.cn Ping Wang Chongqing University of Posts and Telecommunications 2 Chongwen Road Chongqing, 400065 China Phone: (86)-23-6246-1061 Email: wangping@cqupt.edu.cn Chang Zhang Chongqing University of Posts and Telecommunications 2 Chongwen Road Chongqing, 400065 China Phone: (86)-23-6246-1061 Email: zc910522@126.com Yi Yang Chongqing University of Posts and Telecommunications 2 Chongwen Road Chongqing, 400065 China Phone: (86)-23-6246-1061 Email: 15023705316@163.com Wang, et al. Expires December 15, 2017 [Page 16]