Internet Engineering Task Force J. Arkko Internet-Draft Ericsson Intended status: Informational May 15, 2017 Expires: November 16, 2017 Low Latency Applications and the Internet Architecture draft-arkko-arch-low-latency-00 Abstract Some recent Internet technology developments relate to improvements in communications latency. For instance, improvements in radio communications or the recent work in IETF transport, security, and web protocols. There are also potential applications where latency is potentially in a more significant role than it has traditionally been in Internet communications. Modern networking systems offer many tools for building low-latency networks, from highly optimised individual protocol components to software controlled, virtualised and tailored network functions. This memo views the developments from a system viewpoint, and considers the potential future stresses that the strive for low-latency support for applications may bring. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 16, 2017. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of Arkko Expires November 16, 2017 [Page 1] Internet-Draft Low Latency May 2017 publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Applications with Special Focus on Low Latency . . . . . . . 3 3. Role of Low-Latency vs. Other Communications . . . . . . . . 4 4. Selected Improvements to Communications Latency . . . . . . . 4 5. Architectural Considerations . . . . . . . . . . . . . . . . 5 5.1. Background . . . . . . . . . . . . . . . . . . . . . . . 5 5.2. Implications . . . . . . . . . . . . . . . . . . . . . . 6 5.3. Recommendations for Further Work . . . . . . . . . . . . 8 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8 7. Informative References . . . . . . . . . . . . . . . . . . . 9 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction Some recent Internet technology developments relate to improvements in communications latency. For instance, improvements in radio communications or the recent work in IETF transport, security, and web protocols. There are also potential applications where latency is potentially in a more significant role than it has traditionally been in Internet communications. New applications or technologies does not necessarily imply that latency should be the main driving concern, or that any further efforts beyond those already ongoing are needed. Indeed, modern networking systems offer many tools for building low-latency networks, across the stack. At the IETF, for instance, there has been a recent increase in work related to transport, security, and web application protocols, in part to make significant improvements in latency and connection set-up times. Similar efforts in other parts of the stack exist in 3GPP, IEEE, and other standards organisations. Despite a large number of specific developments, it may be interesting to view the developments from a system viewpoint, and to consider the potential future stresses that the strive for low- latency support for applications may bring. Arkko Expires November 16, 2017 [Page 2] Internet-Draft Low Latency May 2017 The rest of this memo is organised as follows. Section 2 discusses potential applications for low-latency communications. Section 4 reviews some of the recent work across the stack, relating to latency improvements. Finally, Section 5 discusses some of the implications (and non-implications) from an architectural perspective. 2. Applications with Special Focus on Low Latency Most Internet applications enjoy significant benefits from low- latency communications. There are also potential applications where latency is potentially in an even more significant role. For instance, embedding communications technology in automation or traffic systems, or consumer applications such as augmented or virtual reality where advance buffering may not be feasible. Many of the Internet-of-Things and critical services use cases in 5G, for instance, have been listed as requiring low latency and high reliability for communications [ER2015] [HU2015] [NGMN2015] [NO2015] [QU2016] [IMT2020]. Some example use cases include optimisation of utility services such as electricity networks, connected automation systems in factories, remote control of machinery such as mining equipment, or embedded technology in road or railway traffic. The different applications vary in terms of their needs. Some may be very focused on high-speed local area communication, others need to connect at optimal speed over a wide-area network, and yet others need to find the right ways to provide global services without incurring unreasonable delays. Note that when we say "low-latency capabilities", there is no intent to imply any specific implementation of those capabilities. In particular, we look at the low-latency requirements from a broader perspective than Quality-of-Service guarantees or separating traffic onto different classes. Indeed, while today's virtualisation and software-driven technologies give us more tools to deal with those kinds of arrangements as well, past experience on deploying Quality- of-Service mechanisms in the Internet should give us pause [CC2015]. It is not the purpose of this memo to analyse the application requirements for low-latency applications much further; for our purposes it suffices to note that there are applications that are enabled by low-latency capabilities of the underlying network infrastructure. Arkko Expires November 16, 2017 [Page 3] Internet-Draft Low Latency May 2017 3. Role of Low-Latency vs. Other Communications There are some limited applications that rely solely on local communication. One example of such an application is vehicles communicating braking status to nearby ones. However, many applications will include also wide-area communication. If the factory automation machines are not talking other than with themselves, at least their control systems are doing so in order to ensure parts orders, monitoring and maintenance by equipment manufacturers, and so on. This does not imply that these perhaps critical applications are openly accessible from the Internet, but many of them are likely to communicate outside their immediate surroundings. Many applications also rely on wide-area connectivity for software updates. As a result, this document recommends that when building architectures for low-latency applications it is important to take into account that these applications can also benefit from communications elsewhere. 4. Selected Improvements to Communications Latency It should be noted that latency is a very broad topic in communications protocol design, almost as broad as "security", or even "correctness". Implementation techniques to satisfy these requirements vary, some applications can be built with sufficient fast local networking capabilities, others may require, for instance, building world-wide, distributed content delivery mechanisms. Modern networking systems offer many tools for building low-latency networks, across the stack. from highly optimised individual protocol components [I-D.ietf-tls-tls13] [I-D.ietf-quic-transport] [RFC7540] to software controlled, virtualised and tailored network functions [NFV2012] [I-D.ietf-sfc-nsh] [OF2008]. Data- and software-driven network managment and orchestration tools enable networks to be built to serve particular needs. Across the stack there are also many other tools, as well as tools being in development, e.g., a new transport design [L4S] at the IETF. On the lower layers, improvements in radio communications are being made. For instance, the IEEE 802.1 Time-Sensitive Networking Task Group [TSN8021] has worked to define precise time synchronization mechanisms for a local area network, and scheduling mechanisms to Arkko Expires November 16, 2017 [Page 4] Internet-Draft Low Latency May 2017 enable different classes of traffic to use the same network while minimising jitter and latency. At the IETF, the DETNET working group is taking these capabilities and applying them for layer 3 networking [DETNET]. The 3GPP 5G requirements for next-generation access technology are stringent, and are leading to the optimization of the radio interfaces. The requirements specify a one-way latency limit of 0.5ms for ultra-reliable low-latency communications [TS38913]. 5. Architectural Considerations Despite a large number of specific developments, it may be interesting to view the developments from a system viewpoint, and to consider the potential future stresses that the strive for low- latency support for applications may bring. 5.1. Background To begin with, it may be useful to observe that the requirements and developments outlined above do not necessarily imply that any specific new technology is needed or that the nature of communications in the Internet would somehow fundamentally change. And certainly not that latency should be the only or primary concern in technology development. With the drive for a new class of applications, there is often an expectation that this means significant changes. However, all changes need to stand on their own, be justifiable and deployable on a global network. For instance, the discussion around the introduction of the newest 4K or 8K high-definition video streaming applications is reminiscent of the discussions about the introduction of VoIP applications in the Internet. At the time, there was some expectation that special arrangements and Quality-of-Service mechanisms might be needed to support this new traffic class. This turned out to be not true, at least not in general networks. Experience tells us, for instance, that deploying Quality-of-Service mechanisms in the Internet is hard, not so much because of the technology itself, but due to lack of forces that would be able to drive the necessary business changes in the ecosystem for the technology to be feasibly deployable [CC2015]. As claffy and Clark note: "Although the Internet has a standards body (the IETF) to resolve technical issues, it lacks any similar forum to discuss business issues such as how to allocate revenues among competing ISPs offering enhanced services. In the U.S., ISPs feared such Arkko Expires November 16, 2017 [Page 5] Internet-Draft Low Latency May 2017 discussions would risk anti-trust scrutiny. Thus, lacking a way to negotiate the business implications of QoS, it was considered a cost rather than a potential source of revenue. Yet, the relentless growth of a diversity of applications with widely varying performance requirements continued on the public Internet, with ISPs using relatively primitive, and not always completely benign, mechanisms for handling them." These difficulties should not be read as prohibiting all changes. Of course, change can also seem unlikely even in cases where it becomes absolutely necessary or the forces necessary to make a change have actually built up. As a result, statements regarding change in the Internet should be carefully evaluated on their merits from both technical and ecosystem perspective. Secondly, we often consider characteristics from a too narrow viewpoint. In the case of latency, it is easy to focus on a particular protocol or link, whereas from the user perspective latency is a property of the system, not a property of an individual component. For instance, improvements on the performance of one link on a communications path can be insignificant, if the other parts make up a significant fraction of the system-level latency. That may seem obvious, but many applications are highly dependent on communications between a number of different parties which may reside in different places. For instance, a third party may perform authentication for a cloud-based service that also interacts with user's devices and a number of different sensors and actuators. We cannot change the speed of light, and a single exchange with another part of the world may result in a 100ms delay, or about 200 times longer than the expected 5G radio link delay for critical applications. It is clear that designing applications from a system perspective is very important. 5.2. Implications As noted above, low-latency applications need to pay particular attention to the placement of services in the global network. Operations that are on the critical path for the low-latency aspects of an application are unlikely to work well if those communications need to traverse half of the Internet. Many widely used services are already distributed and replicated throughout the world, to minimise communications latency. But many other services are not distributed in this manner. For low-latency applications such distribution becomes necessary. Hosting a global Arkko Expires November 16, 2017 [Page 6] Internet-Draft Low Latency May 2017 service in one location is not feasible due to latency, even when from a scale perspective a single server might otherwise suffice for the service. Content-Delivery Networks (CDNs) and similar arrangements are likely to flourish because of this. In the most extreme cases, edge computing services are needed. How the communications are routed also matters. For instance, architectures based on tunneling to a central point may incur extra delay. One way to address this pressure is to use SDN- and virtualisation-based networks that can be provisioned in the desired manner, so that, for instance, instances of tunnel servers can be placed in the topologically optimal place for a particular application. Recent developments in multipath transport protocols [RFC6824] also provide application- and service-level control of some of the networking behaviour. There is tension between application control (often by content providers) and network control (often by network operators). One special case where that tension has appeared in the past is whether there should be ways to provide information from applications to networks on how packets should be treated. This was extensively discussed during the discussion stemming from implications of increased use of encryption in the Internet, and how that affects operators [I-D.nrooney-marnew-report]. Another case where there is tension is between mechanisms designed for a single link or network vs. end-to-end mechanisms. Many of the stated requirements for low-latency applications are explicitly about end-to-end characteristics and capabilities. Yet, the two mechanisms are very different, and most of the deployment difficulties reported in [CC2015] relate to end-to-end mechanisms. Finally, in the search for even faster connection setup times one obvious technique is cross-layer optimisation. We have seen some of this in the IETF in the rethinking of the layers for transport, transport layer security, and application framework protocols. But while cross-layer optimisation can bring benefits, it also has downsides. In particular, it connects different parts of the stack in additional ways. This can lead to difficulties in further evolution of the technology, if done wrong. In the case of the IETF transport protocol evolution, significant improvements were made to ensure better evolvability of the Arkko Expires November 16, 2017 [Page 7] Internet-Draft Low Latency May 2017 protocols than what we've experienced with TCP, starting from an ability to implement the new protocols in applications rather than in the kernel. The effects of badly designed cross-layer optimisation are a particular form of Internet ossification. The general networking trend, however, is for greater flexibility and programmability. Arguably, the ease at which networks can evolve is probably even more important than their specific characteristics. 5.3. Recommendations for Further Work Low-latency applications continue to be a hot topic in networking. The following topics in particular deserve further work from an architectural point of view: o Application architectures for globally connected but low-latency services. o What are the issues with inter-domain Quality-of-Service mechanisms? Are there approaches that would offer progress on this field? o Network architectures that employ tunneling, and mitigations against the delay impacts of tunnels (such as tunnel server placement or "local breakout" techniques). o The emergence of cross-layer optimisations and how that affects the Internet architecture and its future evolution. o Inter-organisatorial matters, e.g., to what extent different standards organisations need to talk about low latency effects and ongoing work, to promote system-level understanding? Overall, this memo stresses the importance of the system-level understanding of Internet applications and their latency issues. Efforts to address specific sub-issues are unlikely to be fruitful without a holistic plan. 6. Acknowledgements The author would like to thank Brian Trammell, Mirja Kuehlewind, Linda Dunbar, Goran Rune, Ari Keranen, Jeff Tantsura, Stephen Farrell, and many others for interesting discussions in this problem space. The author would also like to acknowledge the important contribution that [I-D.dunbar-e2e-latency-arch-view-and-gaps] made in this topic. Arkko Expires November 16, 2017 [Page 8] Internet-Draft Low Latency May 2017 7. Informative References [CC2015] claffy, kc. and D. Clark, "Adding Enhanced Services to the Internet: Lessons from History", September 2015 (https://www.caida.org/publications/papers/2015/ adding_enhanced_services_internet/ adding_enhanced_services_internet.pdf). [DETNET] "Deterministic Networking (DETNET) Working Group", March 2016 (https://tools.ietf.org/wg/detnet/charters). [ER2015] Yilmaz, O., "5G Radio Access for Ultra-Reliable and Low- Latency Communications", Ericsson Research Blog, May 2015 (https://www.ericsson.com/research-blog/5g/5g-radio- access-for-ultra-reliable-and-low-latency- communications/). [HU2015] "5G Vision: 100 Billion connections, 1 ms Latency, and 10 Gbps Throughput", Huawei 2015 (http://www.huawei.com/minisite/5g/en/defining-5g.html). [I-D.dunbar-e2e-latency-arch-view-and-gaps] Dunbar, L., "Architectural View of E2E Latency and Gaps", draft-dunbar-e2e-latency-arch-view-and-gaps-01 (work in progress), March 2017. [I-D.ietf-quic-transport] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed and Secure Transport", draft-ietf-quic-transport-02 (work in progress), March 2017. [I-D.ietf-sfc-nsh] Quinn, P. and U. Elzur, "Network Service Header", draft- ietf-sfc-nsh-12 (work in progress), February 2017. [I-D.ietf-tls-tls13] Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", draft-ietf-tls-tls13-20 (work in progress), April 2017. [I-D.nrooney-marnew-report] Rooney, N., "IAB Workshop on Managing Radio Networks in an Encrypted World (MaRNEW) Report", draft-nrooney-marnew- report-02 (work in progress), August 2016. Arkko Expires November 16, 2017 [Page 9] Internet-Draft Low Latency May 2017 [IMT2020] "Framework and overall objectives of the future development of IMT for 2020 and beyond", ITU Recommendation M.2083-0, September 2015 (http://www.itu.int/rec/R-REC-M.2083-0-201509-I/en). [L4S] "Low Latency Low Loss Scalable throughput (L4S) Birds-of- Feather Session", July 2016 (https://datatracker.ietf.org/wg/l4s/charter/). [NFV2012] "Network Functions Virtualisation - Introductory White Paper", ETSI, http://portal.etsi.org/NFV/NFV_White_Paper.pdf, October 2012. [NGMN2015] "5G White Paper", NGMN Alliance, February 2015 (https://www.ngmn.org/fileadmin/ngmn/content/downloads/ Technical/2015/NGMN_5G_White_Paper_V1_0.pdf). [NO2015] Doppler, K., "5G the next major wireless standard", DREAMS Seminar, January 2015 (https://chess.eecs.berkeley.edu/pubs/1084/doppler- DREAMS_5G_jan15.pdf). [OF2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., and J. Turner, "OpenFlow: Enabling Innovation in Campus Networks", ACM SIGCOMM Computer Communication Review, Volume 38, Issue 2, pp. 69-74 2008. [QU2016] "Leading the world to 5G", Qualcomm, February 2016 (https://www.qualcomm.com/media/documents/files/qualcomm- 5g-vision-presentation.pdf). [RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, . [RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext Transfer Protocol Version 2 (HTTP/2)", RFC 7540, DOI 10.17487/RFC7540, May 2015, . Arkko Expires November 16, 2017 [Page 10] Internet-Draft Low Latency May 2017 [TS38913] "3rd Generation Partnership Project; Technical Specification Group Radio Access Network; Study on Scenarios and Requirements for Next Generation Access Technologies; (Release 14)", 3GPP Technical Report TR 38.913 V14.2.0, March 2017 (https://portal.3gpp.org/desktopmodules/Specifications/ SpecificationDetails.aspx?specificationId=2996). [TSN8021] "Time-Sensitive Networking Task Group", IEEE (http://www.ieee802.org/1/pages/tsn.html). Author's Address Jari Arkko Ericsson Kauniainen 02700 Finland Email: jari.arkko@piuha.net Arkko Expires November 16, 2017 [Page 11]