Overview: Real Time Protocols for
Browser-based ApplicationsGoogleKungsbron 2Stockholm11122Swedenharald@alvestrand.noThis document gives an overview and context of a protocol suite
intended for use with real-time applications that can be deployed in
browsers - "real time communication on the Web".It intends to serve as a starting and coordination point to make sure
all the parts that are needed to achieve this goal are findable, and
that the parts that belong in the Internet protocol suite are fully
specified and on the right publication track.This document is an Applicability Statement - it does not itself
specify any protocol, but specifies which other specifications WebRTC
compliant implementations are supposed to follow.This document is a work item of the RTCWEB working group.The Internet was, from very early in its lifetime, considered a
possible vehicle for the deployment of real-time, interactive
applications - with the most easily imaginable being audio conversations
(aka "Internet telephony") and video conferencing.The first attempts to build this were dependent on special networks,
special hardware and custom-built software, often at very high prices or
at low quality, placing great demands on the infrastructure.As the available bandwidth has increased, and as processors an other
hardware has become ever faster, the barriers to participation have
decreased, and it has become possible to deliver a satisfactory
experience on commonly available computing hardware.Still, there are a number of barriers to the ability to communicate
universally - one of these is that there is, as of yet, no single set of
communication protocols that all agree should be made available for
communication; another is the sheer lack of universal identification
systems (such as is served by telephone numbers or email addresses in
other communications systems).Development of The Universal Solution has proved hard, however, for
all the usual reasons.The last few years have also seen a new platform rise for deployment
of services: The browser-embedded application, or "Web application". It
turns out that as long as the browser platform has the necessary
interfaces, it is possible to deliver almost any kind of service on
it.Traditionally, these interfaces have been delivered by plugins, which
had to be downloaded and installed separately from the browser; in the
development of HTML5, application developers see much promise in the
possibility of making those interfaces available in a standardized way
within the browser.This memo describes a set of building blocks that can be made
accessible and controllable through a Javascript API in a browser, and
which together form a sufficient set of functions to allow the use of
interactive audio and video in applications that communicate directly
between browsers across the Internet. The resulting protocol suite is
intended to enable all the applications that are described as required
scenarios in the use cases document .Other efforts, for instance the W3C WEBRTC, Web Applications and
Device API working groups, focus on making standardized APIs and
interfaces available, within or alongside the HTML5 effort, for those
functions; this memo concentrates on specifying the protocols and
subprotocols that are needed to specify the interactions that happen
across the network.This memo uses the term "WebRTC" (note the case used) to refer to the
overall effort consisting of both IETF and W3C efforts.The goal of the WebRTC protocol specification is to specify a set
of protocols that, if all are implemented, will allow an
implementation to communicate with another implementation using audio,
video and data sent along the most direct possible path between the
participants.This document is intended to serve as the roadmap to the WebRTC
specifications. It defines terms used by other parts of the WebRTC
protocol specifications, lists references to other specifications that
don't need further elaboration in the WebRTC context, and gives
pointers to other documents that form part of the WebRTC suite.By reading this document and the documents it refers to, it should
be possible to have all information needed to implement an WebRTC
compatible implementation.The total WebRTC effort consists of two major parts, each
consisting of multiple documents:A protocol specification, done in the IETFA Javascript API specification, defined in a series of W3C
documents Together, these two specifications aim to provide an
environment where Javascript embedded in any page, when suitably
authorized by its user, is able to set up communication using audio,
video and auxiliary data, as long as the browser supports this
specification. The browser environment does not constrain the types of
application in which this functionality can be used.The protocol specification does not assume that all implementations
implement this API; it is not intended to be necessary for
interoperation to know whether the entity one is communicating with is
a browser or another device implementing this specification.The goal of cooperation between the protocol specification and the
API specification is that for all options and features of the protocol
specification, it should be clear which API calls to make to exercise
that option or feature; similarly, for any sequence of API calls, it
should be clear which protocol options and features will be invoked.
Both subject to constraints of the implementation, of course.For the purpose of this document, we define the following
terminology to talk about WebRTC things:A WebRTC browser (also called a WebRTC User Agent or WebRTC UA)
is something that conforms to both the protocol specification and
the Javascript API cited above.A WebRTC non-browser is something that conforms to the protocol
specification, but does not claim to implement the Javascript API.
This can also be called a "WebRTC device" or "WebRTC native
application".A WebRTC endpoint is either a WebRTC browser or a WebRTC
non-browser. It conforms to the protocol specification.A WebRTC-compatible endpoint is an endpoint that is able to
successfully communicate with a WebRTC endpoint, but may fail to
meet some requirements of a WebRTC endpoint. This may limit where
in the network such an endpoint can be attached, or may limit the
security guarantees that it offers to others. It is not
constrained by this specification; when it is mentioned at all, it
is to note the implications on WebRTC-compatible endpoints of the
requirements placed on WebRTC endpoints.A WebRTC gateway is a WebRTC-compatible endpoint that mediates
media traffic to non-WebRTC entities.All WebRTC browsers are WebRTC endpoints, so any requirement
on a WebRTC endpoint also applies to a WebRTC browser.A WebRTC non-browser may be capable of hosting applications in a
similar way to the way in which a browser can host Javascript
applications, typically by offering APIs in other languages. For
instance it may be implemented as a library that offers a C++ API
intended to be loaded into applications. In this case, similar
security considerations as for Javascript may be needed; however,
since such APIs are not defined or referenced here, this document
cannot give any specific rules for those interfaces.WebRTC gateways are described in a separate document, .The "Mission statement of the IETF" states
that "The benefit of a standard to the Internet is in interoperability
- that multiple products implementing a standard are able to work
together in order to deliver valuable functions to the Internet's
users."Communication on the Internet frequently occurs in two phases:Two parties communicate, through some mechanism, what
functionality they both are able to supportThey use that shared communicative functionality to
communicate, or, failing to find anything in common, give up on
communication.There are often many choices that can be made for
communicative functionality; the history of the Internet is rife with
the proposal, standardization, implementation, and success or failure
of many types of options, in all sorts of protocols.The goal of having a mandatory to implement function set is to
prevent negotiation failure, not to preempt or prevent
negotiation.The presence of a mandatory to implement function set serves as a
strong changer of the marketplace of deployment - in that it gives a
guarantee that, as long as you conform to a specification, and the
other party is willing to accept communication at the base level of
that specification, you can communicate successfully.The alternative - that of having no mandatory to implement - does
not mean that you cannot communicate, it merely means that in order to
be part of the communications partnership, you have to implement the
standard "and then some" - that "and then some" usually being called a
profile of some sort; in the version most antithetical to the Internet
ethos, that "and then some" consists of having to use a specific
vendor's product only.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in .The following terms are used across the documents specifying the
WebRTC suite, in the specific meanings given here. Not all terms are
used in this document. Other terms are used in their commonly used
meaning.The list is in alphabetical order.Undefined term. See "SDP Agent" and "ICE
Agent".Application Programming Interface - a
specification of a set of calls and events, usually tied to a
programming language or an abstract formal specification such as
WebIDL, with its defined semantics.Used synonymously with "Interactive User
Agent" as defined in the HTML specification . See also "WebRTC User
Agent".An abstraction that allows data to be
sent between WebRTC endpoints in the form of messages. Two
endpoints can have multiple data channels between them.An implementation of the Interactive
Connectivty Establishment (ICE) protocol. An ICE Agent may also
be an SDP Agent, but there exist ICE Agents that do not use SDP
(for instance those that use Jingle ).Communication between multiple parties,
where the expectation is that an action from one party can cause a
reaction by another party, and the reaction can be observed by the
first party, with the total time required for the
action/reaction/observation is on the order of no more than
hundreds of milliseconds.Audio and video content. Not to be confused
with "transmission media" such as wires.The path that media data follows from
one WebRTC endpoint to another.A specification of a set of data units,
their representation, and rules for their transmission, with their
defined semantics. A protocol is usually thought of as going
between systems.Media where generation of content
and display of content are intended to occur closely together in
time (on the order of no more than hundreds of milliseconds).
Real-time media can be used to support interactive
communication.The protocol implementation involved in
the SDP offer/answer exchange, as defined in section 3.Communication that happens in order to
establish, manage and control media paths and data paths.The communication channels used
between entities participating in signaling to transfer signaling.
There may be more entities in the signaling path than in the media
path.The model of real-time support for browser-based applications does
not assume that the browser will contain all the functions that need to
be performed in order to have a function such as a telephone or a video
conferencing unit; the vision is that the browser will have the
functions that are needed for a Web application, working in conjunction
with its backend servers, to implement these functions.This means that two vital interfaces need specification: The
protocols that browsers use to talk to each other, without any
intervening servers, and the APIs that are offered for a Javascript
application to take advantage of the browser's functionality.Note that HTTP and WebSockets are also offered to the Javascript
application through browser APIs.As for all protocol and API specifications, there is no restriction
that the protocols can only be used to talk to another browser; since
they are fully specified, any endpoint that implements the protocols
faithfully should be able to interoperate with the application running
in the browser.A commonly imagined model of deployment is the one depicted
below.On this drawing, the critical part to note is that the media path
("low path") goes directly between the browsers, so it has to be
conformant to the specifications of the WebRTC protocol suite; the
signaling path ("high path") goes via servers that can modify, translate
or massage the signals as needed.If the two Web servers are operated by different entities, the
inter-server signaling mechanism needs to be agreed upon, either by
standardization or by other means of agreement. Existing protocols (for
example SIP or XMPP )
could be used between servers, while either a standards-based or
proprietary protocol could be used between the browser and the web
server.For example, if both operators' servers implement SIP, SIP could be
used for communication between servers, along with either a standardized
signaling mechanism (e.g. SIP over WebSockets) or a proprietary
signaling mechanism used between the application running in the browser
and the web server. Similarly, if both operators' servers implement
XMPP, XMPP could be used for communication between XMPP servers, with
either a standardized signaling mechanism (e.g. XMPP over WebSockets or
BOSH) or a proprietary signaling mechanism used between the application
running in the browser and the web server.The choice of protocols for client-server and inter-server
signalling, and definition of the translation between them, is outside
the scope of the WebRTC protocol suite described in the document.The functionality groups that are needed in the browser can be
specified, more or less from the bottom up, as:Data transport: TCP, UDP and the means to securely set up
connections between entities, as well as the functions for deciding
when to send data: Congestion management, bandwidth estimation and
so on.Data framing: RTP, SCTP and other data formats that serve as
containers, and their functions for data confidentiality and
integrity.Data formats: Codec specifications, format specifications and
functionality specifications for the data passed between systems.
Audio and video codecs, as well as formats for data and document
sharing, belong in this category. In order to make use of data
formats, a way to describe them, a session description, is
needed.Connection management: Setting up connections, agreeing on data
formats, changing data formats during the duration of a call; SIP
and Jingle/XMPP belong in this category.Presentation and control: What needs to happen in order to ensure
that interactions behave in a non-surprising manner. This can
include floor control, screen layout, voice activated image
switching and other such functions - where part of the system
require the cooperation between parties. XCON and Cisco/Tandberg's
TIP were some attempts at specifying this kind of functionality;
many applications have been built without standardized interfaces to
these functions.Local system support functions: These are things that need not be
specified uniformly, because each participant may choose to do these
in a way of the participant's choosing, without affecting the bits
on the wire in a way that others have to be cognizant of. Examples
in this category include echo cancellation (some forms of it), local
authentication and authorization mechanisms, OS access control and
the ability to do local recording of conversations.Within each functionality group, it is important to preserve
both freedom to innovate and the ability for global communication.
Freedom to innovate is helped by doing the specification in terms of
interfaces, not implementation; any implementation able to communicate
according to the interfaces is a valid implementation. Ability to
communicate globally is helped both by having core specifications be
unencumbered by IPR issues and by having the formats and protocols be
fully enough specified to allow for independent implementation.One can think of the three first groups as forming a "media transport
infrastructure", and of the three last groups as forming a "media
service". In many contexts, it makes sense to use a common specification
for the media transport infrastructure, which can be embedded in
browsers and accessed using standard interfaces, and "let a thousand
flowers bloom" in the "media service" layer; to achieve interoperable
services, however, at least the first five of the six groups need to be
specified.Data transport refers to the sending and receiving of data over the
network interfaces, the choice of network-layer addresses at each end of
the communication, and the interaction with any intermediate entities
that handle the data, but do not modify it (such as TURN relays).It includes necessary functions for congestion control: When not to
send data.WebRTC endpoints MUST implement the transport protocols described in
.The format for media transport is RTP .
Implementation of SRTP is REQUIRED for all
implementations.The detailed considerations for usage of functions from RTP and SRTP
are given in . The security
considerations for the WebRTC use case are in , and the resulting security
functions are described in .Considerations for the transfer of data that is not in RTP format is
described in , and a
supporting protocol for establishing individual data channels is
described in . WebRTC
endpoints MUST implement these two specifications.WebRTC endpoints MUST implement , , , and the requirements they
include.The intent of this specification is to allow each communications
event to use the data formats that are best suited for that particular
instance, where a format is supported by both sides of the connection.
However, a minimum standard is greatly helpful in order to ensure that
communication can be achieved. This document specifies a minimum
baseline that will be supported by all implementations of this
specification, and leaves further codecs to be included at the will of
the implementor.WebRTC endpoints that support audio and/or video MUST implement the
codecs and profiles required in and .The methods, mechanisms and requirements for setting up, negotiating
and tearing down connections is a large subject, and one where it is
desirable to have both interoperability and freedom to innovate.The following principles apply:The WebRTC media negotiations will be capable of representing the
same SDP offer/answer semantics that are used in SIP , in such a way that it is possible to build a
signaling gateway between SIP and the WebRTC media negotiation.It will be possible to gateway between legacy SIP devices that
support ICE and appropriate RTP / SDP mechanisms, codecs and
security mechanisms without using a media gateway. A signaling
gateway to convert between the signaling on the web side to the SIP
signaling may be needed.When a new codec is specified, and the SDP for the new codec is
specified in the MMUSIC WG, no other standardization should be
required for it to be possible to use that in the web browsers.
Adding new codecs which might have new SDP parameters should not
change the APIs between the browser and Javascript application. As
soon as the browsers support the new codecs, old applications
written before the codecs were specified should automatically be
able to use the new codecs where appropriate with no changes to the
JS applications.The particular choices made for WebRTC, and their implications
for the API offered by a browser implementing WebRTC, are described in
.WebRTC browsers MUST implement .WebRTC endpoints MUST implement the functions described in that
document that relate to the network layer (for example Bundle , RTCP-mux and Trickle ICE ), but do not need to support the API
functionality described there.The most important part of control is the user's control over the
browser's interaction with input/output devices and communications
channels. It is important that the user have some way of figuring out
where his audio, video or texting is being sent, for what purported
reason, and what guarantees are made by the parties that form part of
this control channel. This is largely a local function between the
browser, the underlying operating system and the user interface; this is
specified in the peer connection API , and the media capture API .WebRTC browsers MUST implement these two specifications.These are characterized by the fact that the quality of these
functions strongly influence the user experience, but the exact
algorithm does not need coordination. In some cases (for instance echo
cancellation, as described below), the overall system definition may
need to specify that the overall system needs to have some
characteristics for which these facilities are useful, without requiring
them to be implemented a certain way.Local functions include echo cancellation, volume control, camera
management including focus, zoom, pan/tilt controls (if available), and
more.One would want to see certain parts of the system conform to certain
properties, for instance:Echo cancellation should be good enough to achieve the
suppression of acoustical feedback loops below a perceptually
noticeable level.Privacy concerns MUST be satisfied; for instance, if remote
control of camera is offered, the APIs should be available to let
the local participant figure out who's controlling the camera, and
possibly decide to revoke the permission for camera usage.Automatic gain control, if present, should normalize a speaking
voice into a reasonable dB range.The requirements on WebRTC systems with regard to audio
processing are found in and includes more
guidance about echo cancellation and AGC; the proposed API for control
of local devices are found in .WebRTC endpoints MUST implement the processing functions in . (Together with the requirement in , this means that WebRTC endpoints MUST implement the
whole document.)This document makes no request of IANA.Note to RFC Editor: this section may be removed on publication as an
RFC.Security of the web-enabled real time communications comes in several
pieces:Security of the components: The browsers, and other servers
involved. The most target-rich environment here is probably the
browser; the aim here should be that the introduction of these
components introduces no additional vulnerability.Security of the communication channels: It should be easy for a
participant to reassure himself of the security of his communication
- by verifying the crypto parameters of the links he himself
participates in, and to get reassurances from the other parties to
the communication that they promise that appropriate measures are
taken.Security of the partners' identity: verifying that the
participants are who they say they are (when positive identification
is appropriate), or that their identity cannot be uncovered (when
anonymity is a goal of the application).The security analysis, and the requirements derived from that
analysis, is contained in .It is also important to read the security sections of and .The number of people who have taken part in the discussions
surrounding this draft are too numerous to list, or even to identify.
The ones below have made special, identifiable contributions; this does
not mean that others' contributions are less important.Thanks to Cary Bran, Cullen Jennings, Colin Perkins, Magnus
Westerlund and Joerg Ott, who offered technical contributions on various
versions of the draft.Thanks to Jonathan Rosenberg, Matthew Kaufman and others at Skype for
the ASCII drawings in section 1.Thanks to Alissa Cooper, Bjoern Hoehrmann, Colin Perkins, Colton
Shields, Eric Rescorla, Heath Matlock, Henry Sinnreich, Justin Uberti,
Keith Drage, Magnus Westerlund, Olle E. Johansson, Sean Turner and Simon
Leinen for document review.Jinglescottlu@google.comjbeda@google.comstpeter@jabber.orgrobert.mcqueen@collabora.co.ukseanegan@google.comjhildebr@cisco.comThis section may be deleted by the RFC Editor when preparing for
publication.Added section "On interoperability and innovation"Added data confidentiality and integrity to the "data framing"
layerAdded congestion management requirements in the "data transport"
layer sectionChanged need for non-media data from "question: do we need this?"
to "Open issue: How do we do this?"Strengthened disclaimer that listed codecs are placeholders, not
decisions.More details on why the "local system support functions" section is
there.Added section on "Relationship between API and protocol"Added terminology sectionMentioned congestion management as part of the "data transport"
layer in the layer listRemoved most technical content, and replaced with pointers to
drafts as requested and identified by the RTCWEB WG chairs.Added content to acknowledgments section.Added change log.Spell-checked document.Changed draft name and document date.Removed unused referencesAdded architecture figures to section 2.Changed the description of "echo cancellation" under "local system
support functions".Added a few more definitions.Added pointers to use cases, security and rtp-usage drafts (now WG
drafts).Changed description of SRTP from mandatory-to-use to
mandatory-to-implement.Added the "3 principles of negotiation" to the connection
management section.Added an explicit statement that ICE is required for both NAT and
consent-to-receive.Added references to a number of new drafts.Expanded the description text under the "trapezoid" drawing with
some more text discussed on the list.Changed the "Connection management" sentence from "will be done
using SDP offer/answer" to "will be capable of representing SDP
offer/answer" - this seems more consistent with JSEP.Added "security mechanisms" to the things a non-gatewayed SIP
devices must support in order to not need a media gateway.Added a definition for "browser".Made introduction more normative.Several wording changes in response to review comments from EKRAdded an appendix to hold references and notes that are not yet in
a separate document.Minor grammatical fixes. This is mainly a "keepalive" refresh.Clarifications in response to Last Call review comments. Inserted
reference to draft-ietf-rtcweb-audio.Added a reference to the "unified plan" draft, and updated some
references.Otherwise, it's a "keepalive" draft.Removed the appendix that detailed transports, and replaced it with
a reference to draft-ietf-rtcweb-transports. Removed now-unused
references.Added text to the Abstract indicating that the intended status is
an Applicability Statement.Defined "WebRTC Browser" and "WebRTC device" as things that do, or
don't, conform to the API.Updated reference to data-protocol draftUpdated data formats to reference -rtcweb-audio- and not the
expired -cbran draft.Deleted references to -unified-planDeleted reference to -generic-idp (draft expired)Added notes on which referenced documents WebRTC browsers or
devices MUST conform to.Added pointer to the security section of the API drafts.Added "WebRTC Gateway" as a third class of device, and referenced
the doc describing them.Made a number of text clarifications in response to document
reviews.Refined entity definitions to define "WebRTC endpoint" and
"WebRTC-compatible endpoint".Changed remaining usage of the term "RTCWEB" to "WebRTC", including
in the page header.Changed "WebRTC device" to be "WebRTC non-browser", per decision at
IETF 91. This led to the need for "WebRTC endpoint" as the common
label for both, and the usage of that term in the rest of the
document.Added words about WebRTC APIs in languages other than
Javascript.Referenced draft-ietf-rtcweb-video for video codecs to support.None. This is a "keepalive" update.Changed "gateways" reference to point to the WG document.None. This is a "keepalive" publication.Addressed review comments by Olle E. Johansson and Magnus
WesterlundAddressed review comments from Sean Turner and Alissa Cooper