On the use of HTTP as a Substratemnot@mnot.nethttps://www.mnot.net/
General
substrateHTTP is often used as a substrate for other application protocols. This document specifies best
practices for these protocols’ use of HTTP.The issues list for this draft can be found at https://github.com/mnot/I-D/labels/bcp56bis.The most recent (often, unpublished) draft is at https://mnot.github.io/I-D/bcp56bis/.Recent changes are listed at https://github.com/mnot/I-D/commits/gh-pages/bcp56bis.HTTP is often used as a substrate for other application protocols. This is done for a
variety of reasons, including:familiarity by implementers, specifiers, administrators, developers and users,availability of a variety of client, server and proxy implementations,ease of use,ubiquity of Web browsers,reuse of existing mechanisms like authentication and encryption,presence of HTTP servers and clients in target deployments, andits ability to traverse firewalls.The Internet community has a long tradition of protocol reuse, dating back to the use of Telnet
as a substrate for FTP and SMTP . However, layering new
protocols over HTTP brings its own set of issues:Should an application using HTTP define a new URL scheme? Use new ports?Should it use standard HTTP methods and status codes, or define new ones?How can the maximum value be extracted from the use of HTTP?How does it coexist with other uses of HTTP – especially Web browsing?How can interoperability problems and “protocol dead ends” be avoided?This document contains best current practices regarding the use of HTTP by applications other than
Web browsing. defines what applications it applies to; surveys the properties
of HTTP that are important to preserve, and conveys best practices for those applications
that do use HTTP.It is written primarily to guide IETF efforts, but might be applicable in other situations. Note
that the requirements herein do not necessarily apply to the development of generic HTTP extensions.The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in
.Different applications have different goals when using HTTP. In this document, we say an
application is using HTTP when any of the following conditions are true:The transport port in use is 80 or 443,The URL scheme “http” or “https” is used,The ALPN protocol ID “http/1.1”, “h2” or “h2c” is used, orThe message formats described in and/or are used in conjunction with the IANA registries defined for HTTP.When an application is using HTTP, all of the requirements of the HTTP protocol suite (including
but not limited to , , , , ,
and ) are in force.An application might not be using HTTP according to this definition, but still relying upon the
HTTP specifications in some manner. For example, an application might wish to avoid re-specifying
parts of the message format, but change others; or, it might want to use a different set of methods.Such applications are referred to as protocols based upon HTTP in this document. These have more
freedom to modify protocol operation, but are also likely to lose at least a portion of the
benefits outlined above, as most HTTP implementations won’t be easily adaptable to these changes,
and as the protocol diverges from HTTP, the benefit of mindshare will be lost.Protocols that are based upon HTTP MUST NOT reuse HTTP’s URL schemes, transport ports, ALPN
protocol IDs or IANA registries; rather, they are encouraged to establish their own.There are many ways that HTTP applications are defined and deployed, and sometimes they are brought
to the IETF for standardisation. In that process, what might be workable for deployment in a
limited fashion isn’t appropriate for standardisation and the corresponding broader deployment.This section examines the facets of the protocol that are important to preserve in these situations.When writing an application’s specification, it’s often tempting to specify exactly how HTTP is to
be implemented, supported and used.However, this can easily lead to an unintended profile of HTTP’s behaviour. For example, it’s
common to see specifications with language like this:This sort of specification is bad practice, because it is adding new semantics to HTTP’s status
codes and methods, respectively; a recipient – whether it’s an origin server, client library,
intermediary or cache – now has to know these extra semantics to understand the message.Some applications even require specific behaviours, such as:This forms an expectation in the client that the response will always be 201 Created, when in
fact there are a number of reasons why the status code might differ in a real deployment. If the
client does not anticipate this, the application’s deployment is brittle.Much of the value of HTTP is in its generic semantics – that is, the protocol elements defined
by HTTP are potentially applicable to every resource, not specific to a particular context.
Application-specific semantics are expressed in the payload; mostly, in the body, but also in
header fields.This allows a HTTP message to be examined by generic HTTP software (e.g., HTTP servers,
intermediaries, client implementatiions), and its handling to be correctly determined. It also
allows people to leverage their knowledge of HTTP semantics without special-casing them for a
particular application.Therefore, applications that use HTTP MUST NOT re-define, refine or overlay the semantics of
defined protocol elements. Instead, they SHOULD focus their specifications on protocol elements
that are specific to them; namely their HTTP resources.See for details.Another common practice is assuming that the HTTP server’s name space (or a portion thereof) is
exclusively for the use of a single application. This effectively overlays special,
application-specific semantics onto that space, precludes other applications from using it.As explained in , such “squatting” on a part of the URL space by a standard usurps the
server’s authority over its own resources, can cause deployment issues, and is therefore bad
practice in standards.Instead of statically defining URL paths, it is RECOMMENDED that applications using HTTP define
links in payloads, to allow flexibility in deployment.Using runtime links in this fashion has a number of other benefits. For example, navigating with a
link allows a request to be routed to a different server without the overhead of a redirection,
thereby supporting deployment across machines well. It becomes possible to “mix” different
applications on the same server, and offers a natural path for extensibility, versioning and
capability management.The simplest possible use of HTTP is to POST data to a single URL, thereby effectively tunnelling
through the protocol.This “RPC” style of communication does get some benefit from using HTTP – namely, message framing and the availability of implementations – but fails to realise many others:Caching for server scalability, latency and bandwidth reduction, and reliability;Authentication and access control;Automatic redirection;Partial content to selectively request part of a response;Natural support for extensions and versioning through protocol extension; andThe ability to interact with the application easily using a Web browser.Using such a high-level protocol to tunnel simple semantics has downsides too; because of its more
advanced capabilities, breadth of deployment and age, HTTP’s complexity can cause interoperability
problems that could be avoided by using a simpler substrate (e.g., WebSockets , if
browser support is necessary, or TCP if not), or making the application be based upon
HTTP, instead of using it (as defined in ).Applications that use HTTP are encouraged to accommodate the various features that the protocol
offers, so that their users receive the maximum benefit from it. This document does not require
specific features to be used, since the appropriate design tradeoffs are highly specific to a given
situation. However, following the practices in will help make them available.This section contains best practices regarding the use of HTTP by applications, including practices
for specific HTTP protocol elements.When specifying the use of HTTP, an application SHOULD use as the primary reference;
it is not necessary to reference all of the specifications in the HTTP suite unless there are
specific reasons to do so (e.g., a particular feature is called out).Applications using HTTP MAY specify a minimum version to be supported (HTTP/1.1 is suggested), and
MUST NOT specify a maximum version.Likewise, applications need not specify what HTTP mechanisms – such as redirection, caching,
authentication, proxy authentication, and so on – are to be supported. Full featured support for
HTTP SHOULD be taken for granted in servers and clients, and the application’s function SHOULD
degrade gracefully if they are not (although this might be achieved by informing the user that
their task cannot be completed).For example, an application can specify that it uses HTTP like this:HTTP Applications SHOULD focus on defining the following application-specific protocol elements:Media types , often based upon a format convention such as JSON ,HTTP header fields, as per , andThe behaviour of resources, as identified by link relations .By composing these protocol elements, an application can define a set of resources, identified by
link relations, that implement specified behaviours, including:Retrieval of their state using GET, in one or more formats identified by media type;Resource creation or update using POST or PUT, with an appropriately identified request body format;Data processing using POST and identified request and response body format(s); andResource deletion using DELETE.For example, an application might specify:In HTTP, URLs are opaque identifiers under the control of the server. As outlined in ,
standards cannot usurp this space, since it might conflict with existing resources, and constrain
implementation and deployment.In other words, applications that use HTTP MUST NOT associate application semantics with specific
URL paths. For example, specifying that a “GET to the URL /foo retrieves a bar document” is bad
practice. Likewise, specifying “The widget API is at the path /bar” violates .Instead, applications that use HTTP are encouraged to use typed links to convey the
URIs that are in use, as well as the semantics of the resources that they identify. See
for details.Generally, a client with begin interacting with a given application server by requesting an initial
document that contains information about that particular deployment, potentially including links to
other relevant resources.Applications that use HTTP SHOULD allow an arbitrary URL to be used as that entry point. For
example, rather than specifying “the initial document is at “/foo/v1”, they should allow a
deployment to use any URL as the entry point for the application.In cases where doing so is impractical (e.g., it is not possible to convey a whole URL, but only a
hostname) applications that use HTTP MAY define a well-known URL as an entry point.Applications that use HTTP MUST allow use of the “https” URL scheme, and SHOULD NOT allow use of
the “http” URL scheme, unless interoperability considerations with existing deployments require it.
They MUST NOT use other URL schemes.“https” is preferred to mitigate pervasive monitoring attacks .Using other schemes to denote an application using HTTP makes it more difficult to use with
existing implementations (e.g., Web browsers), and is likely to fail to meet the requirements of
.If it is necessary to advertise the application in use, this SHOULD be done in message payloads,
not the URL scheme.Applications that use HTTP SHOULD use the default port for the URL scheme in use. If it is felt
that networks might need to distinguish the application’s traffic for operational reasons, it MAY
register a separate port, but be aware that this has privacy implications for that protocol’s
users. The impact of doing so MUST be documented in Security Considerations.Applications that use HTTP MAY use stateful cookies to identify a client and/or store
client-specific data to contextualise requests.If it is only necessary to identify clients, applications that use HTTP MAY use HTTP authentication
; if the Basic authentication scheme is used, it MUST NOT be used with the
‘http’ URL scheme.In either case, it is important to carefully specify the scoping and use of these mechanisms; if
they expose sensitive data or capabilities (e.g., by acting as an ambiant authority), exploits are
possible. Mitigations include using a request-specific token to assure the intent of the client.Applications that use HTTP MUST confine themselves to using registered HTTP methods such as GET,
POST, PUT, DELETE, and PATCH.New HTTP methods are rare; they are required to be registered with IETF Review (see ),
and are also required to be generic. That means that they need to be potentially applicable to
all resources, not just those of one application.While historically some applications (e.g., and ) have defined non-generic
methods, now forbids this.When it is believed that a new method is required, authors are encouraged to engage with the HTTP
community early, and document their proposal as a separate HTTP extension, rather than as part of
an application’s specification.Applications that use HTTP MUST only use registered HTTP status codes.As with methods, new HTTP status codes are rare, and required (by ) to be registered
with IETF review. Similarly, HTTP status codes are generic; they are required (by ) to
be potentially applicable to all resources, not just to those of one application.When it is believed that a new status code is required, authors are encouraged to engage with the
HTTP community early, and document their proposal as a separate HTTP extension, rather than as part
of an application’s specification.Status codes’ primary function is to convey HTTP semantics for the benefit of generic HTTP
software, not application-specific semantics. Therefore, applications MUST NOT specify additional
semantics or refine existing semantics for status codes.In particular, specifying that a particular status code has a specific meaning in the context of an
application is harmful, as these are not generic semantics, since the consumer needs to be in the
context of the application to understand them.Furthermore, applications using HTTP MUST NOT re-specify the semantics of HTTP status codes, even
if it is only by copying their definition. They MUST NOT require specific status phrases to be
used; the status phrase has no function in HTTP, and is not guaranteed to be preserved by
implementations.Typically, applications using HTTP will convey application-specific information in the message body
and/or HTTP header fields, not the status code.Specifications sometimes also create a “laundry list” of potential status codes, in an effort to be
helpful. The problem with doing so is that such a list is never complete; for example, if a network
proxy is interposed, the client might encounter a 407 Proxy Authentication Required response; or,
if the server is rate limiting the client, it might receive a 429 Too Many Requests response.Since the list of HTTP status codes can be added to, it’s safer to refer to it directly, and point
out that clients SHOULD be able to handle all applicable protocol elements gracefully (i.e.,
falling back to the generic n00 semantics of a given status code; e.g., 499 can be safely
handled as 400 by clients that don’t recognise it).Applications that use HTTP MAY define new HTTP header fields, following the advice in ,
Section 8.3.1.Typically, using HTTP header fields is appropriate in a few different situations:Their content is useful to intermediaries (who often wish to avoid parsing the body), and/orTheir content is useful to generic HTTP software (e.g., clients, servers), and/orIt is not possible to include their content in the message body (usually because a format does not allow it).If none of these motivations apply, using a header field is NOT RECOMMENDED.New header fields MUST be registered, as per and .It is RECOMMENDED that header field names be short (even when HTTP/2 header compression is in
effect, there is an overhead) but appropriately specific. In particular, if a header field is
specific to an application, an identifier for that application SHOULD form a prefix to the header
field name, separated by a “-“.The semantics of existing HTTP header fields MUST NOT be re-defined without updating their
registration or defining an extension to them (if allowed). For example, an application using HTTP
cannot specify that the Location header has a special meaning in a certain context.See for requirements regarding header fields that carry application state (e.g,. Cookie).This document has no requirements for IANA. discusses the impact of using stateful mechanisms in the protocol as ambiant authority,
and suggests a mitigation. requires support for ‘https’ URLs, and discourages the use of ‘http’ URLs, to mitigate
pervasive monitoring attacks.Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and RoutingThe Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations.Key words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Transport Layer Security (TLS) Application-Layer Protocol Negotiation ExtensionThis document describes a Transport Layer Security (TLS) extension for application-layer protocol negotiation within the TLS handshake. For instances in which multiple application protocols are supported on the same TCP or UDP port, this extension allows the application layer to negotiate which protocol will be used within the TLS connection.URI Design and OwnershipSection 1.1.1 of RFC 3986 defines URI syntax as "a federated and extensible naming system wherein each scheme's specification may further restrict the syntax and semantics of identifiers using that scheme." In other words, the structure of a URI is defined by its scheme. While it is common for schemes to further delegate their substructure to the URI's owner, publishing independent standards that mandate particular forms of URI substructure is inappropriate, because that essentially usurps ownership. This document further describes this problematic practice and provides some acceptable alternatives for use in standards.Cryptographic Algorithm Implementation Requirements and Usage Guidance for Encapsulating Security Payload (ESP) and Authentication Header (AH)This document updates the Cryptographic Algorithm Implementation Requirements for the Encapsulating Security Payload (ESP) and Authentication Header (AH). It also adds usage guidance to help in the selection of these algorithms.ESP and AH protocols make use of various cryptographic algorithms to provide confidentiality and/or data origin authentication to protected data communications in the IP Security (IPsec) architecture. To ensure interoperability between disparate implementations, the IPsec standard specifies a set of mandatory-to- implement algorithms. This document specifies the current set of mandatory-to-implement algorithms for ESP and AH, specifies algorithms that should be implemented because they may be promoted to mandatory at some future time, and also recommends against the implementation of some obsolete algorithms. Usage guidance is also provided to help the user of ESP and AH best achieve their security goals through appropriate choices of cryptographic algorithms.RFC Style GuideThis document describes the fundamental and unique style conventions and editorial policies currently in use for the RFC Series. It captures the RFC Editor's basic requirements and offers guidance regarding the style and structure of an RFC. Additional guidance is captured on a website that reflects the experimental nature of that guidance and prepares it for future inclusion in the RFC Style Guide. This document obsoletes RFC 2223, "Instructions to RFC Authors".Hypertext Transfer Protocol (HTTP/1.1): Range RequestsThe Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines range requests and the rules for constructing and combining responses to those requests.Hypertext Transfer Protocol (HTTP/1.1): CachingThe Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP caches and the associated header fields that control cache behavior or indicate cacheable response messages.MPLS Forwarding Compliance and Performance RequirementsThis document provides guidelines for implementers regarding MPLS forwarding and a basis for evaluations of forwarding implementations. Guidelines cover many aspects of MPLS forwarding. Topics are highlighted where implementers might otherwise overlook practical requirements which are unstated or under emphasized or are optional for conformance to RFCs but are often considered mandatory by providers.Hypertext Transfer Protocol Version 2 (HTTP/2)This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection. It also introduces unsolicited push of representations from servers to clients.This specification is an alternative to, but does not obsolete, the HTTP/1.1 message syntax. HTTP's existing semantics remain unchanged.Media Type Specifications and Registration ProceduresThis document defines procedures for the specification and registration of media types for use in HTTP, MIME, and other Internet protocols. This memo documents an Internet Best Current Practice.Web LinkingThis document specifies relation types for Web links, and defines a registry for them. It also defines the use of such links in HTTP headers with the Link header field. [STANDARDS-TRACK]Guidelines and Registration Procedures for URI SchemesThis document updates the guidelines and recommendations, as well as the IANA registration processes, for the definition of Uniform Resource Identifier (URI) schemes. It obsoletes RFC 4395.Hypertext Transfer Protocol (HTTP/1.1): Conditional RequestsThe Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP/1.1 conditional requests, including metadata header fields for indicating state changes, request header fields for making preconditions on such state, and rules for constructing the responses to a conditional request when one or more preconditions evaluate to false.Hypertext Transfer Protocol (HTTP/1.1): Semantics and ContentThe Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines the semantics of HTTP/1.1 messages, as expressed by request methods, request header fields, response status codes, and response header fields, along with the payload of messages (metadata and body content) and mechanisms for content negotiation.Registration Procedures for Message Header FieldsThis specification defines registration procedures for the message header fields used by Internet mail, HTTP, Netnews and other applications. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.Telnet Protocol SpecificationThis is the specification of the Telnet protocol used for remote terminal access in the ARPA Internet. The purpose of the TELNET Protocol is to provide a fairly general, bi-directional, eight-bit byte oriented communications facility. Its primary goal is to allow a standard method of interfacing terminal devices and terminal-oriented processes to each other. It is envisioned that the protocol may also be used for terminal-terminal communication ("linking") and process-process communication (distributed computation). This RFC specifies a standard for the ARPA Internet community. Hosts on the ARPA Internet are expected to adopt and implement this standard. Obsoletes NIC 18639.File Transfer ProtocolThis memo is the official specification of the File Transfer Protocol (FTP) for the DARPA Internet community. The primary intent is to clarify and correct the documentation of the FTP specification, not to change the protocol. The following new optional commands are included in this edition of the specification: Change to Parent Directory (CDUP), Structure Mount (SMNT), Store Unique (STOU), Remove Directory (RMD), Make Directory (MKD), Print Directory (PWD), and System (SYST). Note that this specification is compatible with the previous edition.Simple Mail Transfer ProtocolThis document is a self-contained specification of the basic protocol for the Internet electronic mail transport. [STANDARDS-TRACK]The WebSocket ProtocolThe WebSocket Protocol enables two-way communication between a client running untrusted code in a controlled environment to a remote host that has opted-in to communications from that code. The security model used for this is the origin-based security model commonly used by web browsers. The protocol consists of an opening handshake followed by basic message framing, layered over TCP. The goal of this technology is to provide a mechanism for browser-based applications that need two-way communication with servers that does not rely on opening multiple HTTP connections (e.g., using XMLHttpRequest or <iframe>s and long polling). [STANDARDS-TRACK]Transmission Control ProtocolThe JavaScript Object Notation (JSON) Data Interchange FormatJavaScript Object Notation (JSON) is a lightweight, text-based, language-independent data interchange format. It was derived from the ECMAScript Programming Language Standard. JSON defines a small set of formatting rules for the portable representation of structured data.This document removes inconsistencies with other specifications of JSON, repairs specification errors, and offers experience-based interoperability guidance.Defining Well-Known Uniform Resource Identifiers (URIs)This memo defines a path prefix for "well-known locations", "/.well-known/", in selected Uniform Resource Identifier (URI) schemes. [STANDARDS-TRACK]Pervasive Monitoring Is an AttackPervasive monitoring is a technical attack that should be mitigated in the design of IETF protocols, where possible.HTTP State Management MechanismThis document defines the HTTP Cookie and Set-Cookie header fields. These header fields can be used by HTTP servers to store state (called cookies) at HTTP user agents, letting the servers maintain a stateful session over the mostly stateless HTTP protocol. Although cookies have many historical infelicities that degrade their security and privacy, the Cookie and Set-Cookie header fields are widely used on the Internet. This document obsoletes RFC 2965. [STANDARDS-TRACK]Hypertext Transfer Protocol (HTTP/1.1): AuthenticationThe Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypermedia information systems. This document defines the HTTP Authentication framework.The 'Basic' HTTP Authentication SchemeThis document defines the "Basic" Hypertext Transfer Protocol (HTTP) authentication scheme, which transmits credentials as user-id/ password pairs, encoded using Base64.CardDAV: vCard Extensions to Web Distributed Authoring and Versioning (WebDAV)This document defines extensions to the Web Distributed Authoring and Versioning (WebDAV) protocol to specify a standard way of accessing, managing, and sharing contact information based on the vCard format. [STANDARDS-TRACK]Calendaring Extensions to WebDAV (CalDAV)This document defines extensions to the Web Distributed Authoring and Versioning (WebDAV) protocol to specify a standard way of accessing, managing, and sharing calendaring and scheduling information based on the iCalendar format. This document defines the "calendar-access" feature of CalDAV. [STANDARDS-TRACK]