Network Working Group Ned Freed Internet Draft SMTP Service Extension for Command Pipelining 19-Oct-1993 Status of this Memo This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet Drafts as reference material or to cite them other than as a "work in progress". 1. Abstract This memo defines an extension to the SMTP service whereby a server can indicate the extent of its ability to accept multiple commands in a single TCP send operation. Using a single TCP send operation for multiple commands can improve SMTP performance significantly. 2. Introduction Although SMTP is widely and robustly deployed, various extensions have been requested by some parts of the Internet community. In particular, many parts of the Internet make use of high latency network links. SMTP's intrinsic one command- one response structure is significantly penalized by high latency links, often to the point where the factors contributing to overall connection time are dominated by the time spent waiting for responses to individual commands (turnaround time). Internet Draft SMTP Pipelining Oct 1993 In the best of all worlds it would be possible to simply deploy SMTP client software that makes use of command pipelining: batching up multiple commands into single TCP send operations. Unfortunately, the original SMTP specification [1] did not explicitly state that SMTP servers must support this type of operation. As a result a non-trivial number of Internet SMTP servers cannot adequately handle command pipelining. Flaws known to exist in deployed servers include: (1) Process forking and connection handoff in the middle of the SMTP dialogue. Creation of server processes for incoming SMTP connections is a useful, obvious, and harmless implementation technique. However, some SMTP servers defer process forking and connection handoff until some intermediate point in the SMTP dialogue is reached. When this is done material read from the TCP connection and kept in process-specific buffers can be lost. (2) Flushing the TCP input buffer when an SMTP command fails. SMTP commands often fail but there is no reason to flush the TCP input buffer when this happens. Nevertheless, some SMTP servers do this. (3) Improper processing and promulgation of SMTP command failures. For example, some SMTP servers will refuse to accept a DATA command if the last RCPT TO command fails, paying no attention to the success or failure of prior RCPT TO command results. Other servers will accept a DATA command even when all previous RCPT TO commands have failed. Although it is possible to accomodate this sort of behavior in a client that employs command pipelining, it does complicate the construction of the client unnecessarily. This memo uses the mechanism described in [2] to define an extension to the SMTP service whereby an SMTP server can declare that it is capable of handling pipelined commands. The SMTP client can then check for this declaration and use pipelining only when the server declares itself capable of handling it. Expires Apr 1994 [Page 2] Internet Draft SMTP Pipelining Oct 1993 3. Framework for the Command Pipelining Extension The Command Pipelining extension is defined as follows: (1) the name of the SMTP service extension is Pipelining; (2) the EHLO keyword value associated with the extension is PIPELINING; (3) no parameter is used with the PIPELINING EHLO keyword; (4) no additional parameters are added to either the MAIL FROM or RCPT TO commands. (5) no additional SMTP verbs are defined by this extension; and, (6) the next section specifies how support for the extension affects the behavior of a server and client SMTP. 4. The Pipelining Service Extension When a client SMTP wishes to employ command pipelining, it first issues the EHLO command to the server SMTP. If the server SMTP responds with code 250 to the EHLO command, and the response includes the EHLO keyword value PIPELINING, then the server SMTP has indicated that it can accomodate SMTP command pipelining. 4.1. Client use of pipelining Once the client SMTP has confirmed that support exists for the pipelining extension, the client SMTP may then elect to transmit groups of SMTP commands in batches without waiting for a response to each individual command. In particular, the commands RSET, MAIL FROM, SEND FROM, SOML FROM, SAML FROM, and RCPT TO can all appear anywhere in a pipelined command group. The EHLO, DATA, VRFY, EXPN, TURN, QUIT, and NOOP commands can only appear as the last command in a group since their success or failure produces a change of state which the client SMTP must accomodate. (NOOP is included in this group so it can be used as a synchronization point.) Expires Apr 1994 [Page 3] Internet Draft SMTP Pipelining Oct 1993 Additional commands added by other SMTP extensions may only appear as the last command in a group unless otherwise specified by the extensions that define the commands. Client SMTP implementations that employ pipelining MUST check all statuses associated with each command in the group. For example, if none of the RCPT TO recipient addresses were accepted the client must then check the response to the DATA command. If the DATA command was properly rejected the client SMTP can just issue RSET, but if the DATA command was accepted the client SMTP should send a single dot. Client SMTP implementations MUST also check the TCP window size and make sure that each group of commands fits entirely within the window. The window size is usually, but not always, 4K octets. Failure to perform this check can lead to deadlock conditions. 4.2. Server support of pipelining A server SMTP implementation that offers the pipelining extension: (1) MUST NOT flush or otherwise lose the contents of the TCP input buffer under any circumstances whatsoever. (2) SHOULD issue a positive response to the DATA command if and only if one or more valid RCPT TO addresses have been previously received. (3) MAY elect to store command responses to RSET, MAIL FROM, SEND FROM, SOML FROM, SAML FROM, and RCPT TO in an internal buffer so they can sent as a unit. (4) MUST NOT buffer responses to EHLO, DATA, VRFY, EXPN, TURN, QUIT, and NOOP. (5) MUST NOT buffer responses to unrecognized commands. (6) MUST send all pending responses immediately whenever the local TCP input buffer is emptied. (7) MUST NOT make assumptions about commands that are yet to be received. Expires Apr 1994 [Page 4] Internet Draft SMTP Pipelining Oct 1993 The overriding intent of these server requirements is to make it as easy as possible for servers to conform to these pipelining extensions. 5. Examples Consider the following SMTP dialogue that does not use pipelining: S: C: S: 220 innosoft.com SMTP service ready C: HELO dbc.mtview.ca.us S: 250 innosoft.com C: MAIL FROM: S: 250 sender OK C: RCPT TO: S: 250 recipient OK C: RCPT TO: S: 250 recipient OK C: RCPT TO: S: 250 recipient OK C: DATA S: 354 enter mail, end with line containing only "." ... C: . S: 250 message sent C: QUIT S: 250 goodbye The client waits for a server response a total of 9 times in this simple example. But if pipelining is employed the following dialogue is possible: S: C: S: 220 innosoft.com SMTP service ready C: EHLO dbc.mtview.ca.us S: 250-innosoft.com S: 250 PIPELINING C: MAIL FROM: C: RCPT TO: C: RCPT TO: C: RCPT TO: C: DATA Expires Apr 1994 [Page 5] Internet Draft SMTP Pipelining Oct 1993 S: 250 sender OK S: 250 recipient OK S: 250 recipient OK S: 250 recipient OK S: 354 enter mail, end with line containing only "." ... C: . C: QUIT S: 250 message sent S: 250 goodbye The total number of turnarounds has been reduced from 9 to 4. The next example illustrates one possible form of behavior when pipelining is used and all recipients are rejected: S: C: S: 220 innosoft.com SMTP service ready C: EHLO dbc.mtview.ca.us S: 250-innosoft.com S: 250 PIPELINING C: MAIL FROM: C: RCPT TO: C: RCPT TO: C: DATA S: 250 sender OK S: 550 remote to not allowed S: 250 remote to not allowed S: 550 no valid recipients given C: QUIT S: 250 goodbye The client SMTP waits for the server 4 times here as well. If the server SMTP does not check for at least one valid recipient prior to accepting the DATA command, the following dialogue would result: S: C: S: 220 innosoft.com SMTP service ready C: EHLO dbc.mtview.ca.us S: 250-innosoft.com S: 250 PIPELINING C: MAIL FROM: Expires Apr 1994 [Page 6] Internet Draft SMTP Pipelining Oct 1993 C: RCPT TO: C: RCPT TO: C: DATA S: 250 sender OK S: 550 remote to not allowed S: 550 remote to not allowed S: 354 enter mail, end with line containing only "." ... C: . C: QUIT S: 550 no valid recipients S: 250 goodbye 6. Security Considerations This RFC does not discuss security issues and is not believed to raise any security issues not endemic in electronic mail and present in fully conforming implementations of [1]. 7. Acknowledgements This document is based on the SMTP service extension model presented in RFC 1425. Marshall Rose's description of SMTP command pipelining in his book "The Internet Message" also served as a source of inspiration for this extension. 8. References [1] J.B. Postel. Simple Mail Transfer Protocol. Request for Comments 821, (August, 1982). [2] J.C. Klensin, N. Freed, M.T. Rose, E.A. Stefferud, D.H. Crocker. SMTP Service Extensions. RFC 1425, (January, 1993). Expires Apr 1994 [Page 7] Internet Draft SMTP Pipelining Oct 1993 9. Author Addresses Ned Freed Innosoft International, Inc. 250 West First Street, Suite 240 Claremont, CA 91711 USA tel: +1 909 624 7907 fax: +1 909 621 5319 email: ned@innosoft.com Expires Apr 1994 [Page 8]