Internet Draft P. Cameron Xylogics, International Ltd. D. Crocker Silicon Graphics, Inc. October 1993 Transport Multiplexing Protocol (TMux) (Internet draft: draft-cameron-tmux-01.txt) Status of this Memo This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a "working draft" or "work in progress." Please check the I-D abstract listing contained in each Internet Draft directory to learn the current status of this or any other Internet Draft. It is intended that this document will be submitted to the IESG for consideration as a standards document. Distribution of this document is unlimited. Abstract One of the problems with the use of terminal servers is the large number of small packets they can generate. Frequently, most of these packets are destined for only one or two hosts. TMux is a protocol which allows multiple short transport segments, independent of application type, to be combined between a server and host pair. Acknowledgments This specification is the result of several discussions and related initiatives through IETF working groups. (We have heard that Danny Cohen, of USC's ISI, suggested a scheme like TMux about 8 years ago, but have not yet located the reference.) 1. Introduction When network designers consider which protocols generate the most load, they naturally tend to consider protocols which transfer large blocks of data (e.g. FTP, NFS). What is often not considered is the load generated by Telnet and Rlogin because of the assumption that users type slowly and the packets are very small. This is a grave underestimation of the load on networks and hosts which have many Telnet and Rlogin ports on Cameron, Crocker Expires: April 94 [Page 1] Internet Draft TMux October 1993 multiple terminal servers. The problem stems from the fact that the work a host must do to process a 1-byte packet is very nearly as much as the work it must do to process a 1500-byte packet. That is, it is the overhead of processing a packet which consumes a hosts resources, not the processing of the data. If one assumes that most users connected to a terminal server will be connecting to only a few hosts, then it should be obvious that the network and host load could be greatly reduced if traffic from multiple users, destined for the same host, could be sent in the same packet. TMux is designed to improve network utilization and reduce the interrupt load on hosts which conduct multiple sessions involving many short packets. It does this by multiplexing transport traffic onto a single IP datagram, thereby resulting in fewer, larger packets. TMux is highly constrained in its method of accomplishing this task, seeking simplicity rather than sophistication. 2. Protocol Design and Subconnection Messages IP hosts may engage in the use of TMux transparently, and may even switch back and forth between use of TMux and carriage of transport segments in the usual, independent IP datagrams. TMux operates by placing a set of transport segments into the same IP datagram. Each segment has a TMux mini-header which specifies the segment length and the actual segment transport protocol. The receiving host demultiplexes the individual transport segments and presents it to the transport layer as if it had been received in the usual IP/transport packaging. The transport layer is, therefore, unaware of the special encapsulation which was used. Hence, a TMux datagram appears as: | IP hdr | TM hdr | Tport segment | TM hdr | Tport segment| ...| Where: TM hdr is a TMux mini-header and specifies the following Tport segment. Tport segment refers to the entire transport segment, including transport headers. Cameron, Crocker Expires: April 94 [Page 2] Internet Draft TMux October 1993 2.1. IP Protocol field value TMux is indicated in an IP datagram by the Protocol (ID) value of XXXXX. 2.2. Header Format Each 4 octet TMux mini-header has the following general format: | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | +-------------------------------+ | Reserved | Length high | +-------------------------------+ | Length low | +-------------------------------+ | Protocol ID | +-------------------------------+ | Checksum | +-------------------------------+ | Transport segment | | ... | | ... | The 'reserved' field is zeroed on output and ignored on input. The LENGTH field specifies the byte count for the transport seg- ment, from 0-4095 octets. For segments that are longer or larger than the maximum TMux packet allowed (see section 4.1), individual datagrams should be sent. The Protocol ID field contains the value that would normally have been placed in the IP header Protocol field. The 'Checksum' field is the XOR of the first 3 octets. To ensure that TCP, UDP and other segments keep their 32 bit alignment, where the segments being multiplexed are not a multi- ple of 32 bits long, extra octets will be added to re-align the end of the segment, and hence the next segment. These octets will be zeroed on output and ignored on input. This padding will not affect the LENGTH field, it will still contain the real length of the segment. 2.3. Sending Data Host endpoints may choose to use TMux at any time and in either (or both) directions. They also may switch between use of TMux packaging and the usual individual packets for individual trans- port associations. The only barrier to the use of TMux is for the sender to know whether TMux is supported by the receiver, which is important, since early use of TMux is likely to be lim- ited. Cameron, Crocker Expires: April 94 [Page 3] Internet Draft TMux October 1993 The easiest way to achieve this is to only send TMux messages to hosts from which a TMux message has already been received. This then leaves the problem of one host starting the TMux connec- tion. This is most easily accomplished by the host sending an IP datagram with no data, but with a Protocol field value for TMux. This is referred to as a TMux ENQ message. The host receiving this message then knows that the originator supports TMux, and can start to send TMux messages. This will in turn cause the originator of the ENQ message to start to use TMux. If for any reason the receiver does not intend to send TMux mes- sages to the originator, but is prepared to accept them, then it can reply with another ENQ message. If an ENQ message does not get a response, then it is reasonable to resend the ENQ a while later in case the message was lost. If this again is lost, the ENQ may be repeated as often as needed, but the time between requests should increase exponen- tially up to a limit of about 1 hour. Suitable times between ENQs would be 15 seconds, 30 seconds, 60 seconds, 120 seconds etc. Note that this checking process does not need to impede any of the transport (user) data, which may be sent as convenient, albeit in its less-efficient small-packet form. The only problem with this scheme is that any host advertising that it supports TMux, then stopping supporting it, will cause any other hosts a problem. The solution to this is to put a Time To Live (TTL) value on a record of a host sending a TMux packet, and expire it after a suitable time, eg. 1 minute. This is implemented as follows. When a packet that could be TMuxed is to be sent to a host, a check is made to see if the time to live has expired. If so the host is marked as being unable to TMux but the packet is still sent as a TMux packet. If the host is really unable to TMux anymore (a rare occurrence) then this packet will be timed out and retried by the transport provider i.e. TCP and this time it will be sent as a normal packet. If the host is still able to TMux then it should send back TMux traffic (even if it has been rebooted) and the local host will mark it as able to TMux again. This removes any per- formance problem of dropping out of TMuxing and having to send probe messages. 3. Protocol Behavior 3.1. Transport Flow Control TMux operates as an extension to the IP datagram protocol. Hence, it has no impact on most flow control mechanisms, since they operate at the transport layer -- above TMux. Cameron, Crocker Expires: April 94 [Page 4] Internet Draft TMux October 1993 3.2. Connection Management The concept of a connection pertains to certain transport proto- cols, but not to IP or to TMux. Hence, when connection manage- ment is required by a transport protocol using TMux, it occurs in the same fashion as it does for IP. In fact, the transport protocol is not to be aware that TMux is being used. 3.3 Multiplexed Message Construction When a transport provider (eg. TCP or UDP) sends a packet, TMux prepends that packet with a header to create a TMux message, then appends the message to the Multiplexed Message under con- struction. When the first message to be transmitted is placed into the Mul- tiplexed Message under construction, a timer is started. When the timer expires, all outstanding message are placed into the Multiplexed Message and this is transmitted. This ensures that all messages constructed before the timer expires are sent in a single Multiplexed Message. If, during construction of the Mul- tiplexed Message, the buffer holding the packet fills, the Mul- tiplexed Message is transmitted immediately. The delay time should be user configurable; a reasonable time is 20 to 30 milliseconds. The time period should be large enough to give a reasonable probability of sending multiple packets but not so large that the echo response time becomes a problem. This suggests that the upper limit for the timer is probably 1/10th second. As the cost of using timeouts on many systems is quite large, it is recommended that a single timer be used and that all connections are serviced on each expiry period. Additionally, configuration options may limit the number of included data packets or the maximum size of the Multiplexed Message before it is transmitted. It is also suggested that larger packets (eg those over 30 octets) should be sent as stan- dard IP packets, and not multiplexed. This is to ensure that the delay used does not put a delay on those packets for which it is inadvisable. 4. Implementation Suggestion 4.1 Maximum Packet Size In section 3.3, a note is made about sending packets immediately if the limit on TMux packet size is reached. On system where Path MTU Discovery (as per RFC 1191) has been implemented this should be used to discover the maximum packet size that can be transmitted, and this should be used as the maximum TMux packet size. Cameron, Crocker Expires: April 94 [Page 5] Internet Draft TMux October 1993 4.2 Deciding Which Packets to Multiplex It is the responsibility of the sender to decide which packets should be TMux'd and which should not. For example, packets sent by FTP should not normally be multiplexed. In many situa- tions, it may be sensible to restrict the sessions that can be multiplexed to just those involved in login traffic (TELNET and RLOGIN) by examining the source and destination TCP port num- bers. However, if a segment that would not normally be multi- plexed is to be sent and a TMux packet is already under con- struction, then the extra segment can be added to the TMux packet under construction, and this can then be sent immedi- ately, rather than waiting for the timer to expire. 5. Security Considerations Because TMux is effectively an extension to IP, it does not have any more impact on site security than does IP. Security should be dealt with by upper layer protocols. Because some routers filter packets on the TCP port numbers, any packets sent using TMux will not be subject to these tests. Thus larger packets (sent as TCP segments) will be subject to filtering, but smaller packets (sent via TMux) on the same con- nection, will not be subject to filtering. For this reason, routers (that do not support TMux) which support this type of filtering should not allow TMux packets through (in either direction) to prevent this being a problem. 6. Author's Addresses P. Cameron Xylogics International, Ltd. Featherstone Rd. Wolverton Mill Milton Keynes MK12 5RD United Kingdom Telephone: +44 908 222112 Fax: +44 908 222115 Email: cameron@xylint.co.uk D. Crocker Silicon Graphics, Inc. 2011 N. Shoreline Blvd. P.O. Box 7311 Mountain View, CA 94039-7311 USA Telephone: +1 415 390 1804 Fax: +1 415 962 8404 Email: dcrocker@sgi.com Cameron, Crocker Expires: April 94 [Page 6] Internet Draft TMux October 1993 7. Discussion List There is a discussion list for this protocol, which for histori- cal reasons is called: cmp-id@xylint.co.uk Request to join the list should be sent to: cmp-id-request@xylint.co.uk Cameron, Crocker Expires: April 94 [Page 7]