INTERNET DRAFT Marko Kaittola FUNET Jul 04, 1993 Expires: January, 1994 Mail based file distribution Part 2: Over-all structure Marko Kaittola FUNET Abstract Mapping between X.400 and Internet mail and X.400 routing are normally done using a table-based approach. In practise tables are normal (ASCII) files. In order to function properly tables must be coordinated carefully. One major problem is the lack of automated procedures. This memo - together with it's companion document - proposes one possible solution. This memo discusses the over-all structure aspects, while the companion document discusses the transactions between two nodes. The same solution can be used to transport binary files. This way it is possible to mirror an entire archive with an e-mail only connectivity. Status of this memo This document is an Internet Draft. Internet Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a ``working draft'' or ``work in progress.'' Please check the 1id-abstracts.txt listing contained in the internet-drafts Shadow Directories on nic.ddn.mil, nnsc.nsf.net, nic.nordu.net, ftp.nisc.sri.com, or munnari.oz.au to learn the current status of any Internet Draft. Kaittola Expires: January, 1994 [Page 1] DRAFT File distribution - structure DRAFT Distribution of this memo is unlimited. Table of contents Abstract 1 Status of this memo 1 Table of contents 2 1. Introduction 3 1.1. More details 3 1.2. About security 4 1.3. Terminology 4 2. Distribution 5 2.1. Receiving a file 6 2.2. Sending files 7 3. Client and server roles 8 4. Security consideratins 9 References 9 Acknowledgements 9 Author's address 9 Kaittola Expires: January, 1994 [Page 2] DRAFT File distribution - structure DRAFT 1. Introduction This paper defines a way on using the dialog defined in [DST-DIALOG]. These two documents should be read together. A third document should also be written. That document [DST-MHS] should explain how the other two documents are applied to the needs of X.400 world. This paper has grown from a real need: mapping and routing information in the international R&D X.400 network must be distributed automatically. The most obvious solution would be to use FTP, but unfortunately this isn't always possible. Indeed, the only common denominator seems to be some kind of e-mail connectivity, so this is what the distribution must be based on. Naturally, other distribution techniques may (and most probably will) be used in addition to this. Using a directory (either dns or X.500) is a natural and attractive alternative. This, however, is not yet realistic in a global scale. This proposal tries to fulfill the following requirements: - Files must be distributed rather quickly. In practice this means some minutes from one node to another. Files should reach even their final destination within a couple of hours. - Transport errors and forged files must be automatically identified (and appropriate actions taken). - Management must be simple and require very little human effort. - Distribution must be based on e-mail. This task is simplified by the fact that files are normally public in a sense that anyone may fetch and see them. 1.1. More details Tables (files) are collected in one place. (As of this writing SWITCH is doing the X.400 coordination.) Distribution is done in a more or less tree-like hierarchy. However, to make it more robust, some level of redundancy is needed. In this model there is only one root, but below it there can be practically any kind of directed graph. This model allows for a natural hierarchy of nodes, but doesn't enforce it. Kaittola Expires: January, 1994 [Page 3] DRAFT File distribution - structure DRAFT As redundance easily leads to loops having some kind of loop control is a mandatory requirement. Any level of hierarchy may generate local changes to the files. These changes must propagate further down, but they are expected not to escape outside their scope. This can be achieved by using some kind of subtree that is rooted on a node that makes those changes. Some support for version management is needed. 1.2. About security The companion document [DST-DIALOG] defines a secure way for two systems can communicate together. If every dialog between any two nodes is secure then the over-all system is also secure. (Obviously cracking a node is still possible. However, discussing it is outside the scope of this document.) 1.3. Terminology A client is a node that receives files and a notification about new files. A server is a node that sends files and a notification about new files. If there is a bi-directional link between any two nodes they can both be clients and servers to each other. Terms client and server are relative to a particular pair of nodes. Kaittola Expires: January, 1994 [Page 4] DRAFT File distribution - structure DRAFT 2. Distribution Files are normally distributed by actively sending new files when they have been received. It is also possible to poll a file server to see if there are any new versions. Active sending is done in three phases (as described in [DST-DIALOG]): new files are announced, they are requested and finally sent. However, file distribution is not limited into tree-like structures. It is perfectly possible to have some redundancy (that is: possible loops) in distribution. +------+ | N1 | +------+ / \ / \ / \ \/ \/ +------+ +------+ | N2 | <----> | N3 | +------+ +------+ / \ / \ / \ / \ / \ / \ \/ \/ \/ \/ +------+ +------+ +------+ | N4 | <----- | N5 | <----> | N6 | +------+ +------+ +------+ / \ \ /\ / \ \ \ / \ \ \ \/ \/ \/ \/ +------+ +------+ +------+ +------+ | N7 | <----> | N8 | | N9 | <----> | N10 | +------+ +------+ +------+ +------+ Picture 1 As can be seen from the picture 1, it is possible to have files coming in from many different input servers. For example N5 (Node 5) could receive files from nodes 2, 3 and 6. However, it is expected to accept only the first copy of the same version. Accepting in this context means both installing it locally and distributing it further. Kaittola Expires: January, 1994 [Page 5] DRAFT File distribution - structure DRAFT As node 4 makes some local modifications to files it sends to N7 and N8 it can't send files to N5. (Actually it could send unmodified files.) Naturally N8 can't send anything from N9 either (or the other way round). Picture 1 also shows that mappings may be passed from N10 to N6 although N6 seems to be closer to the root. There are only three kinds of nodes in this approach: the root, a local root and normal nodes. The local root is a node that may make local modifications (N4 in this example). The root is a source of information distribution. For different hierarchies there may be different distribution trees with different roots. 2.1. Receiving a file When a new file is announced the following actions take place. (Look at picture 1 in [DST-DIALOG].) 1) The validity of the announcement is checked. An announcement is considered to be valid if it comes from an approved source (IAM line), it is about an approved file and it is newer than the local version. 2) If the announcement is valid it is requested to be sent. It is possible to request a file from many different servers, if announcements of it are received from multiple servers before the file is actually received. A password and a serial number are included in the request. 3) The validity of the received file is checked. A file is valid if it comes from an approved source (IAM line) with a serial number and a password that match and are not too old (a local database is needed), it contains requested file(s) and the checksum is correct. On same applications also the contents of a file may be checked. If the same file (or a newer version of it) has already been received then the new file will be discarded. If steps 1-3 have been succesfull the file is installed locally and the sending phase starts (if the node isn't a Kaittola Expires: January, 1994 [Page 6] DRAFT File distribution - structure DRAFT receive-only node). 2.2. Sending files When a file is accepted locally it will be announced to the nodes who are being fed with new files. After announcements have been sent no actions are needed unless a new file is requested by someone. In response to a request a file is sent out provided that it is present locally. Access restrictions are normally expected not to exist, although it is possible to limit the possible set of recipients. If more than one file has been requested, or the requested file is too big, it is possible to split the reply into multiple messages to limit the size of the reply messages. Kaittola Expires: January, 1994 [Page 7] DRAFT File distribution - structure DRAFT 3. Client and server roles A generic node consists of two parts, namely client and server. (Some nodes may in practice be only clients or servers. In that case the other part is considered as being inactive.) +---+ | C | +---+ | S | +---+ / \ / \ / \ / \ +---+ +---+ | C | | C | +---+ +---+ | S | | S | +---+ +---+ | | | +---+ | C | +---+ | S | +---+ Picture 2 This document explains the over-all structure. Kaittola Expires: January, 1994 [Page 8] DRAFT File distribution - structure DRAFT 4. Security consideratins Security is based on keys that are sent with the request and copied in a reply. This gives a protection against forged messages. It doesn't work if an ethernet is tapped or some relaying machine is cracked. However, this is considered to be such an extreme situation that such a cracker could in any case cause a great deal of trouble. References [DST-DIALOG] Mail based file distribution - Part 1: Dialog between two nodes [DST-MHS] One more companion document to be written (informational) Acknowledgements Various peoples have contributed on this document. It is impossible to list everyone here. However, I'd like to give special thanks to the following: Urs Eppenberger, Allan Cargille, Harald Tveit Alvestrand, Paul Andre Pays and Jim Romaquera from RARE WG1 / RARE WG-MSG / COSINE MHS managers / IETF X.400 ops have contributed and kept me going. Pasi Ojala wrote the first implementation while I wrote this paper. He also suggested many improvements. He developed the approach used in Hamming coding. Keijo Ruohonen helped with the Hamming coding. Author's address Marko Kaittola FUNET c/o Tampere University of Technology Software Systems Laboratory P.O. Box 553 SF-33101 Tampere Finland E-mail: Marko.Kaittola@funet.fi G=Marko; S=Kaittola; O=funet; A=fumail; C=fi; Kaittola Expires: January, 1994 [Page 9]