Novell NetWare and AT&T ISN Brian Howell Consultant Systems Engineering Division Abstract: This application note shows the relative performance of a NetWare network when compared to a NetWare network bridged by an AT&T ISN. Introduction Purpose of Report The purpose of this report is to show the relative performance of a Novell NetWare network when its distance and access capabilities are enhanced by the introduction of AT&T's Information Systems Network (ISN). To establish the compatibility of AT&T 802.3 packets and NetWare 802.3 packets a preliminary test was conducted. NetWare network packets on an 802.3 network were passed to another NetWare network across an ISN containing two Ethernet Bridge Interface Modules (EBIMs.) Two Stages of Initial Testing Initial testing was in two stages: first with one network workstation and then with five network workstations performing a variety of common network operations. These tests indicated that response time was significantly longer for one workstation than it was for five workstations when identical operations were performed. This result was opposite of what would be expected with a typical 802.3 Carrier Sense Multiple Access/Collision Detection (CSMA/CD) network. Subsequent Testing and Results Subsequent testing indicated that poor performance of a single workstation was due to the inherent buffering design built into the EBIMs. In other words, the performance actually increased as traffic increased. When only one workstation was performing a given operation, the packets would be buffered or held up until the buffer was full. When the buffer was full it would transmit all of the buffered packets. When five workstations were all performing the identical operations, the buffers filled faster and there was less delay imposed by the ISN than when a single workstation was in use. The ramifications of this design mean only that data traffic and usage patterns should be given careful consideration when using an ISN to connect remote NetWare networks. Special attention should be given to the amount and type of operations that are being performed over the connections provided by the ISN. Description of the ISN The Information Systems Network (ISN) is a networking product from AT&T that consists of virtual packet switching controllers and various communication modules (see Figure XX). The communication modules connect a wide variety of hosts, printers and communications resources. The network can be configured so that resources on the network can be shared by network users. Remote concentrators extend the packet controller to users beyond the reach of the packet controller by way of a fiber optic link or by leased lines. Figure 1: ISN packet controller Packet Controller Connections Packet controllers at different locations can be connected to create a wide area network (WAN). The packet controllers communicate through fiber optic links at 8.64 Mbit/s or through trunk modules that run at speeds of up to 2.048 Mbit/s. These trunk connections make communications possible across town or across the country (see Figure XX). Figure 2: Multinode architecture Using AT&T's Premises Distribution System The ISN uses AT&T's Premises Distribution System to distribute the network to the workstation. This system is based on twisted pair wiring, fiber optic links and modular cross-connect hardware. It makes possible a multitude of configuration options (see Figure XX). Figure 3: Single node architecture Packet Description The data transported on the ISN is broken into short packets containing a fixed number of bits. The packets contain source and destination address information that is appended to the data. When a user at a workstation (or endpoint) issues a call setup request to another endpoint or resource, the controller stores the addresses in memory. Once a call is in place, the packets arriving at the controller have their destination addresses replaced by the stored address and are then sent to their destination. The virtual, or switched, connection can be taken down by one of the users and the address is removed from memory. Contention and Transmit Bus Contention for the transmit and receive buses that run at 8.64 Mbit/s, is handled by the contention bus that runs at 864 Kbit/s. Each module with a packet to send, sends a contention code of 18 bits to the contention bus. The code is compared with the codes of other contending modules. Based on the value of the contention code, one module is declared the winner and given exclusive access to the next available time slot on the transmit bus. The contention and transmit buses of the ISN are synchronized so that the contention bus carries a contention code in the same time that a data packet is transmitted by the transmit bus. Modules that lose contention for the bus back off and contend for the next available time slot. A round robin priority algorithm allows modules that have lost contention to raise their priority code until they win. In the ISN the short length and high speed (8.64 Mbit/s) of the transmit and receive buses makes the propagation delay shorter than the time that defines a single bit. The ISN is thus able to perfectly schedule access to the transmit bus. This technique permits a very high (80-90 percent) use of the backplane without any serious performance degradation. Both long and short messages are broken into a series of short packets that are interspersed in time on the backplane buses. By sending messages a little at a time, delay in buffering large messages is avoided. This results in much more efficient use of transmission facilities, and it allows for shorter delays in transmission times especially when multiple packet controllers are traversed. Implementation By implementing techniques described above, the ISN provides a high-speed and versatile network that will accommodate centralized as well as decentralized computing implementations. This maximizes the effectiveness of the computer and peripheral resources of an organization. Introduction to the EBIM The Ethernet Bridge Interface Module (EBIM) is one of the many communications modules that can be placed in the ISN controller or ISN concentrators. Its purpose is to bridge separate Ethernet network segments. The module conforms to the IEEE 802.3 (Carrier Sense Multiple Access with Collision Detection) and Ethernet version 2 specifications. The EBIM implements only the lower two levels (Physical level and Data Link level) of the International Standards Organization's Open Systems Interconnect (OSI) model. Devices on bridged Ethernet networks must therefore use identical implementations of the higher level protocols. Possible Connections A maximum of nine EBIMs can be interconnected. Each EBIM has eight permanent virtual circuit connections. Since each PVC requires two connections, a total of 36 PVCs can be established. A total of 4,000 Media Access Control (MAC) addresses can be supported across all bridged networks. This limitation is imposed by limited memory space in the EBIM and not by the Ethernet networks. Up to eight interconnected Ethernet networks make up an extended network. Five of the extended networks can be connected to allow a total of 40 Ethernet networks to communicate with each other. However, the extended network configuration reduces the throughput because of the number of hops or bridges that the data must traverse in order to reach its destination. Test Configuration AT&T ISN Configuration Novell NetWare Configuration AT&T ISN Model 60 E File Server - Novell 386A EBIM Firmware Level 2.0 Network Interface Card - NE1000 Software version 4.1.1 Operating System: SFT NetWare v2.12 Figure 4: Test configuration 1 Test Descriptions Packet Arrival Time Packet arrival times were measured to show the impact of an ISN on the performance of a local network. It should be noted that the ISN is designed to take advantage of long distance communications facilities such as telephone trunk lines and fiber optic links. Because of the increased capabilities the ISN adds to a network, it is not expected to perform the same as a native CSMA/CD network. In order to assess the impact of the ISN, a test was made on the Novell network by transmitting a file from the file server to a workstation. A simple file copy was performed from the file server to the hard disk of the workstation. A network protocol analyzer was used to capture, record and measure the transmission. This was done with a Network General Sniffer and a Novell LANalyzer. The test was performed on one workstation and on five workstations. After taking the measurements on the native network, the tests were redone with the same file server and workstations with the ISN acting as a bridge between the file server and the worksations. Figure XX shows the packet arrival time distribution. Master Test Battery The master test battery is a series of 11 tests that consist of 11 of the most common network operations. The test is designed to perform a given operation as many times as it can in the space of the test duration (20 seconds). The program counts how many times per second the operation was performed and displays a numeric value upon completion. Again, this test was run on the native network and then again on the network with the ISN bridging the file server and workstations. The master test results are shown in Figure XX. Kbyte/s As an additional method of showing the impact the ISN has on the performance of the network, a protocol analyzer was used to take a reading of the data flowing on the network for the duration of the tests. In all cases the number shown indicates the peak load of traffic generated. The measurements were taken in unrestricted mode and noted all traffic, regardless of source or destination. Figure XX illustrates the measurements for each of the operations in the master test battery. Packet Arrival Time Distribution Explanation The graph shown in Figure XX displays packet arrival times in milliseconds. It compares the percentage of the total number of packets sent and the number that arrived within a given time frame. The operation performed is a file transfer of a 25,184 byte file. The graph also contrasts the percentage difference for one workstation against that of five workstations. Note: Appendix A lists a summary of the measurements. Observations Without the ISN both one and five workstations had almost 90 percent of the packets arrive within the first four milliseconds. When the same file was transmitted with one workstation over the ISN only 50 percent made it within the first four milliseconds. The other 50 percent arrived within 25 to 29 milliseconds. When five workstations performed the file transfer, 50 percent arrived in the first four milliseconds and 40 percent of the remaining packets arrived within the next five milliseconds, making approximately 90 percent that arrived within the first nine milliseconds. Figure 5: Packet arrival time distribution Master Test Results Explanation The graph shown in Figure XX represents the number of operations per second performed for each of the eleven operations in the test battery. The chart shows the difference in the number of operations per second that can be performed with and without the ISN. The numbers are included for the tests using one workstation and five workstations. The graph represents the average of the five workstations and indicates a per workstation number of operations per second. Observations In every operation except one, the single workstation without the ISN was able to accomplish the most operations per second. This is as expected. However, depending on the operation being executed, the difference in performance between one and five workstations is not as great when using the ISN as it is when the operations are performed on a local network without the ISN. Figure 6: Master test results Kbyte/s Explanation Figure XX, Figure XX and Figure XX are graphs that show the number of Kbyte/s generated during the master test battery, performed first by one workstation and then by five workstations. The first two graphs show the difference in data traffic generated with and without the ISN. Observations The following two graphs show the difference in the traffic generated when the traffic over the ISN is local rather than remote. The percentage of difference is not the same for one workstation as it is for five workstations. Figure 7: Kbyte/s for one workstation Percentage of Kbyte/s Explanation In the following graph 100 percent is the number of Kbyte/s that can be generated on a native NetWare network without the ISN. (See configuration 1.) The chart compares the percentage of traffic that can be generated by one workstation and by five workstations during the master test battery on a network with the ISN. Observations The first category listed is the average percentage difference for all of the operations executed by one workstation compared to the average difference of five workstations performing the same operations. The chart illustrates that one workstation connected remotely by the ISN can generate approximately 32 percent of the traffic that a workstation can generate on a local network. Five workstations can generate approximately 54 percent of the data that the same workstations can generate on a local network. Note: See Appendix A for a summary table of the number of Kbyte/s for each of the operations in the master test battery. Figure 8: Percentage of Kbyte/s Average Operations Per Second The following graph depicts the incremental changes in the performance curve created by adding additional workstations. The operation performed for this graph is the open/close file in multiple directories in the master test battery. The numbers will be different for each operation performed, but the basic curve of the graph will be the same. Observations The line depicting the performance curve of the operations performed on a network without the ISN follows the same curve that a typical CSMA/CD network follows. Initially, the degradation in performance is slight as workstations are added. As increasing numbers of workstations and traffic are added, the network reaches its saturation point and performance drops off dramatically. The line showing the performance of the workstations on the ISN network illustrates that the curve generated by this network varies from typical CSMA/CD networks. An important point to note is that performance actually improves as workstations are added. This is true to a point at which the ISN network follows the same characteristics as a typical CSMA/CD network and performance drops off sharply. The reason performance improves is because the buffers in the EBIMs are not waiting as long to be filled when more workstations generate packets. Note: See Appendix A for the summary table of the master test battery results. Figure 9: Performance curves Throughput Test The throughput test was designed to measure the maximum amount of traffic the ISN is capable of handling. The test was done by simulating a load on the network. During the test it was demonstrated that the ISN could consistently transport 950 Kbit/s each way. If that amount was exceeded, the packets were either discarded or the workstations could not get enough packets through to complete the operation and the application would time out. By tuning the traffic generated on the network, it was possible to determine the maximum amount of traffic that allowed the file transfers to complete. It was shown that the ISN is capable of a throughput of approximately 1.9 Mbit/s (950 Kbit/s in each direction). AT&T representatives say that for network planning purposes, a level of 1.5 to 1.7 Mbit/s should be expected. Conclusions Based on the previously mentioned tests and measurements, the ISN proves to be a viable alternative for many network implementation applications. Because of the 182KB receive buffer and 192KB transmit buffer built into the EBIMs, the functional performance obtained at a low load level will seem inconsistent with the performance capabilities inherent in the EBIM design. In fact when the EBIMs are pushed to their maximum capacity, they approach the throughput levels on a native CSMA/CD network. Based on these results, the ISN is a viable network alternative for extending Novell NetWare networks beyond the reach of existing media limitations. The ISN is another tool in the enterprise-wide implementation of NetWare networks. Appendix A: Raw Data Tables Packet Arrival Time Distribution One workstation w/EBIM One workstation w/o EBIM ms Packets ms Packets 0 - 4 84 51.53% 0 - 4 143 87.73% 5 - 9 0 0.00% 5 - 9 3 1.84% 10 - 14 0 0.00% 10 - 14 5 3.07% 15 - 19 0 0.00% 15 - 19 8 4.91% 20 - 24 0 0.00% 20 - 24 0 0.00% 25 - 29 79 48.47% 25 - 29 0 0.00% >30 0 0.00% >30 4 2.45% 163 100.00% 163 100.00% Five workstations w/EBIM Five workstations w/o EBIM ms Packets ms Packets 0 - 4 419 51.41% 0 - 4 735 90.18% 5 - 9 321 39.39% 5 - 9 17 2.09% 10 - 14 0 0.00% 10 - 14 29 3.56% 15 - 19 42 5.15% 15 - 19 7 0.86% 20 - 24 0 0.00% 20 - 24 8 0.98% 25 - 29 32 3.93% 25 - 29 8 0.98% >30 1 0.12% >30 11 1.35% 815 100.00% 815 100.00% Note: ms = milliseconds Kbyte/s Summary Table Five workstations One workstation Open/close file - mult. directories 45 68 9 31 Open/close file - single directory 45 95 9 42 Small shared file random read 40 176 8 55 Large shared file random read 18 22 4 7 Private file 128KB seq. read 45 48 9 16 Private file 128KB random read 130 341 35 128 Large block file transfer 130 414 40 192 Create/write/close/delete 120 131 35 79 Record lock/unlock 50 132 17 55 Directory search (*.*) 50 131 13 50 Random directory search 60 150 12 45 File Xfer Test Local network Bridged network with ISN Seconds AVG Seconds AVG 1,205 Bytes 0.60 0.78 0.78 0.72 2.20 2.00 1.76 1.99 5,079 Bytes 0.84 0.92 1.02 0.93 2.08 1.88 2.10 2.02 10,752 Bytes 1.08 0.91 0.98 0.99 2.31 2.36 2.22 2.30 25,184 Bytes 1.49 1.33 1.32 1.38 2.87 2.89 2.82 2.86 50,176 Bytes 1.87 1.77 1.69 1.78 3.26 2.80 2.95 3.00 108,048 Bytes 2.34 2.37 2.26 2.32 4.57 5.11 5.70 5.13 214,705 Bytes 5.07 3.64 3.54 4.08 10.71 9.81 9.59 10.04 Test Descriptions A. Network Test Battery The network test battery is a utility developed by Novell for benchmarking and measuring network performance. The battery is a series of commonly performed network operations. The name of each test describes the operation that is being performed. Each of the operations was performed for 20 seconds except for tests eight and nine (the directory search tests), which were performed for a duration of 10 seconds each. The numeric value to the right of the tests indicates the number of operations per second that could be performed within the time frame of the test duration. Each of the individual tests is described in more detail below. 1. Open/close file (multiple directories) (no disk activity) This test measures the number of file opens and closes the file server can perform per second. The files opened are selected randomly within the four levels of subdirectories. Besides testing the speed at which the file server can do an open and close, this test also measures the time it takes the server to traverse the hierarchical directory structure. 2. Open/close file (single directory) (no disk activity) This test is the same as the open/close file test, except the files opened are confined to the highest level directory. Comparing the results of this test with the open/close file test illustrates the difference that traversing the hierarchical directory structure makes. This test focuses more on raw file open and close speeds. 3. Small shared file random read (4KB) (no disk activity) This test measures the software time required to perform disk read operations, excluding managing the disk channel. The requests are for only 1 byte of data, to minimize the packet size and the transfer time between the file server and network adapter. The file is shared so the workstation PC cannot do local buffering and must access the file server for every request. This test is the best measurement of how long it takes the operating system software to service a read request. As such, it is a measurement of software efficiency, excluding, as much as possible, the hardware I/O factor. Disk read is the most common file server request made. 4. Large shared file random read (4MB) (requires disk activity) This test measures the time it takes to randomly read a large database- type file. The requests are for only 1 byte of data, again to minimize the packet size and the transfer time between the file server and network adapter. Since the file is so large, only parts of it can be cached in the server RAM, and each request probably has to go to the hard disk. 5. Create/write close/delete (requires disk activity) This test measures the speed at which the file server can create and write to a new file, then close and delete the file. The write is a large 16KB request, similar to the large block file transfer test. This test is the most disk I/O intensive of all the tests. The elevator seeking and caching algorithms in the server make a difference with this test. 6. Private file random read (128KB) (requires disk activity) This test illustrates the problems of local caching by the PC. A private file is randomly read 64 bytes at a time. The MS Net-based operating systems will read more than 64 bytes around the request and cache it locally. However, the effort spent servicing the larger read is wasted, because the file accessing is random and extra data read will probably not be needed. 7. Large block file transfer (16KB) (no disk activity) This test measures the efficiency of the server in servicing large I/O requests. All test workstations issue 16KB read requests to the same file at the same offset. The MS Net- based servers will service this as one request; the NetWare-based servers will service it as 16 or 32 requests. This test also measures the efficiency of the server in transferring data to and from the network adapter. 8. Directory search test (no disk activity) This test measures the number of wild card directory searches the file server can perform per second. A search request can return several matching directory entries. The next search nexts do not have to make a request to the file server. NetWare does not return multiple directory entries per search request and does not cache directory entries in the local workstations. 9. Random directory search (multiple directories) (no disk activity) This test shows the speed at which a search for a specific directory entry can be performed. The test selects a random file within one of the four directory entries and searches for it. It also randomly selects files that do not exist to measure the time needed to discover that a file is not there. 10. Record lock/unlock (no disk activity) This benchmark tests the speed at which the file server can lock and unlock records. A single physical record is locked and unlocked by the workstation. The file copy test consists of a NetWare copy (NCOPY) performed from a NetWare drive on the file server to the local hard drive of the workstation. Two files of significantly different sizes are used. The file copy is timed from when the enter key is pressed to execute the command until the workstation reports that the file is copied. The copies are performed five consecutive times and the averages are listed. C. Performance Test The performance test is designed by Novell to measure true data throughput or performance of a network. This test is completely end-to-end at the application layer of the ISO model. The test is an application that sends data to the file server and back as fast as the hardware allows. The numeric value measures the actual data throughput listed in Kbyte/s. This does not measure raw bandwidth or transmission capability, but data throughput capability as seen by a network user performing a network application. The actual operation performed by the test consisted of data writes of records 4,096 bytes long. The writes were performed in overlaid fashion-each record was written on top of the previous record.