End-to-end Routing Behavior in the Internet
1996, Computer Communication Review
Abstract
The large-scale behavior of routing in the Internet has gone virtually without any formal study, the exceptions being Chinoy's analysis of the dynamics of Internet routing information [Ch93], and recent work, similar in spirit, by Labovitz, Malan and Jahanian [LMJ97]. We report on an analysis of 40,000 end-to-end route measurements conducted using repeated "traceroutes" between 37 Internet sites. We analyze the routing behavior for pathological conditions, routing stability, and routing symmetry. For pathologies, we characterize the prevalence of routing loops, erroneous routing, infrastructure failures, and temporary outages. We find that the likelihood of encountering a major routing pathology more than doubled between the end of 1994 and the end of 1995, rising from 1.5% to 3.3%. For routing stability, we define two separate types of stability, "prevalence," meaning the overall likelihood that a particular route is encountered, and "persistence," the likelihood that a route remains unchanged over a long period of time. We find that Internet paths are heavily dominated by a single prevalent route, but that the time periods over which routes persist show wide variation, ranging from seconds up to days. About 2/3's of the Internet paths had routes persisting for either days or weeks. For routing symmetry, we look at the likelihood that a path through the Internet visits at least one different city in the two directions. At the end of 1995, this was the case half the time, and at least one different autonomous system was visited 30% of the time.
References (50)
- F. Baker, Ed., "Requirements for IP Version 4 Routers," RFC 1812, DDN Network Information Center, June 1995.
- C. Baransel, W. Dobosiewicz, and P. Gburzynski, "Routing in Multihop Packet Switching Networks: Gb/s Challenge," IEEE Network, 9(3), pp. 38-61, May/June 1995.
- I. Bilinskis and A. Mikelsons, Randomized Signal Processing, Prentice Hall International, 1992.
- J-C. Bolot, "End-to-End Packet Delay and Loss Behavior in the Internet," Proceedings of SIG- COMM '93, pp. 289-298, September 1993.
- R. Braden, D. Clark, and S. Shenker, "Inte- grated Services in the Internet Architecture: an Overview," RFC 1633, DDN Network Information Center, June 1994.
- L. Breslau and D. Estrin, "Design of Inter- Administrative Domain Routing Protocols," Proceedings of SIGCOMM '90, pp. 231-241, September 1990.
- B. Chinoy, "Dynamics of Internet Routing Infor- mation," Proceedings of SIGCOMM '93, pp. 45- 52, September 1993.
- K. Claffy, H-W. Braun and G. Polyzos, "A Param- eterizable Methodology for Internet Traffic Flow Profiling," IEEE JSAC, 13(8), pp. 1481-1494 Oc- tober 1995.
- K. Claffy, G. Polyzos and H-W. Braun, "Measure- ment Considerations for Assessing Unidirectional Latencies," Internetworking: Research and Expe- rience, 4 (3), pp. 121-132, September 1993.
- S. Deering and D. Cheriton, "Multicast Routing in Datagram Internetworks and Extended LANs," ACM Transactions on Computer Systems, 8(2), pp. 85-110, May 1990.
- L. Delgrossi and L. Berger, Ed., "Internet Stream Protocol Version 2 (ST2), Protocol Specification-Version ST2+," RFC 1819, DDN Network Information Center, August 1995.
- Sean Doran, "Route Flapping," with notes by Stan Barber, http://www.nanog.org/
- 95.NANOG.notes/route-flapping.html.
- D. Estrin, Y. Rekhter and S. Hotz, "Scalable Inter- Domain Routing Architecture," Proceedings of SIGCOMM '92, pp. 40-52, August 1992.
- D. Ferrari, A. Banerjea and H. Zhang, "Network support for multimedia: A discussion of the Tenet approach," Computer Networks and ISDN Sys- tems, 26(10), pp. 1267-1280, July 1994.
- S. Floyd and V. Jacobson, "The Synchroniza- tion of Periodic Routing Messages," IEEE/ACM Transactions on Networking, 2(2), pp. 122-136, April 1994.
- E. Gustafsson and G. Karlsson, "A Literature Sur- vey on Traffic Dispersion," IEEE Network, 11(2), pp. 28-36, March/April 1997.
- S. Hares and D. Katz, "Administrative Domains and Routing Domains: A Model for Routing in the Internet," RFC 1136, Network Information Cen- ter, SRI International, Menlo Park, CA, Decem- ber 1989.
- C. Huitema, Routing in the Internet, Prentice Hall PTR, 1995.
- V. Jacobson, traceroute, ftp://ftp.ee.lbl.gov/ traceroute.tar.Z, 1989.
- A. Khanna and J. Zinky, "The Revised ARPANET Metric," Proceedings of SIGCOMM '89, pp. 45-56, 1989.
- C. Labovitz, G. Malan, and F. Jahanian, "Internet Routing Instability," to appear in Proceedings of SIGCOMM '97, September 1997.
- M. Little, "Goals and Functional Requirements for Inter-Autonomous System Routing," RFC 1126, Network Information Center, SRI International, Menlo Park, CA, October 1989.
- M. Lottor, ftp://nic.merit.edu/nsfnet/statistics; October 1995.
- J. McQuillan, G. Falk and I. Richer, "A Re- view of the Development and Performance of the ARPANET Routing Algorithm," IEEE Transac- tions on Communications, 26(12), pp. 1802-1811, December 1978.
- J. McQuillan, I. Richer and E. Rosen, "The New Routing Algorithm for the ARPANET," IEEE Transactions on Communications, 28(5), pp. 711-719, May 1980.
- D. Mills, "Network Time Protocol (Version 3): Specification, Implementation and Analysis," RFC 1305, Network Information Center, SRI In- ternational, Menlo Park, CA, March 1992.
- A. Mukherjee, "On the Dynamics and Signifi- cance of Low Frequency Components of Inter- net Load," Internetworking: Research and Expe- rience, Vol. 5, pp. 163-205, December 1994.
- V. Paxson, "Empirically-Derived Analytic Mod- els of Wide-Area TCP Connections," IEEE/ACM Transactions on Networking, 2(4), pp. 316-336, August 1994.
- V. Paxson, Measurements and Analysis of End-to- End Internet Dynamics, Ph.D. dissertation, Uni- versity of California, Berkeley, April 1997.
- R. Perlman and G. Varghese, "Pitfalls in the De- sign of Distributed Routing Algorithms," Proceed- ings of SIGCOMM '88, pp. 43-54, August 1988.
- R. Perlman, "A comparison between two routing protocols: OSPF and IS-IS," IEEE Network, 5(5), pp. 18-24, September 1991.
- R. Perlman, Interconnections: Bridges and Routers, Addison-Wesley, 1992.
- Y. Rekhter and B. Chinoy, "Injecting Inter- autonomous System Routes into Intra-autonomous System Routing: a Performance Analysis," In- ternetworking: Research and Experience, Vol. 3, pp. 189-202, 1992.
- Y. Rekhter, "Inter-Domain Routing: EGP, BGP, and IDRP," in [St95].
- Y. Rekhter and T. Li, "A Border Gateway Protocol 4 (BGP-4)," RFC 1771, DDN Network Informa- tion Center, March 1995.
- Y. Rekhter and P. Gross, "Application of the Bor- der Gateway Protocol in the Internet," RFC 1772, DDN Network Information Center, March 1995.
- J. Rice, Mathematical Statistics and Data Analy- sis, 2nd edition, Duxbury Press, 1995.
- E. Rosen, "Exterior Gateway Protocol (EGP)," RFC 896, Network Information Center, SRI Inter- national, Menlo Park, CA, October 1982.
- S. Ross, Stochastic Processes, John Wiley & Sons, 1983.
- D. Sanghi, A.K. Agrawal, Ó. Gudmundsson, and B.N. Jain, "Experimental Assessment of End-to- end Behavior on Internet," Proceedings of INFO- COM '93, San Francisco, March, 1993.
- M. Schwartz and T. Stern, "Routing Techniques Used in Computer Communication Networks," IEEE Transactions on Communications, 28(4), pp. 539-552, April 1980.
- D. Sidhu, T. Fu, S. Abdallah, R. Nair, and R. Coltun, "Open Shortest Path First (OSPF) Rout- ing Protocol Simulation," Proceedings of SIG- COMM '93, pp. 53-62, September 1993.
- M. Steenstrup, editor, Routing in Communications Networks, Prentice-Hall, 1995.
- W.R. Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, 1994.
- P. Traina, "Experience with the BGP-4 Protocol," RFC 1773, DDN Network Information Center, March 1995.
- P. Traina, editor, "BGP-4 Protocol Analysis," RFC 1774, DDN Network Information Center, March 1995.
- R. Wolff, "Poisson Arrivals See Time Averages," Operations Research, 30(2), pp. 223-231, 1982.
- ZG-LA92] W. Zaumen and J.J. Garcia-Luna Aceves, "Dy- namics of Link-state and Loop-free Distance- vector Routing Algorithms," Internetworking: Re- search and Experience, Vol. 3, pp. 161-188, 1992.
- L. Zhang, S. Deering, D. Estrin, S. Shenker, and D. Zappala, "RSVP: A New Resource ReSer- Vation Protocol," IEEE Network, 7(5), pp. 8-18, September 1993.