Lecture Series

Conference and Workshop

Scaling phenomena and data network traffic

Walter Willinger

AT&T Labs-Research

Florham Park, NJ 07932-0971

walter@research.att.com

Compared to the Public Switched Telephone Network, data networks such as the global Internet are prime examples of truly large-scale complex systems. To illustrate how various aspects of the Internet's complexity are directly reflected in the nature of the traffic that it carries, we discuss in these five lectures some of the recently observed scaling phenomena in measured Internet traffic (e.g., self-similarity, multifractal scaling), and comment on the few things that they can tell us and on the many things that they may tell us (in due time) about the Internet and its performance. The lectures focus on the philosophical, statistical, and mathematical issues arising in the context of uncovering, understanding, and modeling the dynamic nature of measured Internet traffic based on a wide range of huge and very diverse data sets of high-time resolution

traffic measurements. The emphasis is on "gaining a better understanding of modern data networks by learning about the traffic that these networks transport in reality," and we also illustrate how this approach opens up new areas of mathematical research in the field of performance modeling of modern communication networks.

REFERENCES (for easy reading)

V. Paxson and S. Floyd. "Why we don't know how to simulate the Internet" http://www.aciri.org/floyd/papers/wsc.ps

W. Willinger and V. Paxson. "Where Mathematics meets the Internet" Notices of the American Mathematical Society, Vol. 45, pp. 961--970, 1998. ftp://ftp.ee.lbl.gov/papers/internet-math-AMS98.ps.gz

W. Willinger. "The discovery of self-similar traffic" in: Performance Evaluation: Origins and Directions, Lecture Notes in Computer Science, Vol. 1769, pp. 493--505, Springer-Verlag, 2000.

1. The self-similar nature of network traffic

1.1. Analyzing traffic traces: From statistical to scientific inference

1.2. Inference for scaling phenomena I: Self-similarity

1.2.1. Long-range dependence and asymptotic self-similarity

1.2.2. Heuristic inference methodologies

1.2.3. Whittle's method

1.3. Self-similarity and network traffic: Empirical evidence

1.4. References

1.4.1. W.E. Leland, M.S. Taqqu, W. Willinger and D.V. Wilson, "On the Self-Similar Nature of Ethernet Traffic (Extended Version)," IEEE Transactions on Networking, Vol. 2, pp. 1--15, 1994.

1.4.2. W. Willinger, M.S. Taqqu, W.E. Leland and D.V. Wilson, "Self-Similarity in High-Speed Packet Traffic: Analysis and Modeling of Ethernet Traffic Measurements," Statistical Science, Vol. 10, pp. 67--85, 1995.

1.4.3. V. Paxson and S. Floyd, "Wide Area Traffic: The Failure of Poisson Modeling," IEEE/ACM Transactions on Networking, Vol. 3, pp. 226--244, 1995. http://www.aciri.org/vern/papers.html

1.4.4. M.E. Crovella and A. Bestavros, "Self-similarity in World Wide Web traffic: Evidence and possible causes," IEEE/ACM Transactions on Networking, Vol. 5, pp. 835--846, 1997. http://www.cs.bu.edu/fac/crovella/papers.html

2. Self-similarity through high-variability

2.1. Modeling network traffic: From ``black box'' to physical-based models

2.2. Explaining self-similar scaling in the networking context

2.2.1. LANs: Aggregating many On/Off processes

2.2.2. WANs (Internet): Aggregating many sessions/connections

2.3. Inference for scaling phenomena II: Heavy-tailed distributions

2.3.1. Heavy-tailed On/Off periods: Empirical evidence

2.3.2. Heavy-tailed sessions/connections: Empirical evidence

2.4. Explaining heavy tails in the networking context

2.5. References

2.5.1. V. Paxson and S. Floyd, "Wide Area Traffic: The Failure of Poisson Modeling," IEEE/ACM Transactions on Networking, Vol. 3, pp. 226--244, 1995. http://www.aciri.org/vern/papers.html

2.5.2. W. Willinger, M.S. Taqqu, R. Sherman and D.V. Wilson, "Self-Similarity through High-Variability: Statistical Analysis of Ethernet LAN Traffic at the Source Level," IEEE/ACM Transactions on Networking, Vol. 5, pp. 71--86, 1996.

2.5.3. T.G. Kurtz, "Limit theorems for workload input models," appeared in: Stochastic Networks: Theory and Applications, F.P. Kelly, S. Zachary and I. Ziedins (Eds.), Clarendon Press, Oxford, 1996. http://www.math.wisc.edu/~kurtz/papers/workall.pdf

2.5.4. M.E. Crovella and A. Bestavros, "Self-similarity in World Wide Web traffic: Evidence and possible causes," IEEE/ACM Transactions on Networking, Vol. 5, pp. 835--846, 1997. http://www.cs.bu.edu/fac/crovella/papers.html

2.5.5. W. Willinger, V. Paxson and M.S. Taqqu, "Self-Similarity and Heavy Tails: Structural Modeling of Network Traffic," pp. 27--53, appeared in: ``A Practical Guide to Heavy Tails: Statistical Techniques for Analyzing Heavy Tailed Distributions,'' R. Adler, R. Feldman and M.S. Taqqu (Eds.), Birkhauser Verlag, Boston, MA, 1998. http://math.bu.edu/people/murad/pub/tails-w16-posted.ps

3. Wavelet analysis of scaling phenomena

3.1. The changing nature of Internet traffic

3.1.1. Session characteristics

3.1.2. TCP connection arrival dynamics

3.1.3. IP packet traces

3.1.4. Need for better analysis techniques

3.2. Wavelets and self-similar scaling: Scale-localization

3.2.1. A wavelet-domain view of LRD and self-similarity

3.2.2. Wavelet-based inference methods

3.2.3. Properties of wavelet-based estimators

3.3. Wavelets and multifractal scaling: Time-localization

3.3.1. Beyond self-similar scaling: Multifractals

3.3.2. A wavelet-domain view of multifractal scaling

3.3.3. Multiplicative processes, conservative cascades

3.4. Inference for scaling phenomena III: Multifractals

3.5. References:

3.5.1. P. Abry and D. Veitch, "Wavelet Analysis of Long-Range Dependent Traffic", IEEE Transactions on Information Theory, Vol. 44, pp. 2--15, 1998.

3.5.2. A. Feldmann, A.C. Gilbert, W. Willinger and T.G. Kurtz, "The changing nature of network traffic: Scaling phenomena," Computer Communication Review, Vol. 28, No. 2, pp. 5--29, 1998. http://www.research.att.com/~anja/feldmann/papers.html

3.5.3. R. Riedi, "An introduction to multifractals," http://www-dsp.rice.edu/~riedi/

3.5.4. A.C. Gilbert, W. Willinger and A. Feldmann, "Scaling analysis of conservative cascades, with applications to network traffic," IEEE Transaction on Information Theory, Vol. 45, pp. 971-991, 1999. http://www.research.att.com/~anja/feldmann/papers.html

4. The small-time scaling behavior of network traffic

4.1. Understanding network traffic: What is the impact of the user/network?

4.2. On the nature of network traffic over fine time scales

4.2.1. Small-time scaling phenomena in aggregate TCP/IP traffic

4.2.2. Multifractal scaling behavior of individual TCP connections

4.2.3. Signatures of networking mechanisms

4.3. On explaining multifractal scaling in the networking context

4.3.1. Conservative cascades

4.3.2. TCP/IP

4.3.3. Open issues

4.4. References

4.4.1. R. Riedi and J. Levy Vehel, "Multifractal properties of TCP traffic: a numerical study," http://www-dsp.rice.edu/~riedi/

4.4.2. A. Feldmann, A.C. Gilbert and W. Willinger, "Data networks as cascades: Investigating the multifractal nature of Internet WAN traffic," Computer Communication Review, Vol. 28, No. 4 (Proc. of the ACM Sigcomm'98, Vancouver, Canada), pp. 42--55, 1998. http://www.research.att.com/~anja/feldmann/papers.html

4.4.3. A. Feldmann, P. Huang, A.C. Gilbert and W. Willinger, "Dynamics of IP traffic: A study of the role of variability and the impact of control," Computer Communication Review, Vol. 29, No. 4 (Proc. of the ACM Sigcomm'99, Cambridge, MA), pp. 301--313, 1999. http://www.research.att.com/~anja/feldmann/papers.html

5. Scaling phenomena and network performance

5.1. Self-similarity and performance evaluation

5.1.1. Same old queueing problems - with novel workload models

5.1.2. Qualitative performance evaluation

5.2. Self-similarity through high-variability

5.2.1. Same old queueing problems - with heavy-tailed service times

5.2.2. The ``many mice'' and ``few elephants''

5.3. Small-time scaling behavior and performance evaluation

5.3.1. On the relevance of conventional queueing theory

5.3.2. On the need for closed-loop queueing models

5.3.3. Dealing with large-scale, highly-interacting networks of queues

5.4. Some recent developments and new challenges

5.4.1. Network-wide measurement infrastructures

5.4.2. Large-scale network simulators

5.4.3. New breed of network measurements and analysis tools

5.5. References

5.5.1. W. Willinger, M.S. Taqqu and A. Erramilli, "A bibliographical guide to self-similar traffic and performance modeling for modern high- speed networks," appeared in: Stochastic Networks: Theory and Applications, F.P. Kelly, S. Zachary and I. Ziedins (Eds.), pp. 339--366, Clarendon Press, Oxford, 1996.

5.5.2. K. Park and W. Willinger, Self-Similar Network Traffic and Performance Evaluation, J. Wiley & Sons Inc., New York, 2000 (to appear).