CaltechAUTHORS
  A Caltech Library Service

A pragmatic approach to dealing with high-variability in network measurements

Willinger, Walter and Alderson, David and Li, Lun (2004) A pragmatic approach to dealing with high-variability in network measurements. In: IMC '04 Proceedings of the 4th ACM SIGCOMM conference on Internet measurement. ACM , New York, NY, pp. 88-100. ISBN 1-58113-821-0. https://resolver.caltech.edu/CaltechAUTHORS:20170105-123015213

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20170105-123015213

Abstract

The Internet is teeming with high variability phenomena, from measured IP flow sizes to aspects of inferred router-level connectivity, but there still exists considerable debate about how best to deal with this encountered high variability and model it. While one popular approach favors modeling highly variable event sizes with conventional, finite variance distributions such as lognormal or Weibull distributions, Mandelbrot has argued for the last 40 years that there are compelling mathematical, statistical, and practical reasons for why infinite variance distributions are natural candidates for capturing the essence behind high variability phenomena. In this paper, we elaborate on Mandelbrot's arguments and present a methodology that often allows for a clear distinction between the two approaches. In particular, by requiring the resulting models to be resilient to ambiguities (i.e., robust to real-world deficiencies in the underlying network measurements) and internally self-consistent (i.e., insensitive with respect the duration, location, or time of the data collection), we provide a rigorous framework for a qualitative assessment of the observed high variability. We apply the proposed framework to assess previously reported findings about measured Internet traffic and inferred router- and AS-level connectivity. In the process, we also discuss what our approach has to say about recent discussions concerning network traffic being Poisson or self-similar and router-level or AS-level connectivity graphs of the Internet being scale-free or not.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1145/1028788.1028800DOIArticle
http://dl.acm.org/citation.cfm?doid=1028788.1028800PublisherArticle
Additional Information:© 2004 ACM.
Subject Keywords:Measurement, Experimentation, Verification, High variability, power-laws, heavy-tailed distributions, scaling distributions, Pareto distributions, lognormal distributions, borrowing strength, sequential moment plots, model consistency, model robustness
Classification Code:G.3 [ Probability and Statistics ]: Distribution functions
Record Number:CaltechAUTHORS:20170105-123015213
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20170105-123015213
Official Citation:Walter Willinger, David Alderson, and Lun Li. 2004. A pragmatic approach to dealing with high-variability in network measurements. In Proceedings of the 4th ACM SIGCOMM conference on Internet measurement (IMC '04). ACM, New York, NY, USA, 88-100. DOI=http://dx.doi.org/10.1145/1028788.1028800
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:73270
Collection:CaltechAUTHORS
Deposited By: Kristin Buxton
Deposited On:05 Jan 2017 21:09
Last Modified:03 Oct 2019 16:26

Repository Staff Only: item control page