Characterisation of Web traffic

2015

Trabajo presentado al XXX Simposium Nacional de la Union Cientifica Internacional de Radio, 2-4 de septiembre de 2015, Pamplona

Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems - SIGMETRICS '01, 2001

We report the results of a large-scale empirical study of web traffic. Our study is based on over 500 GB of TCP/IP protocolheader traces collected in 1999 and 2000 (approximately one year apart) from the high-speed link connecting The University of North Carolina at Chapel Hill to its Internet service provider. We also use a set of smaller traces from the NLANR repository taken at approximately the same times for comparison. The principal results from this study are: (1) empirical data suitable for constructing traffic generating models of contemporary web traffic, (2) new characterizations of TCP connection usage showing the effects of HTTP protocol improvement, notably persistent connections (e.g., about 50% of web objects are now transferred on persistent connections), and new characterizations of web usage and content structure that reflect the influences of "banner ads," server load balancing, and content distribution. A novel aspect of this study is a demonstration that a relatively light-weight methodology based o n passive tracing of only TCP/IP headers and off-line analysis tools can provide timely, high quality data about web traffic. We hope this will encourage more researchers to undertake ongoing data collection and provide the research community with data about the rapidly evolving characteristics of web traffic.

1998

The rapid growth of the World Wide Web in recent years has caused a significant shift in the composition of Internet traffic. Although past work has studied the behavior of TCP dynamics in the context of bulk-transfer applications and some studies have begun to investigate the interactions of TCP and HTTP, few have used extensive realworld traffic traces to examine the problem. This interaction is interesting because of the way in which current Web browsers use TCP connections: multiple concurrent short connections from a single host.

This dissertation deals with monitoring, collecting, analyzing, and modeling of World Wide Web (WWW) traffic and client interactions. The rapid growth of WWW usage has not been accompanied by an overall understanding of models of information resources and their deployment strategies. Consequently, the current Web architecture often faces performance and reliability problems. Scalability, latency, bandwidth, and disconnected operations are some of the important issues that should be considered when attempting to adjust for the growth in Web usage. The WWW Consortium launched an effort to design a new protocol that will be able to support future demands. Before doing that, however, we need to characterize current users' interactions with the WWW and understand how it is being used.

Proceedings of the 14th international conference on World Wide Web - WWW '05, 2005

We offer the first large-scale analysis of Web traffic based on network flow data. Using data collected on the Inter-net2 network, we constructed a weighted bipartite clientserver host graph containing more than 18 × 10 6 vertices and 68 × 10 6 edges valued by relative traffic flows. When considered as a traffic map of the WorldWide Web, the generated graph provides valuable information on the statistical patterns that characterize the global information flow on the Web. Statistical analysis shows that client-server connections and traffic flows exhibit heavy-tailed probability distributions lacking any typical scale. In particular, the absence of an intrinsic average in some of the distributions implies the absence of a prototypical scale appropriate for server design, Web-centric network design, or traffic modeling. The inspection of the amount of traffic handled by clients and servers and their number of connections highlights non-trivial correlations between information flow and patterns of connectivity as well as the presence of anomalous statistical patterns related to the behavior of users on the Web. The results presented here may impact considerably the modeling, scalability analysis, and behavioral study of Web applications.

Performance Evaluation, 2006

Most of the studies on TCP assume a fixed number of persistent TCP connections. A few of the recent studies look at non-persistent TCP connections but with many limiting assumptions. We look at a bottleneck router which has real time UDP traffic, TCP ON-OFF traffic and a stochastic flow of non-persistent TCP connections going through it. Our analysis is able to handle a large number (100s) of TCP connections and still provides the mean file download times of each of these connections when all of them have different packet sizes, round trip times and max window size. Using this approach we are able to model almost all the different traffic types that exist in the Internet today. We also study systems employing RED control, which is recommeded to be used in the Internet. We demostrate the effectiveness of RED for congestion control in realistic Internet scenarios both theoretically and via simulations.

2000

We discuss the performance effects of using per-transaction TCP connections for HTTP access, and the proposed optimizations of avoiding per-transaction re-connection and TCP slow-start restart overheads. We analyzed the performance penalties of the interaction of HTTP and TCP. Our observations indicate that the proposed optimizations do not affect Web access for the vast majority of users. Most users see end-to-end

International Journal of Business & Technology

With the rapid increase demand for data usage, Internet has become complex and harder to analyze. Characterizing the Internet traffic might reveal information that are important for Network Operators to formulate policy decisions, develop techniques to detect network anomalies, help better provision network resources (capacity, buffers) and use workload characteristics for simulations (typical packet sizes, flow durations, common protocols). In this paper, using passive monitoring and measurements, we show collected data traffic at Internet backbone routers. First, we reveal main observations on patterns and characteristics of this dataset including packet sizes, traffic volume for inter and intra domain and protocol composition. Second, we further investigate independence structure of packet size arrivals using both visual and computational statistics. Finally, we show the temporal behavior of most active destination IP and Port addresses.

2000

When performance studies about proxy cache server systems are made, one of the most common difficulties is to identify and to obtain representative workloads. Traces have been used as traditional workload. Gathering traces imply a large amount of time. If a self-similar traffic generator could be used, this problem would be solved, therefore evaluation studies become faster and more flexible. This work contains two parts; first, we perform a study of the self-similar property of several characteristics of the arrival collected traces, such as response size pattern, elapsed request time pattern and so on. Secondly, we model a source and develop a self-similar traffic arrival pattern generator.

The most common and wide spread activity on the internet is web browsing supported by Hypertext Transfer Protocol (HTTP) and Transmission Control Protocol (TCP). Therefore to estimate web user behavior and browsing experience a penetration look into HTTP/ TCP sessions is required. This paper is comprised of user interrupted and uninterrupted testing on some popular Web servers, Web browsers and Data objects (video, audio, image and text) to explore HTTP/TCP abnormal terminations (by reset flag) and its causes. The testing results in this paper supports that the anomaly of TCP port termination by reset flag is commonly caused by two Web browsers and end user. Also we infer that some Web servers also contribute to reset termination and the measure of reset termination rises while getting video and audio contents on the web in general. After quantifying TCP reset terminations and its causes, an improved criterion based on testing results is presented that can efficiently distinguish impatient user interruptions (reset terminations) from the network interruptions to estimate user behavior more precisely.