Luan Nguyen
580 California St., Suite 400
San Francisco, CA, 94104
Clustering Web content for efficient replication
10th IEEE International Conference on Network Protocols, 2002. Proceedings.
https://doi.org/10.1109/ICNP.2002.1181397
Sign up for access to the world's latest research
Abstract
Recently there has been an increasing deployment of content distribution networks (CDNs) that offer hosting services to Web content providers. In this paper, we first compare the uncooperative pulling of Web contents used by commercial CDNs with the cooperative pushing. Our results show that the latter can achieve comparable users' perceived performance with only 4-5% of replication and update traffic compared to the former scheme. Therefore we explore how to efficiently push content to CDN nodes. Using trace-driven simulation, we show that replicating content in units of URLs can yield 60-70% reduction in clients' latency, compared to replicating in units of Web sites. However, it is very expensive to perform such a fine-grained replication. To address this issue, we propose to replicate content in units of clusters, each containing objects which are likely to be requested by clients that are topologically close. To this end, we describe three clustering techniques, and use various topologies and several large Web server traces to evaluate their performance. Our results show that the cluster-based replication achieves 40-60% improvement over the per Web site based replication. In addition, by adjusting the number of clusters, we can smoothly trade off the management and computation cost for better client performance. To adapt to changes in users' access patterns, we also explore incremental clusterings that adaptively add new documents to the existing content clusters. We examine both offline and online incremental clusterings, where the former assumes access history is available while the latter predicts access pattern based on the hyperlink structure. Our results show that the offline clusterings yield close to the performance of the complete re-clustering at much lower overhead. The online incremental clustering and replication cut down the retrieval cost by 4.6-8 times compared to no replication and random replication, so it is especially useful to improve document availability during flash crowds.
References (26)
- S. Jamin, C. Jin, A. Kurc, D. Raz, and Y. Shavitt, "Constrained mirror placement on the Internet," in Proc. of INFOCOM, 2001.
- L. Qiu, V. Padmanabhan, and G. Voelker, "On the placement of Web server replica," in Proc. of IEEE INFOCOM, 2001.
- B. Li et al., "On the optimal placement of Web proxies in the Internet," in Proc. of IEEE INFOCOM, 1999.
- L. Kaufman and P. J. Rousseeuw, Finding Groups in Data: An Intro- duction to Cluster Analysis, John Wiley & Sons, 1990.
- E. M. Voorhees, "Implementing agglomerative hierarchical clustering algorithms for use in document retrieval," Information Processing & Management, , no. 22, pp. 465-476, 1986.
- R. Ng and J. Han, "Efficient and effective clustering methods for data mining," in Proc. of VLDB, 1994.
- E. Cohen, B. Krishnamurthy, and J. Rexford, "Improving end-to-end performance of the Web using server volumes and proxy filters," in Proc. of ACM SIGCOMM, 1998.
- Z. Su et al., "Correlation-based document clustering using Web," in Proc. of the Int. conf. on System Sciences, 2001.
- M. Charikar, C. Chekuri, T. Feder, and R. Motwani, "Incremental clus- tering and dynamic information retrieval," in Proceedings of STOC, May 1997.
- T. Zhang, R. Ramakrishnan, and M. Livny, "BIRCH: An efficient data clustering method for very large databases," in Proceedings of SIG- MOD, 1996.
- E. Zegura, K. Calvert, and S. Bhattacharjee, "How to model an inter- network," in Proc. of IEEE INFOCOM, 1996.
- Y. Chen, L. Qiu, W. Chen, L. Nguyen, and R. Katz, "Clustering web content for efficient replication," in UCB/CSD Tech Report No. CSD- 02-1193, 2002.
- IPMA Project, "http://www.merit.edu/ipma".
- MSNBC, "http://www.msnbc.com".
- MediaMetrix, "http://www.mediametrix.com".
- NASA Kennedy Space Center Server Traces, "http://ita.ee.lbl.gov/html/contrib/nasa-http.html".
- B. Krishnamurthy and J. Wang, "On network-aware clustering of Web clients," in Proc. of ACM SIGCOMM, 2000.
- Akamai, "http://www.akamai.com".
- DigitalIsland, "http://www.digitalisland.com".
- A. Venkataramani et al., "The potential costs and benefits of long term prefetching for content distribution," in Proc. of Web Content Caching and Distribution Workshop, 2001.
- V. N. Padmanabhan and L. Qiu, "Content and access dynamics of a busy Web site: Findings and implications," in Proc. of ACM SIGCOMM, 2000.
- T. F. Gonzalez, "Clustering to minimize the maximum intercluster dis- tance," Theoretical Computer Science, vol. 38, pp. 293-306, 1985.
- J. Edachery, A. Sen, and F. Brandenburg, "Graph clustering using distance-k cliques," in Proc. of Graph Drawing, 1999.
- A. Adya, P. Bahl, and L. Qiu, "Analyzing browse patterns of mobile clients," in Proc. of SIGCOMM IMW, 2001.
- Alec Wolman et al., "Organization-based analysis of web-object sharing and caching," in Proc. of USITS, 1999.
- WebReaper, "http://www.webreaper.net".