Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Conclusion
References
All Topics
Computer Science
Data Science

Search Engines Evaluation

Profile image of RAKESH KumarRAKESH Kumar

2005, DESIDOC Bulletin of Information Technology

https://doi.org/10.14429/DBIT.25.2.3649
visibility

…

description

8 pages

link

1 file

Abstract

The volume of world wide web ( WWW) is increasing enormously due to a world wide move to migrate information to online sources. To search some information on WWW, search engines are used, which when presented with queries, return a list of web pages ranked on the basis of estimation of relevance. Generally the search engines due to the abundance of information available on the web return millions of pages. But user studies indicate that a common user browses through top 10 or 20 documents only. So it's all-important to get into those top 10 documents. To achieve this web authors are increasingly beginning to rely on underhand techniques to ensure their sites get seen, in turn affecting the performance of search engines. The existing measures to evaluate these systems' performance are not adequate in the current world of highly interactive end-user systems. In this study a metric 'Ranked Precision' is proposed to evaluate the performance of search engines.

References (29)

  1. Belkin, N., and Croft, W.B. Retrieval techniques. Ann. Rev. Inf. Sci. and Tech. (ARIST), 1987, 22, 109-45.
  2. Pressman, S. Roger. Software engineering: A practitioner's approach. McGraw-Hill, 1997.
  3. IEEE Software Engineering Standards, Std. 610.12-1990, pp.47-48,1993.
  4. Cleverdon, C.W.; Mills, J.; & Keen, E.M, An inquiry in testing of information retrieval systems. Aslib Cranfield Research Project, College of Aeronautics, Cranfield, UK, 1966.
  5. Raghavan, V.; Bollman, P.; & Jung, G.S. A critical investigation of recall and precision as measures of retrieval system performance. Communication of the ACM, 1989, 7(3), 205-29.
  6. Saracevic, T. Evaluation of evaluation in information retrieval. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Special issue of SIGIR Forum, 1995, 138-46.
  7. Nielsen, J. User interface directions for the web. Communications of the ACM, 1999, 42 (1), 65-72.
  8. Chu, H., & Rosenthal, M. Search engine for the world wide web: A comparative study and evaluation methodology. In Proceedings of the Annual Conference for the American Society for Information Science, 19-24 October 1996, 127-35.
  9. Ding, Wei and Gary Marchionini. A comparative study of web search service performance. In Proceedings of the Annual Conference for the American Society for Information Science, 19-24 October 1996, 136-42.
  10. Westera, Gillian. Search engine comparison: Testing retrieval and accuracy. http://WWW.curtin.edu.au/curtin/library/ staffpages/ gwpersonal/senginestudy/ results.htm (7 February 1997).
  11. Leighton, H. Vernon. Performance of four world wide web ( WWW) index services: Infoseek, Lycos, WebCrawler, and WWWWorm. 1995. http://WWW.winona.edulibrary//webind.h tm (1 July 1996).
  12. Gauch, Susan and Wang, Guijun. Information fusion with profusion. Webnet 96 Conference, San Francisco, CA, October 15-19, 1996. http://WWW.csbs.utsa.edu:80/info/webn et96/html/155.htm [22 February 1997].
  13. Tomaiuolo, Nicholas G. & Packer, Joan G. An analysis of Internet search engines: Assessment of over 200 search queries. Computers in Libraries, 1996, 16 (6), 58. The list of queries used is in: Quantitative Analysis of Five WWW 'Search Engines' http://neal. ctstateu. edu: 2001/ htdocs/ websearch.html (7 February 1997).
  14. Scoville, Richard. Special report: Find it on the net! PC World, 1996, 14(1), 125-26. http:// WWW. pcworld. com/ reprints/ lycos.htm (1 February 1997).
  15. Meghabghab, D.B. and Meghabghab, G.V. Information retrieval in cyberspace. Proceedings of ASIS Mid-Year Meeting, 1996.
  16. Holscher, C. & Strube, G. Web search behaviour of internet experts and newbies. International Journal of Computer and Telecommunication Networking , 2000, 33, 337-46.
  17. Silverstein C.; Henzinger, M.; Marais, H.; & Moricz, M. Analysis of a very large web search engine query log. SIGIR Forum, 1999, 33, 6-12.
  18. Jansen, B.J.; Spink, A.; & Saracevic, T. Real life, real users, and real needs: A study and analysis of user queries on the web. Information Processing and Management, 2000, 36, 207-27.
  19. Markatos, E.P. On caching search engine query results. In Proceedings of the 5th Inter-national Web Caching and Content Delivery Workshop, May 2000.
  20. Lempel, R.; & Moran, S. Predictive caching and prefetching of query results in search engines. In Proceedings of the 12th World Wide Web Conference ( WWW2003), Budapest, Hungary, 2003.
  21. Broder, A. Web search. Invited talk at the 9th Text Retrieval Conference (TREC-9), Gaithersburg, Maryland, November 2000.
  22. Marchiori, M., The quest for correct information on the web: Hyper search engines. In Proceedings of 6th International Conference on WWW, 1997.
  23. Lempel, R. & Moran, S. SALSA: Stochastic approach for link structure analysis. ACM Transactions on Information Systems, 2001, 19(2), 131-60.
  24. Chakrabarti, S., et al. Hypersearching the web. Sci Am, 1999.
  25. Papadimitriou, C.H.; Tamaki H.; Raghavan, P.; & Vempala S. Latent semantic indexing: a probabilistic analysis. In Proceedings of the 17th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, 1998.
  26. Ejiogu, L. Software engineering with formal metrics, QED Publishing, 1991.
  27. Roche, J.M. Software metrics and measurement principle. ACM Software Engineering Notes, 1994, 19 (1), 76-85.
  28. Basili, V.R., and Weiss, D.M. A methodology for collecting valid software engineering data. IEEE Trans. Software Engineering, 1984, SE-10 , 728-38.
  29. DESIDOC Bulletin of Inf Technol, 2005, 25 (2)

Related papers

Performance Evaluation of Selected Search Engines on the Internet
Adebanjo Adekiigbe

2008

Search Engines have become an integral part of daily internet usage. The search engine is the first stop for web users when they are looking for a product. Information retrieval may be viewed as a problem of classifying items into one of two classes corresponding to interesting and uninteresting items respectively. A natural performance metric in this context is classification accuracy, defined as the fraction of the system's interesting/uninteresting predictions that agree with the user's assessments. On the other hand, the field of information retrieval has two classical performance evaluation metrics: precision, the fraction of the items retrieved by the system that are interesting to the user, and recall, the fraction of the items of interest to the user that are retrieved by the system. Measuring the information retrieval effectiveness of World Wide Web search engines is costly because of human relevance judgments involved. However, both for business enterprises and people it is important to know the most effective Web search engines, since such search engines help their users find higher number of relevant Web pages with less effort. Furthermore, this information can be used for several practical purposes. This study evaluates the performance of three Web search engines. A set of measurements is proposed for evaluating Web search engine performance.

View PDFchevron_right
IOSR Journal of Engineering (IOSRJEN) Performance Evaluation of Selected Search Engines
Ajayi Olusola Olajide

Search Engines have become an integral part of daily internet usage. The search engine is the first stop for web users when they are looking for a product. Information retrieval may be viewed as a problem of classifying items into one of two classes corresponding to interesting and uninteresting items respectively. A natural performance metric in this context is classification accuracy, defined as the fraction of the system's interesting/uninteresting predictions that agree with the user's assessments. On the other hand, the field of information retrieval has two classical performance evaluation metrics: precision, the fraction of the items retrieved by the system that are interesting to the user, and recall, the fraction of the items of interest to the user that are retrieved by the system. Measuring the information retrieval effectiveness of World Wide Web search engines is costly because of human relevance judgments involved. However, both for business enterprises and people it is important to know the most effective Web search engines, since such search engines help their users find higher number of relevant Web pages with less effort. Furthermore, this information can be used for several practical purposes. This study evaluates the performance of three Web search engines. A set of measurements is proposed for evaluating Web search engine performance.

View PDFchevron_right
Towards information retrieval measures for evaluation of Web search engines
Jacek Gwizdka

Unpublished manuscript, 1999

Information retrieval on the Web is very different from retrieval in traditional indexed databases. This difference arises from: the high degree of dynamism of the Web; its hyper-linked character; the absence of a controlled indexing vocabulary; the heterogeneity of document types and authoring styles; the easy access that different types of users may have to it. Thus, since Web retrieval is substantially different from information retrieval, new or revised evaluative measures are required to assess retrieval performance using Web search engines. This paper suggests a number of different measures to evaluate information retrieval from the Web. The motivation behind each of these measures is presented, along with their descriptions and definitions. In the second part of the paper, application of these measures is illustrated in the evaluation of three search engines. The purpose of this paper is not to give the definite prescription for evaluating information retrieval from the Web, but rather to present some examples and to initiate a wider discussion of how to enhance measures of Web search performance.

View PDFchevron_right
Automatic performance evaluation of Web search engines
Rashid Ali

Measuring the information retrieval effectiveness of World Wide Web search engines is costly because of human relevance judgments involved. However, both for business enterprises and people it is important to know the most effective Web search engines, since such search engines help their users find higher number of relevant Web pages with less effort. Furthermore, this information can be used for several practical purposes. In this study we introduce automatic Web search engine evaluation method as an efficient and effective assessment tool of such systems. The experiments based on eight Web search engines, 25 queries, and binary user relevance judgments show that our method provides results consistent with human-based evaluations. It is shown that the observed consistencies are statistically significant. This indicates that the new method can be successfully used in the evaluation of Web search engines.

View PDFchevron_right

Related topics

  • Computer Science
  • Information Retrieval
  • Library and Information Studies
  • Web Search Engine
  • Organic Search
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved