Web Search Engine

Abstr act. The Internet has become for many the most important medium for staying informed about current news events. Some events cause heightened interest on a topic, which in turn yields a higher frequency of the search queries related to it. These queries are going through a "query burst". In this paper we examine the behavior of search engine users during a query burst, compared to before and after the burst. We are interested in how this behavior changes, and how it affects other stake-holders in web search. We analyze one year of web-search and news-search logs, looking at query bursts from multiple perspectives. First, we adopt the perspective of search engine users, describing changes in their effort and interest while searching. Second, we adopt the perspective of news providers by comparing web search and news search query bursts. Third, we look at the burst from the perspective of content providers. We study the conditions under which content providers can "ride" a wave of increased interest on a topic, and obtain a share of the user's increased attention. We do so by identifying the class of queries that can be considered as an opportunity for content providers that are "late-comers" for a query, in the sense of not being among the first to write about its topic. We also present a simple model for predicting the click share content providers could obtain if they decide to provide content about these queries.

Information Retrieval Systems (IRS) are so popular through World Wide Web. Availability of Text Information related to all types of objects like Documents, Web Pages,Images, Videos and Audio files on web are increasing day by day in an exponential manner. When the text repository grows to the maximum extent of the memory size in the server, the methods used to

This paper reports the results of a study conducted to explore and compare the features of independently built Malaysian Web search engines, as well as evaluate their performance and search capabilities. Four Malaysian independently built search engines were chosen for the study and Google served as the benchmark engine. This study applied two co-related methodologies, namely Information Retrieval Evaluation Methodology and common features comparison. The Information Retrieval Evaluation Methodology measures include search capabilities, retrieval performance (recall & precision) and user effort. Results show that each search engine portrays different strengths and weaknesses. Malaysia Directory appears to be the best Malaysian Web search engine in term of feature presentation but fails in term of retrieval performance; whereas Malaysia Central managed to score the highest for mean precision but failed to excel in terms of feature presentation. SajaSearch outperformed Cari in most related documents, even though Cari did return the highest total hits. Overall comparison with Google leaves Malaysian search engines with much to improve on in terms of performance and search capabilities. Implications for the design of Malaysian Web Search Engines are also discussed.

In online affiliate marketing networks advertising web sites offer their affiliates revenues based on provided web site traffic and associated leads and sales. Advertising web sites can have a network of thousands of affiliates providing them with web site traffic through hyperlinks on their web sites. Search engines such as Google, MSN, and Yahoo, consider hyperlinks as a proof of quality and/or reliability of the linked web sites, and therefore use them to determine the relevance of web sites with regard to search queries. In this research we investigate the potential impact of online affiliate marketing networks on the ranking of advertisers' web sites in search results. This article empirically explores how seven different affiliate marketing networks affect the rankings of the advertising web sites within web search engines. The field study followed intensively seven online affiliate marketing networks for twelve weeks after their launch. The results indicate that newly started affiliate networks effectively improve the rankings of advertising web sites in search engine results. Also, it was found that the effects of affiliate marketing networks on search engine rankings were smaller for advertising web sites operating in highly competitive markets. Another finding was that a growth in visitors coming from search engines was present as a result of the improvement of search engine rankings. Finally, the results indicate that cost-benefit metrics associated with affiliate marketing programs, such as the average marketing cost will decrease when the positive effects of affiliate marketing on search engine rankings are taken into account.

AOIS is a multi-agent system that supports the sharing of information among a community of users connected through the Internet. In respect to Web search engines, this system enhances the search through domain ontologies, avoids the burden of publishing the information on the Web and guaranties a controlled and dynamic access to the information. The use of agent technologies has made the realization of three of the main features of the system straightforward: i) filtering of information coming from different users, on the basis of the previous experience of the local user, ii) pushing of new information that can be of interest for a user, and iii) delegation of access capabilities, on the basis of a reputation network, built by the agents of the system on the community of its users.

Search Engine for South-East Europe (SE4SEE) is a socio-cultural search engine running on the grid infrastructure. It offers a personalized, on-demand, country-specific, category-based Web search facility. The main goal of SE4SEE is to attack the page freshness problem by performing the search on the original pages residing on the Web, rather than on the previously fetched copies as done in the traditional search engines. SE4SEE also aims to obtain high download rates in Web crawling by making use of the geographically distributed nature of the grid. In this work, we present the architectural design issues and implementation details of this search engine. We conduct various experiments to illustrate performance results obtained on a grid infrastructure and justify the use of the search strategy employed in SE4SEE.

Search Engine for South-East Europe (SE4SEE) is an application project aiming to develop a grid-enabled search engine that specifically targets the countries in the South-East Europe. It is one of the two selected regional applications currently implemented in the SEE-GRID FP6 project. This paper describes the design details of SE4SEE and provides an architectural overview of the application.

In recent years, we have witnessed the proliferation of applications that generate thousands of terabytes of data per day, due to the explosive increase in storage capacity across various devices. As a consequence, a new concept called Data Deluge has emerged. Data deluge refers to the situation where the quantity of data generated exceeds the processing power available, and spatio-temporal data is no exception to this phenomenon. In this context, the efficient processing of spatio-temporal queries becomes crucial to address this challenge, as slow query processing can result in obsolete answers, which may lead to errors. Considering this dynamic context of storage and processing, we explore a new online workload algorithm in a distributed parallel environment using hybrid spatio-temporal indexes. This algorithm is able to update the indexes with the most appropriate data, aiming to achieve more efficient query processing. To measure the efficiency of this algorithm, we present its time complexity along with an empirical evaluation of its performance, considering processing time, number of accessed nodes, and communication costs. The empirical results show a significant reduction in processing time, communication costs, and number of accessed nodes.

During the last years web search engines have moved from the simple but inefficient syntactical analysis (first generation) to the more robust and usable web graph analysis (second generation). Much of the current research is focussed on the so-called third generation search engines that, in principle, inject "human characteristics" on how results are obtained and presented to the end user. Approaches exploited towards this direction include (among others): an alteration of PageRank [1] that takes into account user specific characteristics and bias the page ordering using the user preferences (an approach, though, that does not scale well with the number of users). The approach is further exploited in , where several PageRanks are computed for a given number of distinct search topics. A similar idea is used in , where the PageRank computation takes into account the content of the pages and the query terms the surfer is looking for. In , a decomposition of PageRank to basic components is suggested that may be able to scale the different PageRank computations to a bigger number of topics or even distinct users. Another approach to web search is presented in , where a rich extension of the web, called semantic web, and the application of searching over this new setting is described. In this work we depart from the above lines of research and propose a new conceptual framework for representing the web and potentially improving search results. In particular, the new framework views the web as a collection of webrelated data collected and semi-organized by individual users inside their information spaces. These data can be explicitly collected (e.g., bookmarks) or implicitly collected (e.g., web-browsing history). Our approach is based on the observation that users act as small crawlers seeking information on the web using various media (search engines, catalogs, word-of-mouth, hyperlinks, direct URL typing, etc). They tend to store and organize important-for-them pages in tree-like structures, referred to as bookmark collections, where the folder names act as tags over the collected URLs. This method of organizing data helps people to recall collected URLs faster, but can also be used as a kind of semantic tagging over the URLs (the path to the URL can be perceived as different ways to communicate the URL itself). This information constitutes part of the user's personal information space and it is indicative of his interests. One might argue that people do not collect bookmarks or that they do not organize them in any reasonable manner. However, as our experiments show, people indeed collect and organize bookmarks under certain patterns that follow power law distributions.

Thinking of today's web search scenario which is mainly keyword based, leads to the need of effective and meaningful search provided by Semantic Web. Existing search engines are vulnerable to provide relevant answers to users query due to their dependency on simple data available in web pages. On other hand, semantic search engines provide efficient and relevant results as the semantic web manages information with well defined meaning using ontology. A Meta-Search engine is a search tool that forwards users query to several existing search engines and provides combined results by using their own page ranking algorithm. SemanTelli is a meta-semantic search engine that fetches results from different semantic search engines such as Hakia, DuckDuckGo, SenseBot through intelligent agents. This paper proposes enhancement of SemanTelli with improved snippet analysis based page ranking algorithm and support for image and news search.

Online questionnaire-based research is growing at a fast pace. Mouse-tracking methods provide a potentially important data source for this research by enabling the capture of respondents' online behaviour while answering questionnaire items. This behaviour can give insight into respondents' perceptual, cognitive and affective processes. The present work focused on the potential use of mouse movements to indicate uncertainty when answering questionnaire items and used machine learning methods as a basis to model these. N=79 participants completed an online questionnaire while mouse data was tracked. Mouse movement features were extracted and selected for model training and testing. Using logistic regression and k-fold cross-validation, the model achieved an estimated performance accuracy of 89%. The findings show that uncertainty is indicated by an increase in the number of horizontal direction inversions and the distance covered by the mouse and by longer interaction times with and a higher number of revisits to questionnaire items that evoked uncertainty. Future work should validate these methods further.

There is growing interest in the field of human-computer interaction in the use of mouse movement data to infer e.g. user's interests, preferences and personality. Previous work has defined various patterns of mouse movement behavior. However, there is a paucity of mouse tracking measures and defined movement patterns for use in the specific context of data collection with online surveys. The present study aimed to define and visualize patterns of mouse movements while the user provided responses in a survey (with questions to be answered using a 5-point Likert response scale). The study produced a wide range of different patterns, including new patterns, and showed that these can easily be distinguished. The identified patterns may -in conjunction with machine learning algorithms -be used for further investigation toward e.g. the recognition of the user's state of mind or for user studies.

The volume of world wide web ( WWW) is increasing enormously due to a world wide move to migrate information to online sources. To search some information on WWW, search engines are used, which when presented with queries, return a list of web pages ranked on the basis of estimation of relevance. Generally the search engines due to the abundance of information available on the web return millions of pages. But user studies indicate that a common user browses through top 10 or 20 documents only. So it's all-important to get into those top 10 documents. To achieve this web authors are increasingly beginning to rely on underhand techniques to ensure their sites get seen, in turn affecting the performance of search engines. The existing measures to evaluate these systems' performance are not adequate in the current world of highly interactive end-user systems. In this study a metric 'Ranked Precision' is proposed to evaluate the performance of search engines.

The Internet is increasingly used to find health information worldwide. Online health information search can be beneficial for novice users but due to the overwhelming quantity and uneven quality of online health information it may also be a threat. Although medical professionals are qualified to evaluate information quality, they can be overwhelmed with its quantity and it is impossible to be up to date even in a specialized field. However, the main threat is information quality. The main initiatives in online health information quality are explored, and the HONcode certification is described. The workshop will also detail other aspects of medical information search, text retrieval using ontologies for semantic text analysis and visual retrieval to include images and videos. The EU project KHRESMOI will be presented -it aims at using semantic, textual and visual information retrieval targeted at three main groups: the general population, GPs and radiologists. Time is foreseen for discussion among participants.

The volume of world wide web ( WWW) is increasing enormously due to a world wide move to migrate information to online sources. To search some information on WWW, search engines are used, which when presented with queries, return a list of web pages ranked on the basis of estimation of relevance. Generally the search engines due to the abundance of information available on the web return millions of pages. But user studies indicate that a common user browses through top 10 or 20 documents only. So it's all-important to get into those top 10 documents. To achieve this web authors are increasingly beginning to rely on underhand techniques to ensure their sites get seen, in turn affecting the performance of search engines. The existing measures to evaluate these systems' performance are not adequate in the current world of highly interactive end-user systems. In this study a metric 'Ranked Precision' is proposed to evaluate the performance of search engines.

(147) One of the major problems in the process of Information Retrieval (IR) arises at the stage where the user reviews the results list. This paper presents the latest research in a series of research works that aims at finding the most vital information components, within a list of search results, so as to assist the user in high-quality decision making as to which of the resulting documents are included within the sought after results of the search task We propose here a new model for displaying the results named TTLS (Taxonomy Tree & LCC&K Snippet). The experimentation setup included execution of different search tasks by a group of 60 participants. The tasks were performed via the BASE and TTLS interfaces. From the resulting times comparison it is clear that the execution times of tasks done via the TTLS interface is shorter that those done via the BASE interface. It can be seen that in the BASE interface it was needed to open more documents in order to locate the relevant information than in the TTLS interface. It turns out that the majority of users (77%) prefer to use the TTLS interface.

(140) One of the major problems in the process of Information Retrieval (IR) arises at the stage where the user reviews the results list. This paper presents the latest research in a series of research works that aims at finding the most vital information components, within a list of search results, so as to assist the user in high-quality decision making as to which of the resulting documents are included within the sought after results of the search task We propose here a new model for displaying the results named TTLS (Taxonomy Tree & LCC&K Snippet). The experimentation setup included execution of different search tasks by a group of 60 participants. The tasks were performed via the BASE and TTLS interfaces. From the resulting times comparison it is clear that the execution times of tasks done via the TTLS interface is shorter that those done via the BASE interface. It can be seen that in the BASE interface it was needed to open more documents in order to locate the relevant information than in the TTLS interface. It turns out that the majority of users (77%) prefer to use the TTLS interface.

Frequent requests from users to search engines on the World Wide Web are to search for information about people using personal names. Current search engines only return sets of documents containing the name queried, but, as several people usually share a personal name, the resulting sets often contain documents relevant to several people. It is necessary to disambiguate people in these result sets in order to to help users find the person of interest more readily. In the task of name disambiguation, effective measurement of similarities in the documents is a crucial step towards the final disambiguation. We propose a new method that uses web directories as a knowledge base to find common contexts in documents and uses the common contexts measure to determine document similarities. Experiments, conducted on documents mentioning real people on the web, together with several famous web directory structures, suggest that there are significant advantages in using web directories to disambiguate people compared with other conventional methods.

In this paper we present a semi-supervised learning method for a problem of learning to rank where we exploit Markov random walks and graph regularization in order to incorporate not only "labeled" web pages but also plenty of "unlabeled" web pages (click logs of which are not given) into learning a ranking function. In order to cope with scalability which existing semi-supervised learning methods suffer from, we develop a scalable and incremental method for semi-supervised learning to rank. In the graph regularization framework, we first determine features which well reflects data manifold and then make use of them to train a linear ranking function. We introduce a matrix-fee technique where we compute the eigenvectors of a huge similarity matrix without constructing the matrix itself. Then we present an incremental algorithm to learn a linear ranking function using features determined by projecting data onto the eigenvectors of the similarity matrix, which can be applied to a task of web-scale ranking. We evaluate our method on Live Search query log, showing that search performance is much improved when Live Search yields unsatisfactory search results.

Search engines have become an integral part of our lives. To augment the power of such engines-even while offline-was our goal. To accomplish this, a distinct offline search engine was created to retrieve data from archives. An updated UI was achieved to make interacting with the software more appealing and convenient. A spell check technique to recommendations emanated from the assessed query is the most major addition to the interface. The general goal is to make offline search engines more accessible to use. To provide satisfactory results with the spell check mechanism, to use a binary converter so that the query correction and evaluation become even more accurate.

World Wide Web is a collection of online resources and websites, including e-commerce, social sites, educational content, etc. To find relevant online resources, people search these by using search engines by providing their desired keywords. After filtering those keywords, search engine list the most relevant websites which are more optimized and efficient in terms of loading speed. Search engine optimization (SEO) is a set of techniques used to make a website optimized and relevant to those keywords and set the website's rank. An online resource or website will be on the top of the search result set if it has a higher rank in search engines. Page speed is one of the most important on-page search engine optimization techniques that is used to make web site efficient in load time, so the user will get the content of the websites in a minimum time. Google has set page speed as the main factor in a higher ranking in search engines. Getting higher page speed is not an easy task, as several performance matrices must be optimized to get efficient loading time. There are many audit rules which are irrelevant or have less impact on the performance score. So selecting audit rules to be optimized is one of the main decisions before starting page speed optimization work. It will be a waste of time to investigate audit rules for their impact on performance scores. In this paper, we have analyzed all of the audit rules and identified the most important and relevant audit rules in optimizing page speed. A tool is used to generate the best sequence of relevant audit rules based on weighted performance benefit scores in the execution of each audit rule. The same audit rule sequence is applied on five different websites and results in more than 80% improvements in performance scores by applying the first three audit rules only and above 90% performance scores obtained by using the first five to seven audit rules in our proposed audit rules sequence.

As the World Wide Web is growing rapidly and data in the present day scenario is stored in a distributed manner. The need to develop a search Engine based architectural model for people to search through the Web. Broad web search engines as well as many more specialized search tools rely on web crawlers to acquire large collections of pages for indexing and analysis. The crawler is an important module of a web search engine. The quality of a crawler directly affects the searching quality of such web search engines. Such a web crawler may interact with millions of hosts over a period of weeks or months, and thus issues of robustness, flexibility, and manageability are of major importance. Given some URLs, the crawler should retrieve the web pages of those URLs, parse the HTML files, add new URLs into its queue and go back to the first phase of this cycle. The crawler also can retrieve some other information from the HTML files as it is parsing them to get the new URLs. In this paper, we describe the design of a web crawler that uses Page Rank algorithm for distributed searches and can be run on a network of workstations. The crawler initially search for all the stop words (such as a, an, the, and etc). While searching the web pages for some keyword the crawler will initially remove all collected stop word. Also at the same time the crawler will search for snippets from web documents. All the matching word & collected snippet will be stored in temporary cache memory created at central server of crawlers. Where after applying page rank algorithm on the basis of no. of visit of web pages we will arrange the pages according to their ranks & display the results. Since, due to extensive search on web through web crawlers the chances of various virus attacks are more & processing capacity of system may get halt so to provide solution in such scenario we can provide backup to our system by creating web services. The web service will be designed in such manner that any valid updations to any database servers will automatically updates the backup servers. Therefore, even in failure of any server system, we can continue with crawling process.

As the World Wide Web is growing rapidly and data in the present day scenario is stored in a distributed manner. The need to develop a search Engine based architectural model for people to search through the Web. Broad web search engines as well as many more specialized search tools rely on web crawlers to acquire large collections of pages for indexing and analysis. The crawler is an important module of a web search engine. The quality of a crawler directly affects the searching quality of such web search engines. Such a web crawler may interact with millions of hosts over a period of weeks or months, and thus issues of robustness, flexibility, and manageability are of major importance. Given some URLs, the crawler should retrieve the web pages of those URLs, parse the HTML files, add new URLs into its queue and go back to the first phase of this cycle. The crawler also can retrieve some other information from the HTML files as it is parsing them to get the new URLs. In this paper, we describe the design of a web crawler that uses Page Rank algorithm for distributed searches and can be run on a network of workstations. The crawler initially search for all the stop words (such as a, an, the, and etc). While searching the web pages for some keyword the crawler will initially remove all collected stop word. Also at the same time the crawler will search for snippets from web documents. All the matching word & collected snippet will be stored in temporary cache memory created at central server of crawlers. Where after applying page rank algorithm on the basis of no. of visit of web pages we will arrange the pages according to their ranks & display the results. Since, due to extensive search on web through web crawlers the chances of various virus attacks are more & processing capacity of system may get halt so to provide solution in such scenario we can provide backup to our system by creating web services. The web service will be designed in such manner that any valid updations to any database servers will automatically updates the backup servers. Therefore, even in failure of any server system, we can continue with crawling process.

We propose a simple approach to search engine personalization based on Web communities . User information -in particular, the Web communities whose neighborhoods the user has selected in the past-is used to change the order of the returned search results. We present experimental evidence suggesting that our method indeed improves the quality of the ranking. Our experiments were carried out on a search engine created by our research group and focusing on the Greek fragment of the worldwide Web (1.33 million documents); we also discuss the issue of scaling.

Federated search has the potential of improving web search: the user becomes less dependent on a single search provider and parts of the deep web become available through a unified interface, leading to a wider variety in the retrieved search results. However, a publicly available dataset for federated search reflecting an actual web environment has been absent. As a result, it has been difficult to assess whether proposed systems are suitable for the web setting. We introduce a new test collection containing the results from more than a hundred actual search engines, ranging from large general web search engines such as Google and Bing to small domain-specific engines. We discuss the design and analyze the effect of several sampling methods. For a set of test queries, we collected relevance judgements for the top 10 results of each search engine. The dataset is publicly available and is useful for researchers interested in resource selection for web search collections, result merging and size estimation of uncooperative resources.

Many experts predict that the next huge step forward in Web information technology will be achieved by adding semantics to Web data, and will possibly consist of (some form of) the Semantic Web. In this paper, we present a novel approach to Semantic Web search, called Serene, which allows for a semantic processing of Web search queries, and for evaluating complex Web search queries that involve reasoning over the Web. More specifically, we first add ontological structure and semantics to Web pages, which then allows for both attaching a meaning to Web search queries and Web pages, and for formulating and processing ontology-based complex Web search queries (i.e., conjunctive queries) that involve reasoning over the Web. Here, we assume the existence of an underlying ontology (in a lightweight ontology language) relative to which Web pages are annotated and Web search queries are formulated. Depending on whether we use a general or a specialized ontology, we thus obtain a general or a vertical Semantic Web search interface, respectively. That is, we are actually mapping the Web into an ontological knowledge base, which then allows for Semantic Web search relative to the underlying ontology. The latter is then realized by reduction to standard Web search on standard Web pages and logically completed ontological annotations. That is, standard Web search engines are used as the main inference motor for ontologybased Semantic Web search. We develop the formal model behind this approach and also provide an implementation in desktop search. Furthermore, we report on extensive experiments, including an implemented Semantic Web search on the Internet Movie Database.

The volume of world wide web ( WWW) is increasing enormously due to a world wide move to migrate information to online sources. To search some information on WWW, search engines are used, which when presented with queries, return a list of web pages ranked on the basis of estimation of relevance. Generally the search engines due to the abundance of information available on the web return millions of pages. But user studies indicate that a common user browses through top 10 or 20 documents only. So it's all-important to get into those top 10 documents. To achieve this web authors are increasingly beginning to rely on underhand techniques to ensure their sites get seen, in turn affecting the performance of search engines. The existing measures to evaluate these systems' performance are not adequate in the current world of highly interactive end-user systems. In this study a metric 'Ranked Precision' is proposed to evaluate the performance of search engines.

This paper introduces the reader to the approach we are taking to develop an ontology that could be used to represent the knowledge inherent in filmed materials. Such an ontology could be used as the semantic basis for multimedia retrieval systems. The proposed approach to ontology development is informed by the earlier work of researchers into folksonomy development and facet analysis. A brief survey of this earlier work is presented before our approach is described.

Most of the ontology alignment tools use terminological techniques as the initial step and then apply the structural techniques to refine the results. Since each terminological similarity measure considers some features of similarity, ontology alignment systems require exploiting different measures. While a great deal of effort has been devoted to developing various terminological similarity measures and also developing various ontology alignment systems, little attention has been paid to develop similarity search algorithms which exploit different similarity measures in order to gain benefits and avoid limitations. We propose a novel terminological search algorithm which tries to find an entity similar to an input search string in a given ontology. This algorithm extends the search string by creating a matrix from its synonym and hypernyms. The algorithm employs and combines different kind of similarity measures in different situations to achieve a higher performance, accuracy, and stability in comparison with previous methods which either use one measure or combine more measures in a naive ways such as averaging. We evaluated the algorithm using a subset of OAEI Bench mark data set. Results showed the superiority of proposed algorithm and effectiveness of different applied techniques such as word sense disambiguation and semantic filtering mechanism.

A new indexing method. called Compressed Multi-Framed Signature File (C-MFSF). that uses a partial query evaluation strategy with compressed signature bit slices is presented. In C-MFSF. a signature tile is divided into variable sized compressed vertical frames with different on-bit densities to optimize the response time. Experiments with a real database of 152,850 records show that a response time less than I50 milliseconds is possible. For multi-term queries C-MFSF obtains the query results with fewer disk accesses than the inverted tiles. The method requires no indexing vocabulary. These attributes have important implications; for example, web search engines process multi-term queries in very large databases with sizeable vocabularies.

In this paper, we address the problem of selection collections. This is important for locating responses in digital libraries. The aim of methods, which deal with the area of information retrieval, is to reduce the amount of the exchanged messages by selecting the best servers from the beginning of the search. We propose a new function of selection based on statistics. Our function takes into account the relevance degree of documents in order to rank collections. We have implemented and compared our function with other methods. The experimentations have shown that our proposition is very competitive.

The Semantic Web is usually envisaged as a collection of Web acces- sible RDF documents that re-use RDF schemas. These schemas are expected to be most often independently designed and hence not sharing many categories. We are unconvinced that this approach is viable because the lack of semantic relation- ships between the categories will most often make it impossible for

In this paper we propose a novel parallel algorithm for frequent itemset mining. The algorithm is based on the filter-stream programming model, in which the frequent itemset mining process is represented as a data flow controlled by a series of producer and consumer components (called filters), and the data flow (communication) between such filters is made via streams. When production rate matches consumption rate, and communication overhead between producer and consumer filters is minimized, a high degree of asynchrony is achieved. Following this strategy, our algorithm employs an asynchronous candidate generation, and minimizes communication between filters by transferring only the necessary aggregated information. Another nice feature of our algorithm is a look forward approach which accelerates frequent itemset determination. Extensive evaluation shows the parallel performance and scalability of our algorithm.

Search engines, an information retrieval tool are the main source of information for users' information need now a day. For every query, the search engine explores its repository and/or indexer to find the relevant documents/URLs for that query. Page ranking algorithms rank the Uniform Resource Locator in abstract section (URLs) according to its relevancy with respect to users' query. It is analyzed that many of the queries fired by users on search engines are duplicate. There is a scope to improve the performance of search engine to reduce its efforts for duplicate queries. In this paper a proxy server is created that keep store the search results of user queries in web log. The proposed proxy server uses this web log to find results faster for duplicate queries fired next time. The proposed scheme has been tested and found prominent. The proposed architecture tested for ten duplicate user queries. it return all relevant web pages for duplicate user query (if query is found in web log at proxy server) from a particular domain instead of entire database. It reduces the perceived latency for duplicate query and also improves the value of precession and accuracy up to 81.8% and 99% respectively for all duplicate user queries.

Purpose: Due to the exponential growth of internet users and internet traffic, information seekers are highly dependent upon search engines to extract relevant information. Due to the accessibility of a large amount of textual, audio, video etc. contents, the responsibility of search engines has increased.Design/methodology/approach: The search engine provides relevant information to internet users concerning to their query; based on content, link structure etc. However, it does not provide the guarantee of the correctness of the information. The performance of a search engine is highly dependent upon the ranking module. The performance of the ranking module is dependent upon the link structure of web pages, which analyse through Web structure mining (WSM) and their content, which analyses through Web content mining (WCM). Web mining plays an important role in computing the rank of web pages.Findings: In this article, web mining types, techniques, tools, algorithms and their challen...

In this paper, we put forward a technique for parallel crawling of the web. The World Wide Web today is growing at a phenomenal rate. It has enabled a publishing explosion of useful online information, which has produced the unfortunate side effect of information overload. The size of the web as on February 2007 stands at around 29 billion pages. One of the most important uses of crawling the web is for indexing purposes and keeping web pages up-to-date, later used by search engine to serve the end user queries. The paper puts forward an architecture built on the lines of client server architecture. It discuses a fresh approach for parallel crawling the web using multiple machines and integrates the trivial issues of crawling also. A major part of the web is dynamic and hence, a need arises to constantly update the changed web pages. We have used a three-step algorithm for page refreshment. This checks for whether the structure of a web page has been changed or not, the text content has been altered or whether an image is changed. For The server we have discussed a unique method for distribution of URLs to client machines after determination of their priority index. Also a minor variation to the method of prioritizing URLs on the basis of forward link count has been discussed to accommodate the purpose of frequency of update.

World Wide Web (WWW) is a huge repository of interlinked hypertext documents known as web pages. Users access these hypertext documents via Internet. Since its inception in 1990, WWW has become many folds in size, and now it contains more than 50 billion publicly accessible web documents distributed all over the world on thousands of web servers and still growing at exponential rate. It is very difficult to search information from such a huge collection of WWW as the web pages or documents are not organized as books on shelves in a library, nor are web pages completely catalogued at one central location. Search engine is basic information retrieval tool, used to access information from WWW. In response to the search query provided by users, Search engines use their database to search the relevant documents and produce the result after ranking on the basis of relevance. In fact, the Search engine builds its database, with the help of WebCrawlers. To maximize the download rate and to retrieve the whole or significant portion of the Web, search engines run multiple crawlers in parallel. Overlapping of downloaded web documents, quality, network bandwidth and refreshing of web documents are the major challenging problems faced by existing parallel WebCrawlers that are addressed in this work. A Multi Threaded (MT) server based novel architecture for incremental parallel web crawler has been designed that helps to reduce overlapping, quality and network bandwidth problems. Additionally, web page change detection methods have been developed to refresh the web document by detecting the structural, presentation and content level changes in web documents. These change detection methods help to detect whether version of a web page, existing at Search engine side has got changed from the one existing at Web server end or not. If it has got changed, the WebCrawler should replace the existing version at Search engine database side to keep its repository up-to-date

The volume of world wide web ( WWW) is increasing enormously due to a world wide move to migrate information to online sources. To search some information on WWW, search engines are used, which when presented with queries, return a list of web pages ranked on the basis of estimation of relevance. Generally the search engines due to the abundance of information available on the web return millions of pages. But user studies indicate that a common user browses through top 10 or 20 documents only. So it's all-important to get into those top 10 documents. To achieve this web authors are increasingly beginning to rely on underhand techniques to ensure their sites get seen, in turn affecting the performance of search engines. The existing measures to evaluate these systems' performance are not adequate in the current world of highly interactive end-user systems. In this study a metric 'Ranked Precision' is proposed to evaluate the performance of search engines.

The World Wide Web has huge amount of information that is retrieved using information retrieval tool like Search Engine. Page repository of Search Engine contains the web documents downloaded by the crawler. This repository contains variety of web documents from different domains. In this paper, a technique called "Retrieval of Web documents using a fuzzy hierarchical clustering" is being proposed that creates the clusters of web documents using fuzzy hierarchical clustering.

Accurate gaze region estimation on the web is important for the purpose of placing marketing advertisements in web pages and monitoring authenticity of user’s response in web forms. To identify gaze region on the web, we need cheap, less technical and non-laboratory setup that minimize user interruptions. This study uses the mouse and keyboard as input devices to collect web usage data. The data collected is categorized into mouse activities, keyboard activities and scroll activities. A regression model is then used to estimate gaze region from the categorized data. The results indicate that although mouse cursor data is generated throughout the user session, the use of data from keyboard presses and scroll from both keyboard and mouse improves the accuracy of identified gaze regions in web usage data.

Existing search engines use web crawlers to gather web pages. The extracted information is used to build indexes, which are later used to answer user queries. This approach is useful for general queries, but ignores the special properties of sound files, making it difficult to accurately locate specific sound files on the web. AROOOGA, or the Articulated Resource for Obsequious Opinionated Observations into Gathered Audio, is a web crawling system designed specifically to find and analyze audio resources on the web. The AROOOGA web crawler uses both audio information and the associated web pages to produce higher-quality search indexes for music information retrieval. Information about sound files on the web is discussed, and some preliminary search results are included.

The essence of a web page is an inherently predisposed issue, one that is built on behaviors, interests, and intelligence. There are relatively a ton of reasons web pages are critical to the new world, as the matter cannot be overemphasized. The meteoric growth of the internet is one of the most potent factors making it hard for search engines to provide actionable results. With classified directories, search engines store web pages. To store these pages, some of the engines rely on the expertise of real people. Most of them are enabled and classified using automated means but the human factor is dominant in their success. From experimental results, we can deduce that the most effective and critical way to automate web pages for search engines is via the integration of machine learning.

Abstract. Search engines on the Web are valuable tools for searching information according to a user's interests whether an individual or a software agent. In the present article we describe the design and the operation mode of SpiderServer, a metasearch engine used for the ...

This paper considers the use of controlled languages for query translation in a legislative document retrieval system. Problem statement and analysis of the approach are described. The use of controlled languages is motivated by the fact that precision is very important in our case. In many information retrieval systems, the use of unrestricted language resources such as general purpose machine translation or bilingual lexica, provides better recall at the expense of precision. Ambiguities and polysemy make the search engine retrieve irrelevant documents as semantic knowledge is missing. Controlled languages help to better specify the word sense according to the domain of interest. Thus ambiguities are avoided and polysemy is specified according to the domain. We will implement our idea in the area of VAT regulation in Europe.

World Wide Web content continuously grows in size and importance. Furthermore, users ask Web search engines to satisfy increasingly disparate information needs. New techniques and tools are constantly developed aimed at assisting users in the interaction with the Web search engine. Query recommender systems suggesting interesting queries to users are an example of such tools. Most query recommendation techniques are based on the knowledge of the behaviors of past users of the search engine recorded in query logs. A recent query-log mining approach for query recommendation is based on Query Flow Graphs (QFG). In this paper we propose an evaluation of the effects of time on this query recommendation model. As users interests change over time, the knowledge extracted from query logs may suffer an aging effect as new interesting topics appear. In order to validate experimentally this hypothesis, we build different query flow graphs from the queries belonging to a large query log of a real-world search engine. Each query flow graph is built on distinct query log segments. Then, we generate recommendations on different sets of queries. Results are assessed both by means of human judgments and by using an automatic evaluator showing that the models inexorably age.

Abstract. Through their interaction with search engines, users provide implicit feedback that can be used to extract useful knowledge and improve the quality of the search process. This feedback is encoded in the form of a query log that consists of a sequence of search actions, which contain information about submitted queries, documents viewed, and documents clicked by the users. In this paper, we propose characterizing documents and queries via the information available within a query log, with the goal of detecting either query polysemy ...