Detecting spam web pages through content analysis

ACM Transactions on the Web, 2008

This article addresses the problem of spam blog (splog) detection using temporal and structural regularity of content, post time and links. Splogs are undesirable blogs meant to attract search engine traffic, used solely for promoting affiliate sites. Blogs represent popular online media, and splogs not only degrade the quality of search engine results, but also waste network resources. The splog detection problem is made difficult due to the lack of stable content descriptors.

Proceedings of the 24th International Conference on World Wide Web - WWW '15, 2015

Increased popularity of smartphones has attracted a large number of developers to develop apps for various smartphone platforms. As a result, app markets are also populated with spam apps, which reduce the users' quality of experience and increase the workload of app market operators. The latter resort to remove those apps, upon user complaints or to deny the developers' publication approval requests by relying on continuous human efforts to check the app compliance with anti-spam policies. Apps can be "spammy" in multiple ways including not having a specific functionality, unrelated app description or unrelated keywords and publishing similar apps several times and across diverse categories. Through a systematic crawl of a popular app market and by identifying a set of removed apps, we propose a method to detect spam apps solely using apps' metadata available at the time of publication. We first propose a methodology to manually label a sample of removed apps, according to a set of checkpoint heuristics to reveal the reasons behind apps removal. This analysis suggests that approximately 35% of the apps being removed are very likely to be spam apps. We then map the identified heuristics to several quantifiable features and show how distinguishing these features are for spam apps. Finally, we build an Adaptive Boost classifier for early identification of spam apps using only the metadata of the apps. Our classifier achieves an accuracy over 95% with precision varying between 85%-95% and recall varying between 38%-98%. By applying the classifier on a set of apps present at the app market at the time of our crawl, we estimate that at least 2.7% of them are spam apps.

Innovations in Computer Science and Engineering, 2019

In this smart world, mobile app fraud is growing rapidly. The fraud might be ranking, review, or rating. In play store, the purpose of fraud is promoting or bumping up the apps to top list. So the user might misunderstand while downloading. Hence, a new mechanism is required in order to detect or prevent mobile fraud. Limited research work has been done on fraud detection. Mostly data mining techniques are applied for this work. But nowadays, large volume of data is getting generated which might be of different formats like structured or unstructured. In this paper, taking review from different researches and identifying the problems, machine learning algorithms are recommended as future scope for resolving these issues. Not only discovering the fraud within particular time but also the most frequent frauds which are committed on the mobile app should get detected.

Knowledge and Information Systems, 2018

Isotonic separation is a classification technique which constructs a model by transforming the training set into a linear programming problem (LPP). It is computationally expensive to solve large-scale LPPs using traditional methods when data set grows. This paper proposes a hybrid binary classification algorithm, meta-heuristic isotonic separation with particle swarm optimization and convergence criterion (MeHeIS-CPSO), in which a particle swarm optimization-based meta-heuristic is embedded in the training phase to find a solution for LPP. The proposed framework formulates the LPP as a directed acyclic graph (DAG) and arranges decision variables using topological sort. It obtains a new threshold value from training set and sets up a convergence criterion using this threshold. It also deploys a new correlation coefficient-based supervised feature selection technique to select isotonic features and improves predictive accuracy of the classifier. Experiments are conducted on publicly available data sets and synthetic data set. Theoretical, empirical, and statistical analyses show that MeHeIS-CPSO is superior to its predecessors in terms of training time and predictive ability on large data sets. It also outperforms state-of-the-art machine learning and isotonic classification techniques in terms of predictive performance on small-and large-scale data sets.

Journal of the American Society for Information Science and Technology, 2012

Search engines are essential tools for web users today. They rely on a large number of features to compute the rank of search results for each given query. The estimated reputation of pages is among the effective features available for search engine designers, probably being adopted by most current commercial search engines. Page reputation is estimated by analyzing the linkage relationships between pages. This information is used by link analysis algorithms as a query-independent feature, to be taken into account when computing the rank of the results. Unfortunately, several types of links found on the web may damage the estimated page reputation and thus cause a negative effect on the quality of search results. This work studies alternatives to reduce the negative impact of such noisy links. More specifically, the authors propose and evaluate new methods that deal with noisy links, considering scenarios where the reputation of pages is computed using the PageRank algorithm. They show, through experiments with real web content, that their methods achieve significant improvements when compared to previous solutions proposed in the literature.

Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, 2016

Document coherence describes how much sense text makes in terms of its logical organisation and discourse flow. Even though coherence is a relatively difficult notion to quantify precisely, it can be approximated automatically. This type of coherence modelling is not only interesting in itself, but also useful for a number of other text processing tasks, including Information Retrieval (IR), where adjusting the ranking of documents according to both their relevance and their coherence has been shown to increase retrieval effectiveness [34, 37]. The state of the art in unsupervised coherence modelling represents documents as bipartite graphs of sentences and discourse entities, and then projects these bipartite graphs into one-mode undirected graphs. However, one-mode projections may incur significant loss of the information present in the original bipartite structure. To address this we present three novel graph metrics that compute document coherence on the original bipartite graph of sentences and entities. Evaluation on standard settings shows that: (i) one of our coherence metrics beats the state of the art in terms of coherence accuracy; and (ii) all three of our coherence metrics improve retrieval effectiveness because, as closer analysis reveals, they capture aspects of document quality that go undetected by both keyword-based standard ranking and by spam filtering. This work contributes document coherence metrics that are theoretically principled, parameter-free, and useful to IR.

ACM Transactions on Information Systems, 2008

Link-based ranking algorithms are among the most important techniques to improve web search. In particular, the PageRank algorithm has been successfully used in the Google search engine and has been attracting much attention recently. However, we find that PageRank has a “zero-one gap” problem which, to the best of our knowledge, has not been addressed in any previous work. This problem can be potentially exploited to spam PageRank results and make the state-of-the-art link-based antispamming techniques ineffective. The zero-one gap problem arises as a result of the current ad hoc way of computing transition probabilities in the random surfing model. We therefore propose a novel DirichletRank algorithm which calculates these probabilities using Bayesian estimation with a Dirichlet prior. DirichletRank is a variant of PageRank, but does not have the problem of zero-one gap and can be analytically shown substantially more resistant to some link spams than PageRank. Experiment results ...