Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Previous Works
Methodology
Data Sets
The Examples of Comments from Various Datasets
Methods of Tests and Evaluations
Results and Discussion
Conclusions
References
All Topics
Computer Science
Artificial Intelligence

Antyscam – Practical Web Spam Classifier

Profile image of Paweł SobkowiakPaweł Sobkowiak

2023, International Journal of Electronics and Telecommunications

https://doi.org/10.24425/IJET.2019.130255
visibility

…

description

10 pages

link

1 file

Abstract

To avoid of manipulating search engines results by web spam, anti spam system use machine learning techniques to detect spam. However, if the learning set for the system is out of date the quality of classification falls rapidly. We present the web spam recognition system that periodically refreshes the learning set to create an adequate classifier. A new classifier is trained exclusively on data collected during the last period. We have proved that such strategy is better than an incrementation of the learning set. The system solves the starting-up issues of lacks in learning set by minimisation of learning examples and utilization of external data sets. The system was tested on real data from the spam traps and common known web services: Quora, Reddit, and Stack Overflow. The test performed among ten months shows stability of the system and improvement of the results up to 60 percent at the end of the examined period.

Related papers

Web Spam Detection
Fabrizio Silvestri

Encyclopedia of Database Systems, 2009

View PDFchevron_right
Machine learning algorithm-based spam detection in social networks
Joyece Jane

Many social media (SM) platforms have emerged as a result of the online social network's (OSN) rapid expansion. SM has become important in day-today life, and spammers have turned their attention to SM. Spam detection (SD) is done in two different ways, such as machine learning (ML) and expert-based detection. The expert-based detection technique's accuracy depends on expert knowledge, and it takes huge time to detect the spams. Thus, ML-based spam detection is preferred in OSN. Spam identification on social networks is a difficult operation involving a variety of factors, and spam and ham have resulted in an imbalanced data distribution, which gives flexibility to spammers for corrupting our devices. SD based on ML algorithms like logistic regression (LR), K-nearest neighbor (KNN), decision trees (DT), random forest (RF), support vector machine (SVM) and eXtreme gradient boosting (XGB), voting classifier (VC) and extra tree classifier (ETC) are used to design the address balance and to attain high assessment accuracy in an imbalanced datasets. ETC method minimizes the bias through the original sampling process. For reducing processing complexity, the ETC method uses a smaller size constant factor instead of a larger one. Thus, the ETC technique produces better data splitting than DT and RF techniques. Text is vectorized by vectorizers, and all the relative results are stored in it. The VC is an ensemble method that integrates predictions form several methods to forecast an output class depending on which predictions have the highest probability. The multi-class results are aggregated and forecast for the majority voted class. The experimental result shows that, as compared to KN, NB, ETC, RF, SVC, LR, XGB and DT, the proposed VC provides a higher classification accuracy rate of 97.96%, 97.56% of precision, 89.95% of recall and 91.96% of F1-measures. Similarly, ETC provides 97.77% accuracy, 98.31% of precision, 84.78% of recall and 91.05% of F1-measures. Compared to conventional ML algorithms, VC and ETC provide higher accuracy, precision, recall and F1-measures. Thus, ETC and VC are preferable for spam detection. The website has been designed to detect messages as spam or not.

View PDFchevron_right
Using Machine Learning Algorithms for Analysis of Spam and Its Detection
kumar parmar

2016

Web spam is one of the major problems of search engines because it reduces the quality of the Web page. Web spam also effects economically because spammers provide a large free advertising data or sites on the search engines and so an increase in the web traffic. There are certain ways to distinguish such spam pages and one of them is using classification techniques. Comparative analysis of web spam detection using machine learning algorithm like LAD Tree, and Random Forest, C4.5 and Naive bayes have been presented in this paper. Experiments were carried out on feature sets of universally accepted dataset WEB SPAM UK-2007 using WEKA. By observing all the results we found that Random forest works well on content based features, link based features and transformed link based features. But few techniques were found time consuming as compared to other classification techniques used. Keywords—Machine learning, Spamdexing, cloaking, link spam, content spam, C4.5, Naive bayes, LAD tree, de...

View PDFchevron_right
A review of spam email detection: analysis of spammer strategies and the dataset shift problem
FRANCISCO JÁÑEZ MARTINO

Artificial Intelligence Review

Spam emails have been traditionally seen as just annoying and unsolicited emails containing advertisements, but they increasingly include scams, malware or phishing. In order to ensure the security and integrity for the users, organisations and researchers aim to develop robust filters for spam email detection. Recently, most spam filters based on machine learning algorithms published in academic journals report very high performance, but users are still reporting a rising number of frauds and attacks via spam emails. Two main challenges can be found in this field: (a) it is a very dynamic environment prone to the dataset shift problem and (b) it suffers from the presence of an adversarial figure, i.e. the spammer. Unlike classical spam email reviews, this one is particularly focused on the problems that this constantly changing environment poses. Moreover, we analyse the different spammer strategies used for contaminating the emails, and we review the state-of-the-art techniques to...

View PDFchevron_right
Performance Evaluation of User-Behaviour Techniques of Web Spam Detection Models
bodunde akinyemi

Network and Complex Systems, 2019

Web spam detection is a critical issue in today's rapidly growing usage of the Internet and the World Wide Web. The upsurge of web spam has significantly deteriorated the Quality of Services (QoS) of the World Wide Web. The degeneration of the quality of search engine results has given rise to researches on the detection of spam pages efficiently and accurately. Existing user-behaviour oriented web spam detection models employed the content-based, link-based and other features of webpages for classification of web spams. These user-behaviour techniques either implemented singly or combined has achieved good detection performance. However, the effectiveness of these features in identifying Web spams correctly needs to be determined. In this study, predictive web spam detection models that employed all related user-behaviour features of webpages were developed and evaluated. The content, link, and obvious-based features datasets were collected from an online repository. Relevant features were extracted using an improved Filter-based method. Six user-behaviour related features extracted from the datasets were used to combine the datasets to generate all possible subset of feature space required, such that 7 new datasets were generated for the study. Multi-Layer Perceptron (MLP) approach was adopted as a classifier for each of the identified features. Python Machine Learning Library was used to simulate the models using percentage splits of 60/40%, 70/30% and 80/20% ratio for training/testing dataset and the performances were evaluated using accuracy, True Positive (TP) rate, False Positive (FP) rate and precision as metrics. The result showed that for the majority of the datasets the formulated models have shown an increase in efficiency after feature selection. The MLP classifier was able to achieve the best result of 66.0% accuracy when the link-based dataset was used with feature selection. The study concluded that link-based features of a user is sufficient and effective for the detection of web spams.

View PDFchevron_right
Towards Web Spam Filtering with Neural-Based Approaches
Tiago Almeida

2012

The steady growth and popularization of the Web increases the competition between the websites and creates opportunities for profit in several segments. Thus, there is a great interest in keeping the website in a good position in search results. The problem is that many websites use techniques to circumvent the search engines which deteriorates the search results and exposes users to dangerous content. Given this scenario, this paper presents a performance evaluation of different models of artificial neural networks to automatically classify web spam. We have conducted an empirical experiment using a well-known, large and public web spam database. The results indicate that the evaluated approaches outperform the state-of-the-art web spam filters.

View PDFchevron_right
Improving web spam classification using rank-time features
Aaswath Raman

2007

Abstract In this paper, we study the classification of web spam. Web spam refers to pages that use techniques to mislead search engines into assigning them higher rank, thus increasing their site traffic. Our contributions are two fold. First, we find that the method of datset construction is crucial for accurate spam classification and we note that this problem occurs generally in learning problems and can be hard to detect.

View PDFchevron_right
Web Spam Classification Using Supervised Artificial Neural Network Algorithms
Rizwan Beg

Due to the rapid growth in technology employed by the spammers, there is a need of classifiers that are more efficient, generic and highly adaptive. Neural Network based technologies have high ability of adaption as well as generalization. As per our knowledge, very little work has been done in this field using neural network. We present this paper to fill this gap. This paper evaluates performance of three supervised learning algorithms of artificial neural network by creating classifiers for the complex problem of latest web spam pattern classification. These algorithms are Conjugate Gradient algorithm, Resilient Backpropagation learning, and Levenberg-Marquardt algorithm.

View PDFchevron_right
Spam Mail Classifier
IJRASET Publication

International Journal for Research in Applied Science & Engineering Technology (IJRASET), 2022

Email is the worldwide use of communication application. It is because of the ease of use and faster than other communication application. However, its inability to detect whether the mail content is either spam or ham degrade its performance. Nowadays, lot of cases have been reported regarding stealing of personal information or phishing activities via email from the user. This project will discuss how machine learning help in spam detection. Machine learning is an artificial intelligence application that provides the ability to automatically learn and improve data without being explicitly programmed. Binary classifier will be used to classify the text into two different categories: spam and ham. The algorithm will predict the score more accurately. The objective of developing this model is to detect and score word faster and accurately.

View PDFchevron_right
Investigating Spam Mass Variations for Detecting Web Spam
Nasir Touheed

2011

In this paper, we investigate variations of Spam Mass for filtering web spam. Firstly, we propose two strategies for designing new variations of the Spam Mass algorithm. Then, we perform experiments among different versions of Spam Mass using WEBSPAM-UK2006 data set. Finally, we show improvement through proposed strategy by up to 1.33 times in recall and 1.02 times in precision over the original version of Spam Mass.

View PDFchevron_right

References (43)

  1. J. Carpinter and R. Hunt, "Tightening the net: A review of current and next generation spam filtering tools," Computers & Security, vol. 25, no. 8, pp. 566 -578, 2006. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167404806000939
  2. Q. Xu, E. Xiang, Q. Yang, J. Du, and J. Zhong, "Sms spam detection using noncontent features," Intelligent Systems, IEEE, vol. 27, no. 6, pp. 44-51, 2012.
  3. J. W. Yoon, H. Kim, and J. H. Huh, "Hybrid spam filtering for mobile communication," Computers & Security, vol. 29, no. 4, pp. 446 -459, 2010. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167404809001266
  4. Y. Gao and A. Choudhary, "Active learning image spam hunter," in Advances in Visual Computing, ser. Lecture Notes in Computer Science, G. Bebis, R. Boyle, B. Parvin, D. Koracin, Y. Kuno, J. Wang, R. Pajarola, P. Lindstrom, A. Hinkenjann, M. Encarnao, C. Silva, and D. Coming, Eds. Springer Berlin Heidelberg, 2009, vol. 5876, pp. 293-302.
  5. S. Wakade, K. Liszka, and C.-C. Chan, "Application of learning algo- rithms to image spam evolution," in Emerging Paradigms in Machine Learning, ser. Smart Innovation, Systems and Technologies, S. Ra- manna, L. C. Jain, and R. J. Howlett, Eds. Springer Berlin Heidelberg, 2013, vol. 13, pp. 471-495.
  6. F. Benevenuto, T. Rodrigues, A. Veloso, J. Almeida, M. Goncalves, and V. Almeida, "Practical detection of spammers and content promoters in online video sharing systems," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 3, pp. 688-701, June 2012.
  7. A. Luz, E. Valle, and A. A. Arajo, "Non-collaborative content detecting on video sharing social networks," Multimedia Tools and Applications, vol. 1, pp. 1-19, 2012.
  8. V. Potdar, F. Ridzuan, P. Hayati, A. Talevski, E. A. Yeganeh, N. Firuzeh, and S. Sarencheh, "Spam 2.0: The problem ahead," in ICCSA (2)'10, 2010, pp. 400-411.
  9. C. Castillo, D. Donato, L. Becchetti, P. Boldi, S. Leonardi, M. Santini, and S. Vigna, "A reference collection for web spam," SIGIR Forum, vol. 40, no. 2, pp. 11-24, Dec. 2006.
  10. M. Erdélyi, A. A. Benczúr, J. Masanés, and D. Siklósi, "Web spam filtering in internet archives," in Proceedings of the 5th International Workshop on Adversarial Information Retrieval on the Web, ser. AIRWeb '09. New York, NY, USA: ACM, 2009, pp. 17-20. [Online]. Available: http://doi.acm.org/10.1145/1531914.1531918
  11. J. Martinez-Romo and L. Araujo, "Web spam identification through language model analysis," in Proceedings of the 5th International Workshop on Adversarial Information Retrieval on the Web, ser. AIRWeb '09. New York, NY, USA: ACM, 2009, pp. 21-28.
  12. L. Araujo and J. Martinez-Romo, "Web spam detection: New classifi- cation features based on qualified link analysis and language models," Information Forensics and Security, IEEE Transactions on, vol. 5, no. 3, pp. 581-590, 2010.
  13. K. L. Goh, A. Singh, and K. H. Lim, "Multilayer perceptrons neural network based web spam detection application," in Signal and Infor- mation Processing (ChinaSIP), 2013 IEEE China Summit International Conference on, July 2013, pp. 636-640.
  14. M. Luckner, M. Gad, and P. Sobkowiak, "Stable web spam detection using features based on lexical items," Computers & Security, vol. 46, pp. 79-93, 2014. [Online]. Available: http://dx.doi.org/10.1016/j.cose.2014.07.006
  15. R. Colbaugh and K. Glass, "Predictive defense against evolving ad- versaries," in Intelligence and Security Informatics (ISI), 2012 IEEE International Conference on, June 2012, pp. 18-23.
  16. M. Brückner and T. Scheffer, "Nash equilibria of static prediction games," in NIPS, Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, and A. Culotta, Eds. Curran Associates, Inc., 2009, pp. 171-179.
  17. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The weka data mining software: An update," SIGKDD Explor. Newsl., vol. 11, no. 1, pp. 10-18, Nov. 2009. [Online]. Available: http://doi.acm.org/10.1145/1656274.1656278
  18. J. Mathew, A. K. Singh, K. L. Goh, and A. K. Singh, "Proceedings of the 4th international conference on eco-friendly computing and communication systems comprehensive literature review on machine learning structures for web spam classification," Procedia Computer Science, vol. 70, pp. 434 -441, 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1877050915032330
  19. M. Erdélyi, A. Garzó, and A. A. Benczúr, "Web spam classification: A few features worth more," in Proceedings of the 2011 Joint WICOW/AIRWeb Workshop on Web Quality, ser. WebQuality '11. New York, NY, USA: ACM, 2011, pp. 27-34. [Online]. Available: http://doi.acm.org/10.1145/1964114.1964121
  20. L. Shengen, N. Xiaofei, L. Peiqi, and W. Lin, "Generating new features using genetic programming to detect link spam," in Proceedings of the 2011 Fourth International Conference on Intelligent Computation Technology and Automation -Volume 01, ser. ICICTA '11. Washington, DC, USA: IEEE Computer Society, 2011, pp. 135-138.
  21. M. Mahmoudi, A. Yari, and S. Khadivi, "Web spam detection based on discriminative content and link features," in Telecommunications (IST), 2010 5th International Symposium on, 2010, pp. 542-546.
  22. S. Algur and N. Pendari, "Hybrid spamicity score approach to web spam detection," in Pattern Recognition, Informatics and Medical Engineering (PRIME), 2012 International Conference on, 2012, pp. 36-40.
  23. C. Dong and B. Zhou, "Effectively detecting content spam on the web using topical diversity measures," in Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology -Volume 01, ser. WI-IAT '12.
  24. Washington, DC, USA: IEEE Computer Society, 2012, pp. 266-273. [Online]. Available: http://dl.acm.org/citation.cfm?id=2457524.2457693
  25. I. Bíró, D. Siklósi, J. Szabó, and A. A. Benczúr, "Linked latent dirichlet allocation in web spam filtering," in Proceedings of the 5th International Workshop on Adversarial Information Retrieval on the Web, ser. AIRWeb '09. New York, NY, USA: ACM, 2009, pp. 37-40. [Online]. Available: http://doi.acm.org/10.1145/1531914.1531922
  26. G. V. Cormack, M. D. Smucker, and C. L. Clarke, "Efficient and effective spam filtering and re-ranking for large web datasets," Inf. Retr., vol. 14, no. 5, pp. 441-465, Oct. 2011.
  27. A. Heydari, M. ali Tavakoli, N. Salim, and Z. Heydari, "Detection of review spam: A survey," Expert Systems with Applications, vol. 42, no. 7, pp. 3634 -3642, 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0957417414008082
  28. X.-C. Yin, K. Huang, C. Yang, and H.-W. Hao, "Convex ensemble learning with sparsity and diversity," Information Fusion, vol. 20, pp. 49 -59, 2014. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1566253513001413
  29. B. Manaskasemsak and A. Rungsawang, "Web spam detection using trust and distrust-based ant colony optimization learning," International Journal of Web Information Systems, vol. 11, no. 2, pp. 142-161, 2015. [Online]. Available: http://dx.doi.org/10.1108/IJWIS-12-2014-0047
  30. S. M. Lee, D. S. Kim, J. H. Kim, and J. S. Park, "Spam detection using feature selection and parameters optimization," in Complex, Intelligent and Software Intensive Systems (CISIS), 2010 International Conference on, 2010, pp. 883-888.
  31. A. Alarifi and M. Alsaleh, "Web spam: A study of the page language effect on the spam detection features," in Machine Learning and Applications (ICMLA), 2012 11th International Conference on, vol. 2, 2012, pp. 216-221.
  32. N. Dai, B. D. Davison, and X. Qi, "Looking into the past to better classify web spam," in Proceedings of the 5th International Workshop on Adversarial Information Retrieval on the Web, ser. AIRWeb '09. New York, NY, USA: ACM, 2009, pp. 1-8.
  33. T. Urvoy, E. Chauveau, P. Filoche, and T. Lavergne, "Tracking web spam with html style similarities," ACM Trans. Web, vol. 2, no. 1, pp. 3:1-3:28, Mar. 2008.
  34. J. Piskorski, M. Sydow, and D. Weiss, "Exploring linguistic features for web spam detection: a preliminary study," in Proceedings of the 4th international workshop on Adversarial information retrieval on the web, ser. AIRWeb '08. New York, NY, USA: ACM, 2008, pp. 25-28.
  35. J. Fdez-Glez, D. Ruano-Ordas, J. R. Méndez, F. Fdez-Riverola, R. Laza, and R. Pavón, "A dynamic model for integrating simple web spam classification techniques," Expert Syst. Appl., vol. 42, no. 21, pp. 7969-7978, Nov. 2015. [Online]. Available: http://dx.doi.org/10.1016/j.eswa.2015.06.043
  36. --, "Wsf2: A novel framework for filtering web spam," Scientific Programming, p. 18, 2016.
  37. C. Seiffert, T. Khoshgoftaar, J. Van Hulse, and A. Napolitano, "Rus- boost: A hybrid approach to alleviating class imbalance," Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, vol. 40, no. 1, pp. 185-197, Jan 2010.
  38. B. Schölkopf, J. C. Platt, J. C. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Comput., vol. 13, no. 7, pp. 1443-1471, Jul. 2001. [Online]. Available: http://dx.doi.org/10.1162/089976601750264965
  39. V. Vapnik, The Nature of Statistical Learning Theory. Springer-Verlag, 1995.
  40. W. Homenda, M. Luckner, and W. Pedrycz, "Classification with rejection based on various SVM techniques," in 2014 International Joint Conference on Neural Networks, IJCNN 2014, Beijing, China, July 6-11, 2014. IEEE, 2014, pp. 3480-3487. [Online]. Available: http://dx.doi.org/10.1109/IJCNN.2014.6889655
  41. L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001.
  42. P. Faltstrom, P. E. Hoffman, and A. M. Costello, "Internationalizing domain names in applications (idna)," Internet RFC 3490, March 2003.
  43. N. Japkowicz and M. Shah, Evaluating Learning Algorithms: A Classifi- cation Perspective. New York, NY, USA: Cambridge University Press, 2011.

Related papers

Web spam classification
András Garzó, Miklós Erdélyi

Proceedings of the 2011 Joint WICOW/AIRWeb Workshop on Web Quality - WebQuality '11, 2011

In this paper we investigate how much various classes of Web spam features, some requiring very high computational effort, add to the classification accuracy. We realize that advances in machine learning, an area that has received less attention in the adversarial IR community, yields more improvement than new features and result in low cost yet accurate spam filters. Our original contributions are as follows:

View PDFchevron_right
Looking into the past to better classify web spam
Brian D Davison

2009

Web spamming techniques aim to achieve undeserved rankings in search results. Research has been widely conducted on identifying such spam and neutralizing its influence. However, existing spam detection work only considers current information. We argue that historical web page information may also be important in spam classification. In this paper, we use content features from historical versions of web pages to improve spam classification. We use supervised learning techniques to combine classifiers based on current page content with classifiers based on temporal features. Experiments on the WEBSPAM-UK2007 dataset show that our approach improves spam classification F-measure performance by 30% compared to a baseline classifier which only considers current page content.

View PDFchevron_right
Stable web spam detection using features based on lexical items
Paweł Sobkowiak

Computers & Security, 2014

Web spam is a method of manipulating search engines results by improving ranks of spam pages. It takes various forms and lacks a consistent definition. Web spam detectors use machine learning techniques to detect spam. However, the detectors are mostly verified on data sets coming from the same year as the learning sets. In this paper we compared Support Vector Machine classifiers trained and tested on WEBSPAMeUK data sets from different years. To obtain stable results we proposed new lexical-based features. The HTML document e transformed into a text without HTML tags, a set of visible symbols, and a list of links including the ones from tags e gave information about weird combinations of letters; consonant clusters; statistics on syllables, words, and sentences; and the Gunning Fog Index. Using data collected in 2006 as a learning set, we obtained very stable accuracy among years. This choice of the training set reduced the sensitivity in 2007, but that can be improved by managing the acceptance threshold. Finally, we proved that the balance between the sensitivity and the specificity measured by the Area Under the Curve (AUC) is improved by our selection of features.

View PDFchevron_right
Web Spam Detection Using Machine Learning in Specific Domain Features
Ismail Mail

2008

In the last few years, as Internet usage becomes the main artery of the life's daily activities, the problem of spam becomes very serious for internet community. Spam pages form a real threat for all types of users. This threat proved to evolve continuously without any clue to abate. Different forms of spam witnessed a dramatic increase in both size and negative impact. A large amount of E-mails and web pages are considered spam either in Simple Mail Transfer Protocol (SMTP) or search engines. Many technical methods were proposed to approach the problem of spam. In E-mails spam detection, Bayesian Filters are widely and successfully applied for the sake of detecting and eliminating spam.

View PDFchevron_right
Web Spam Detection Using Different Features
Sumit Sahu
View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Machine Learning
  • Electronics and Telecommunications
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved