Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Data Preprocessing
Data Cleaning
Conclusion
References

A Survey on Preprocessing of Web-Log Data in Web Usage Mining

Profile image of Anyam Venkata SrinivasAnyam Venkata Srinivas

2017, IJMTST

Abstract

Web mining is to determine and extract useful information. In the internet age web applications are increasing at enormous speed and the web users are increasing at exponential speed. As number of users grows, web site publishers are having increasing their information for attracting and satisfying users. It is possible to trace the users' essence and interactions with web applications through web server log file and Web log file contains only (.txt) file. The data stored in the web log file consist of large amount of eroded, incomplete, and unnecessary information. Because of large amount of irrelevant data's available in the web log file, an original log file cannot be directly used in the web usage mining. So preprocessing technique is applied to improve the quality and efficiency of a web log file. Different techniques are applied in preprocessing that is data cleaning, data fusion, data integration. In this paper we will survey different preprocessing technique to identify the issues in web log file and to improve web usage mining preprocessing for pattern mining and analysis.

Related papers

Preprocessing Techniques in Web Usage Mining: A Survey
Mitali Srivastava

Due to huge, unstructured and scattered amount of data available on web, it is very tough for users to get relevant information in less time. To achieve this, improvement in design of web site, personalization of contents, prefetching and caching activities are done according to user’s behavior analysis. User’s activities can be captured into a special file called log file. There are various types of log: Server log, Proxy server log, Client/Browser log. These log files are used by web usage mining to analyze and discover useful patterns. The process of web usage mining involves three interdependent steps: Data preprocessing, Pattern discovery and Pattern analysis. Among these steps, Data preprocessing plays a vital role because of unstructured, redundant and noisy nature of log data. To improve later phases of web usage mining like Pattern discovery and Pattern analysis several data preprocessing techniques such as Data Cleaning, User Identification, Session Identification, Path Completion etc. have been used. In this paper all these techniques are discussed in detail. Moreover these techniques are also categorized and incorporated with their advantage and disadvantage that will help scientist, researchers and academicians working in this direction.

View PDFchevron_right
A Review Analysis of Preprocessing Techniques in Web usage Mining
Jitendra Upadhyay

International Journal of Engineering Research and, 2015

The Internet web has become popular tool to assist human for their information needs from web server. Due to increasing number of users for web access day by day, there is a need to analyze behavior of such user, in order to monitor and improve performance and throughput of website. Web usage mining is one of the data mining applications which deal with web log files and extract useful information from web. There are different phases are for web usage mining: Data preprocessing, discover pattern and pattern analysis. Among them data preprocessing is the most crucial phase of web usage mining because without good quality of data it is difficult to identify pattern of users behavior. This paper provides reviews of different data preprocessing methods like data collection, data cleaning, User identification, session identification and path completion which will be useful for the community to select one or combination of available techniques in order to carry out efficient preprocessing in order to obtain reliable data mining outcome.

View PDFchevron_right
A Comprehensive Survey on Data Preprocessing Methods in Web Usage Mining
Sujith Jayaprakash

2015

Abstract—Web usage mining is the application of data mining technique which is used to extract information about user’s interest from web server log files. Web usage mining is widely used by companies to analyze the customer’s interest and predict future of their business. It is used in various fields like E-Business, E-Commerce, E- learning, etc., Web usage mining entails of three phases:- Data Preprocessing, Pattern Discovery and Pattern analysis. Data Preprocessing is one of the essential and a preliminary step in web mining to enforce quality in the input data. The raw data from web server log file is preprocessed to eliminate the noisy, vague and redundant data for efficient mining. It involves different phases namely Field Extraction and Data cleaning, User Identification, Session Identification, Path completion and

View PDFchevron_right
PATTERN DETECTION WITH IMPROVED PREPROCESSING IN WEB LOG
Publisher ijmra.us UGC Approved

The past fifteen years are characterized by an exponential growth of the Web both in the number of Web sites available and in the number of their users. This growth generated huge quantities of data related to the user's interaction with the Web sites, recorded in Web log files. Moreover, the Web sites owners expressed the need to better understand their visitors in order to better serve them. The Web Use Mining (WUM) is a rather recent research field and it corresponds to the process of knowledge discovery from databases (KDD) applied to the Web usage data. It comprises three main stages: the preprocessing of raw data, the discovery of schemas and the analysis (or interpretation) of results. A WUM process extracts behavioral patterns from the Web usage data and, if available, from the Web site information (structure and content) and on the Web site users (user profiles). In this thesis, we bring two significant contributions for a Web Use Mining process. We propose a customized application specific methodology for preprocessing the Web logs and a modified frequent pattern tree for the discovery of patterns efficiently.

View PDFchevron_right
A Survey on Preprocessing Methods for Web Usage Data
chi raj

World Wide Web is a huge repository of web pages and links. It provides abundance of information for the Internet users. The growth of web is tremendous as approximately one million pages are added daily. Users' accesses are recorded in web logs. Because of the tremendous usage of web , the web log files are growing at a faster rate and the size is becoming huge. Web data mining is the application of data mining techniques in web data. .Web Usage Mining applies mining techniques in log data to extract the behaviour of users which is used in various applications like personalized services, adaptive web sites, customer profiling, prefetching, creating attractive web sites etc., Web usage mining consists of three phases preprocessing, pattern discovery and pattern analysis. Web log data is usually noisy and ambiguous and preprocessing is an important process before mining.

View PDFchevron_right
Data Extraction from Web Server Logs using Web Usage Mining
IJIRT Journal

The growth of the Internet is growing rapidly, and the use of Web sites and Web-based systems has become very common. The main problem facing website administrators or web application systems is increasing the number of data types and formats stored in server log files in seconds, protecting users, future needs, and structure. Website or web service content depending on previous data. Web Usage Mining aims to discover useful information or information about usage data recorded in log files based on the main data types used in the mining process. The data extracted from the web service registry and the browsing behaviour of the user are used to get the same user generated web access order. In this document, we will use one of the web mining techniques to learn about web server log files that record the entire user's browsing history using the web mining technique.

View PDFchevron_right
Log Data Preparation for Mining Web Usage Patterns
Anna Maria Fanelli

2007

In this paper we focus on log data preprocessing, the first step of a common Web Usage Mining process. In particular, we present LODAP (LOg DAta Preprocessor), a software tool which we designed and implemented in order to perform preprocessing of log data. The working scheme of LODAP embraces several steps. Firstly, log files are cleaned by removing irrelevant data. Then, the remaining requests are structured into user sessions, encoding the browsing behavior of users. Successively, the uninteresting sessions and the least visited pages are removed in order to reduce the size of data concerning the previously extracted user sessions. In addition, LODAP allows to create reports containing the results obtained in each step and information summaries mined from the analysis of the considered log files. During the preprocessing through LODAP, the analyst is guided by a sequence of panels representing the wizard-based interface which characterizes the tool. Each panel is a graphical window which offers a basic function of the preprocessor. Preliminary results on log files of a specific Web site show that the implemented tool can effectively reduce the log data size and identify user sessions encoding the user browsing behavior in a significant manner.

View PDFchevron_right
Analysis of Data Extraction and Data Cleaning in Web Usage Mining
Mitali Srivastava

Data preprocessing is considered as an important phase of Web usage mining due to unstructured, heterogeneous and noisy nature of log data. Complete and e ective data pre- processing insures the eciency and scalability of algorithms used in pattern discovery phase of Web usage mining. Data preprocessing generally includes the steps- Data fusion, Data cleaning, User identi cation, Session identi cation, Path com- pletion etc. Data cleaning is the initial and important step in preprocessing to extract cleaned data for further process- ing. It is important to apply data extraction before data cleaning on raw log data in analysis of speci c time-duration i.e. one day, one week or one month etc. In this paper we have mainly focused on data fusion, data extraction and data cleaning steps of preprocessing and proposed an algo- rithm for data extraction which extracts log data according to analysis of time duration. This algorithm also sorts log entries according to their date and time which will be further used in prediction of browsing sequence of user. After that we have applied data cleaning algorithm on extracted real Web server log. In data cleaning almost all irrelevant les, irrelevant HTTP methods and wrong HTTP status codes are considered and after experiment it is analyzed that raw log data reduces to almost 80% which shows the importance of initial phases of data preprocessing.

View PDFchevron_right
A Study on Prediction of User Behavior Based on Web Server Log Files in Web Usage Mining
anurag kumar

Nowadays, the growth of World Wide Web has exceeded a lot with more expectations. The internet is growing day by day, so online users are also rising. The interesting information for knowledge of extracting from such huge data demands for new logic and the new method. Every user spends their most of the time on the internet and their behavior is different from one and another. Web usage mining is the category of web mining that helps in automatically discovering user access pattern. Web usage mining is leading research area in Web Mining concerned about the web user's behavior. In this paper emphasizes is given on the user Behaviors using web server log file prediction using web server log record, click streams record and user information. Users using web pages, frequently visited hyperlinks, frequently accessed web pages, links are stored in web server log files. A Web log along with the individuality of the user captures their browsing behavior on a website and discussing regarding the behavior from analysis of different algorithms and different methods

View PDFchevron_right
A Survey on Web Usage Mining: Process, Application and Tools
Dr. M S Shekhawat

2017

An enormous amount of data is available in form of web documents over the World Wide Web and is increasing day by day. Web mining is used to extract useful information from web documents. Web mining is categorized into three types, namely, web content mining, web structure mining and web usage mining. Web usage mining is the data mining technique to mine the web log data from World Wide Web to extract useful information. Web usage mining useful for the applications like ecommerce to do personalized marketing, fight against terrorism, fraud detection, to identify criminal activities, web design etc. This paper is going to explain in detail about the process involved in Web Usage Mining, Web Usage Mining applications and tools. Keywords-Web Usage Mining, Web log files, Web usage mining process, Web usage mining applications and Tools. TYPES OF WEB SERVER LOG FILES There are four types of web server log files[4]: a. Access log file: Access log is used to capture the information about the user and it has many numbers of attributes like Date,

View PDFchevron_right

References (31)

  1. Zidrina Pabarskaite, Aistis Raudys (2007), A process of knowledge discovery from web usage data: Systemization and critical review in Journal of Intelligent Information System, Springer Vol.28 Issue.1 Page(s): 79-104.
  2. C. Shahabi, F. Banaei-Kashani (2002), A Framework for Efficient and Anonymous Web Usage Mining Based on Client-Side Tracking in WEBKDD Third International Workshop on Mining Web Log Data, Page(s): 113-144.
  3. D. Pierrakos, G. Paliouras, C. Papatheodorou, and C. D. Spyropoulos (2003), Web usage mining as a tool for personalization: A survey in User Modeling and User Adapted Interaction journal, Vol. 13 Issues. 4 Page(s): 311-372.
  4. B. Naresh Kumar Reddy, M.H.Vasantha, Y.B.Nithin Kumar and Dheeraj Sharma, "Communication Energy Constrained Spare Core on NoC", 6 th International Conference on Computing, Communication and Networking Technologies (ICCCNT), PP. 1-4, 2015.
  5. B. Naresh Kumar Reddy, M.H.Vasantha, Y.B.Nithin Kumar and Dheeraj Sharma, "A Fine Grained Position for Modular Core on NoC IEEE International Conference on Computer, Communication and Control, PP. 1-4, 2015.
  6. Robert.Cooley,Bamshed Mobasher and Jaideep Srinivastava,"Data Preparation for Mining World Wide Web Browsing Patterns ,", journal of knowledge and Information Systems,1999.
  7. Cyrus Shahabi, Amir M.Zarkessh, Jafar Abidi and Vishal Shah "Knowledge discovery from users Web page navigation, ", In.Workshop on Research Issues in Data Engineering, Birmingham, England,1997.
  8. Yan Li, Boqin FENG and Qinjiao MAO, "Research on Path Completion Technique in Web Usage Mining,,", International Symposium on Computer Science and Computational Technology, IEEE,2008.
  9. Yan Li and Boqin FENG "The Construction of Transactions for Web Usage Mining,,", International Conference on Computational Intelligence and Natural Computing, IEEE,2009.
  10. R. Cooley, B. Mobasher, J. Srivastav (1999), Data preparation for mining world wide web browsing pattern in Journal of Knowledge and Data Engineering Workshop, IEEE, Vol.1 Page(s): 5-32.
  11. B. Naresh Kumar Reddy, M.H.Vasantha and Y.B.Nithin Kumar, "A Gracefully Degrading and Energy-Efficient Fault Tolerant NoC Using Spare core", 2016 IEEE Computer Society Annual Symposium on VLSI, pp. 146-151, 2016.
  12. D. Tanasa, B. Trousse (2004), Advanced Data Preprocessing for Intersites Web Usage Mining in IEEE Intelligent Systems, Vol. 19 Issues. 2 Page(s): 59-65.
  13. R. F. Dell (2008),Web user session reconstruction using integer programming in International Conference on Web Intelligence and Intelligent Agent Technology, IEEE/ACM/WIC, Vol. 1 Page(s): 385-388.
  14. Yan LI (2008), Research on path completion technique in web usage mining in International Symposium on Computer Science and Computational Technology, IEEE, Vol. 1 Page(s): 554-559.
  15. Xiang-ying Li (2013), Data Preprocessing in Web Usage Mining in 19th International Conference on Industrial Engineering and Engineering Management Page(s): 257-266.
  16. Sanjay Bapu Thakare, Prof. Sangram Z Gawali, "A Effective and Complete Preprocessing for Web Usage Mining", International Journalon Computer Science and Engineering, Vol. 02, No. 03, pp. 848-851,2010,
  17. Amit Dipchandji Kasliwal, Dr. Girish S. Katkar, " Web Usage mining for predicting User Access Behavior", International Journal of Computer Science and Information Technology, Vol. 6 (1), 2015, 201-204
  18. Navin Kumar Tyagi, A.K.Solanki, Sanjay Tyagi, " An Algorithmicapproach to data preprocessing in Web Usage Mining", InternationalJournal of Information Technology and Knowledge Management, Vol.2, No. 2, pp. 279-283,Dec-2010
  19. K.R. Suneetha , Dr. R. Krishnamoorthi, "Indentifying User Behavior byanalyzing Web Server Access Log File", International Journal ofComputer Science and Network Security, Vol. 9, No. 4, April 2009
  20. Jaideep Srivastava, Robert Cooley, Mukund Despande, Pang-Ning Tan,"Web Usage Mining: Discovery and Applications of Usage Patternsfrom Web Data", SIGKDD Explorations, Vol. 1, Issue 2 Jan 2000
  21. V.Chitraa, Dr. Antony Selvadoss Thanamani, "Web Log Data Cleaning for Enhancing Mining Process", International Journal of Communication and Computer Technologies", Vol. 01, No. 11, Issue 03, December 2012.
  22. B. Naresh Kumar Reddy, et al., An Efficient Data Transmission by using Modern USB Flash Drive", International Journal of Electrical and Computer Engineering, Vol. 4, Issue 5, 2014.
  23. Vellingiri J. and S. Chenthur Pandian, " A Novel Technique for Web Log mining with Better Data Cleaning and Transaction Identification ",Journal of Computer Science 7(5): 683-689, ISSN: 1549-3636, 2011
  24. P. Nithya and Dr. P. Sumathi, "Novel Pre-Processing Technique for Web Log Mining by Removing Global Noise, Cookies and Web Robots", International Journal of Computer Applications, Vol. 53, No.17, September-2012
  25. Vellingiri J., S. Kaliraj, S. Satheeshkumar and T. Parhtiban, " A Novel Approach for User Navigation Pattern Discovery and Analysis for Web Usage Mining", Journal of Computer Science 11(2); 372-382, 2015
  26. Ashwin G. Raiyani, Sheetal S. Pandya, "Discovering user identification mining techniques for preprocessed web log data", Journal ofInformation, Knowledge and Research in Computer Engineering, ISSN: 0975-6760, Vol. 2, Issue. 2, Pages 477-482, OCT-2013
  27. Renata I., Sandor J., "Analysis of Web User Identification Methods",World Academy of Science, Engineering and Technology, 2007
  28. Arvindkumar Dangi, Sunita Sangwan, " A new approach for user identification in web usage mining preprocessing", IOSR Journal of Computer Engineering, e-ISSN: 2278-0661, p-ISSN: 2287-8727, Vol. 11, Issue. 3, (May-June2013), Pages 57-61
  29. Vijaya Sree Boddu and et. al., "Low power and area efficient N-bit parallel processor on a chip", 13th International IEEE India Conference INDICON 2016, pp. 1-4, 2016.
  30. Shaily Langhnoja, Mehul Barot, Darshak Mehta, " Pre-processing: Procedure on Web Log File for Web Usage Mining", InternationalJournal of Emerging Technology and Advanced Engineering, ISSN: 2250-2459, ISO 9001:2008 Certified Journal, Vol. 2, Issue. 12,December 2012
  31. C.E. Dinuca, D. Ciobanu, " Improving the session identification using the mean time", International Journal of Mathematical Models andMethods in Applied Sciences, Vol. 6, Issue 2, 2012

Related papers

A Survey on Preprocessing of Web Log File in Web Usage Mining to Improve the Quality of Data
R Lokeshkumar

— Web mining is to discover and extract useful information. In the internet era web applications are increasing at enormous speed and the web users are increasing at exponential speed. As number of users grows, web site publishers are having increasing their information for attracting and satisfying users. it is possible to trace the users' essence and interactions with web applications through web server log file and Web log file contains only (.txt) file. The data stored in the web log file consist of large amount of eroded, incomplete, and unnecessary information. Because of large amount of irrelevant data's available in the web log file, an original log file can not be directly used in the web usage mining. So prepeocessing technique is applied to improve the quality and efficiency of a web log file. Different techniques are applied in preprocessing that is data cleaning, data fusion, data integration. In this paper we will survey different preprocessing technique to identify the issues in web log file and to improve web usage mining preprocessing for pattern mining and analysis.

View PDFchevron_right
Survey on Pre-Processing Web Log Files in Web Usage Mining
rita roy

2019

Web mining is to decide and extricate helpful data. In the web age web applications are expanding at gigantic speed and the web clients are expanding at exponential speed. As number of clients develops, site distributers are having expanding their data for pulling in and fulfilling clients. It is conceivable to follow the clients' quintessence and collaborations with web applications through web server log record and Web log document contains just (.txt) record. The information put away in the web log record comprise of enormous measure of disintegrated, inadequate, and pointless data. In view of enormous measure of superfluous information's accessible in the web log record, a unique log document can't be straightforwardly utilized in the web use mining. So pre-processing method is connected to improve the quality and effectiveness of a web log record. Various procedures are connected in pre-processing that is information cleaning, information combination, information jo...

View PDFchevron_right
Pre-Processing: Procedure on Web Log File for Web Usage Mining
Mehul Barot

2014

— These days World Wide Web becomes very popular and interactive for transferring of Information. Web usage mining is the area of data mining which deals with the discovery and analysis of usage patterns from Web data, specifically web logs, in order to improve web based applications. Web usage mining consists of three phases, preprocessing, pattern discovery, and pattern analysis. After the completion of these three phases the user can find the required usage patterns and use these information for the specific needs. The web access log file is saved to keep a record of every request made by the users. However, the data stored in the log files does not specify accurate details of the users’ accesses to the Web site. So, preprocessing of the Web log data is first and important phase before web log file can be applied for pattern analysis & pattern discovery. The preprocessed Web Log file can then be suitable for the discovery and analysis of useful information referred to as Web mini...

View PDFchevron_right
Study on Various Web Mining Functionalities using Web Log Files
IJCSMC Journal
View PDFchevron_right
A Survey: Investigation for Apache Log Preprocessing in Web Usage Mining
Editor IJIRMF

Web use mining is a key procedure to concentrate information from web. Web use mining gives use design and client route for the site. Utilizing WUM association advantageously extricates visit get to designs and likewise enhances the website composition. In this Paper, The Principle Procedure is 3 Phases, Information Cleaning, User Recognizable proof and Session Distinguishing proof. We are execution of the information cleaning procedure of web log information utilizing Web utilization mining Procedure. Web log preprocessing we are enhancing for exactness and proficiency.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved