A Review on Feature Selection Methods For Classification Tasks

IJARKE Science & Technology Journal, 2018

Cardiovascular diseases (CVDs) are currently the number one cause of death globally (WHO,2017) and in Kenya Cardiovascular issues such as heart attacks are the number one cause of death in adults over 30.However, the trend of the disease is fast shifting to the youth as more young people are diagnosed with heart conditions. At least 60 per cent of patients who go to hospital with heart attacks are between 20 and 30 (Merab, 2016). Medical diagnosis is an essential yet difficult task that needs to be effected accurately and efficiently. The computerization of medical diagnosis is exceptionally advantageous. A classification system can assist the physicians to examine a patient .The system can predict if the patient is likely to have a certain disease. This paper explores the different feature selection techniques which are used to reduce features, in order to improve accuracy of classification techniques in health care field to aid in the prediction of heart diseases. These techniques provide unseen patterns which can be used for disease diagnosis in the healthcare data. Feature selection techniques are effective approaches to the latest and high dimensional data. The relevant features" obtained which lack noise will improve disease diagnosis in the healthcare industry. Feature selection techniques like Correlation Feature Selection, Information Gain, Relief, and Genetic Search are used to choose the most relevant features from the high dimensional feature set. The rest of the paper is organized as follows: in section 2 we provide an introduction to feature selection frameworks and different feature selection techniques as well as a comprehensive comparison between them; section 3 briefly introduces several feature selection research works for heart disease prediction. Finally we draw conclusions and recommendation in section 4. 2. Feature Selection A realization to use data mining tools more effectively is through data processing, most researchers and practitioners had realized this fact too (Sindhiya and Gunasundari, 2014). Feature selection is one of the essential and commonly used methods in data preprocessing. Computation of the result by using the entire feature set may not always give the greatest result because of redundant and irrelevant features, also referred as noisy features. To remove these redundant features, it is essential to use a

International Journal of Computer Science and Information Security (IJCSIS), Vol. 23, No. 1, January-February, 2025

Abstract- the increasing volume of medical data presents both opportunities and challenges for enhancing classification methods. While Support Vector Machines (SVMs) are widely recognized for their effectiveness in various classification tasks, traditional SVM approaches exhibit significant limitations when applied to large-scale datasets. This paper presents a hybrid feature selection framework designed to enhance the classification performance of medical data by identifying the most relevant features for Support Vector Machine (SVM) classifiers. The proposed framework consists of two phases. The first phase combines Information Gain with several optimization techniques, including Particle Swarm Optimization (PSO), Bat Search Method, Elephant Search Algorithm (ESA), and Firefly Algorithm, to effectively select key features for medical datasets. Our results show that the hybrid framework significantly improves classification accuracy compared to traditional feature selection methods, with the ESA Algorithm excelling in their respective categories. In the second phase, we integrate enhancements into the traditional SVM model to address uncertainties in medical data, resulting in a more adaptable and accurate classifier. The enhanced SVM refines both training and testing sets, ensuring that the model is trained on the most relevant and accurate data, thus improving its performance in uncertain or noisy conditions. Overall, the proposed hybrid framework provides a more robust and efficient solution for medical data classification, improving both the accuracy and adaptability of SVM classifiers in realworld applications. Keywords— Machine learning, Medical data classification, KNN, feature selection, optimized method, SVM, Data uncertainty, Hybrid model, Classification accuracy

Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies

In the last decade, feature selection (FS), was one of the most investigated preprocessing tasks for heart disease prediction. Determining the optimal features which contribute more towards the diagnosis of heart disease can reduce the number of clinical tests needed to be taken by a patient, decrease the model cost, reduce the storage requirements and improve the comprehensibility of the induced model. In this study a comparison of three filter feature ranking methods was carried out. Feature ranking methods need to set a threshold (i.e. the percentage of the number of relevant features to be selected) in order to select the final subset of features. Thus, the aim of this study is to investigate if there is a threshold value which is an optimal choice for three different feature ranking methods and four classifiers used for heart disease classification in four heart disease datasets. The used feature ranking methods and selection thresholds resulted in optimal classification performance for one or more classifiers over small and large heart disease datasets. The size of the dataset takes an important role in the choice of the selection threshold.

featureselection.asu.edu

Intelligent Data Analysis, 2010

In the paper, we present an empirical evaluation of five feature selection methods: ReliefF, random forest feature selector, sequential forward selection, sequential backward selection, and Gini index. Among the evaluated methods, the random forest feature selector has not yet been widely compared to the other methods. In our evaluation, we test how the implemented feature selection can affect (i.e. improve) the accuracy of six different classifiers by performing feature selection. The results show that ReliefF and random forest enabled the classifiers to achieve the highest increase in classification accuracy on the average while reducing the number of unnecessary attributes. The achieved conclusions can advise the machine learning users which classifier and feature selection method to use to optimize the classification accuracy, which may be important especially in risk-sensitive applications of Machine Learning (e.g. medicine, business decisions, control applications) as well as in the aim to reduce costs of collecting, processing and storage of unnecessary data.

Opportunities and Challenges, 2003

2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2015

Feature selection (FS) methods can be used in data pre-processing to achieve efficient data reduction. This is useful for finding accurate data models. Since exhaustive search for optimal feature subset is infeasible in most cases, many search strategies have been proposed in literature. The usual applications of FS are in classification, clustering, and regression tasks. This review considers most of the commonly used FS techniques. Particular emphasis is on the application aspects. In addition to standard filter, wrapper, and embedded methods, we also provide insight into FS for recent hybrid approaches and other advanced topics. I.

International Journal of Computer Applications, 2014

The aim of this paper is to discuss about various feature selection algorithms applied on different datasets to select the relevant features to classify data into binary and multi class in order to improve the accuracy of the classifier. Recent researches in medical diagnose uses the different kind of classification algorithms to diagnose the disease. For predicting the disease, the classification algorithm produces the result as binary class. When there is a multiclass dataset, the classification algorithm reduces the dataset into a binary class for simplification purpose by using any one of the data reduction methods and the algorithm is applied for prediction. When data reduction on original dataset is carried out, the quality of the data may degrade and the accuracy of an algorithm will get affected. To maintain the effectiveness of the data, the multiclass data must be treated with its original form without maximum reduction, and the algorithm can be applied on the dataset for producing maximum accuracy. Dataset with maximum number of attributes like thousands must incorporate the best feature selection algorithm for selecting the relevant features to reduce the space and time complexity. The performance of Classification algorithm is estimated by how accurately it predicts the individual class on particular dataset. The accuracy constrain mainly depends on the selection of appropriate features from the original dataset. The feature selection algorithms play an important role in classification for better performance. The feature selection is one of the preprocessing techniques in the classification. This research paper deals with different feature selection algorithms and their performance on different dataset.

Journal of Software Engineering and Applications, 2010

Many medical diagnosis applications are characterized by datasets that contain under-represented classes due to the fact that the disease is much rarer than the normal case. In such a situation classifiers such as decision trees and Naïve Bayesian that generalize over the data are not the proper choice as classification methods. Case-based classifiers that can work on the samples seen so far are more appropriate for such a task. We propose to calculate the contingency table and class specific evaluation measures despite the overall accuracy for evaluation purposes of classifiers for these specific data characteristics. We evaluate the different options of our case-based classifier and compare the performance to decision trees and Naïve Bayesian. Finally, we give an outlook for further work.

2020

Developing a good prediction model from excessive-dimensional data in machine intelligence is a difficult endeavor. Attribute subset collection is a crucial step in data preprocessing to find out the best attribute group for improving the efficiency of the directed learning methods. Attribute reduction approaches has been used in many research fields, which deals with the intelligent machine and its capability to learning and giving quick, economical and greater correct for classifying methods. Selecting the small attribute set is a crucial step for making an intelligent system. The current study has shown that meaningful knowledge can be discovered by applying Machine Learning techniques on medical dataset for disease diagnosis. But due to high dimensional of medical data, there are big computational challenges in this field. If the staging data have inappropriate characteristics, it may degrade classifier accuracy and less comprehensible outcomes. And here, the role of attribute c...