Rule Induction

Present-day healthcare witnesses a growing demand for coordination of patient care. Coordination is needed especially in those cases in which hospitals have structured healthcare into specialtyoriented units, while a substantial portion of patient care is not limited to single units. From a logistic point of view, this multi-disciplinary patient care creates a tension between controlling the hospital's units, and the need for a control of the patient flow between units. A possible solution is the creation of new units in which different specialties work together for specific groups of patients. A first step in this solution is to identify the salient patient groups in need of multi-disciplinary care. Grouping techniques seem to offer a solution. However, most grouping approaches in medicine are driven by a search for pathophysiological homogeneity. In this paper, we present an alternative logistic-driven grouping approach. The starting point of our approach is a database with medical cases for 3603 patients with peripheral arterial vascular (PAV) diseases. For these medical cases, six basic logistic variables (such as the number of visits to different specialist) are selected. Using these logistic variables, clustering techniques are used to group the medical cases in logistically homogeneous groups. In our approach, the quality of the resulting grouping is not measured by statistical significance, but by (i) the usefulness of the grouping for the creation of new multi-disciplinary units; (ii) how well patients can be selected for treatment in the new units. Given a priori knowledge of a patient (e.g. age, diagnosis), machine learning techniques are employed to induce rules that can be used for the selection of the patients eligible for treatment in the new units. In the paper, we describe the results of the aboveproposed methodology for patients with PAV diseases. Two groupings and the accompanied classification rule sets are presented. One grouping is based on all the logistic variables, and another Artificial Intelligence in Medicine 26 (2002) 87-107

Currently, the data mining and machine learning fields are facing new challenges because of the amount of information that is collected and needs processing. Many sophisticated learning approaches cannot simply cope with large and complex domains, because of the unmanageable execution times or the loss of prediction and generality capacities that occurs when the domains become more complex. Therefore, to cope with the volumes of information of the current realworld problems there is a need to push forward the boundaries of sophisticated data mining techniques. This thesis is focused on improving the efficiency of Evolutionary Learning systems in large scale domains. Specifically the objective of this thesis is improving the efficiency of the Bioinforma tic Hierarchical Evolutionary Learning (BioHEL) system, a system designed with the purpose of handling large domains. This is a classifier system that uses an Iterative Rule Learning approach to generate a set of rules one by one using consecutive Genetic Algorithms. This system have shown to be very competitive so far in large and complex domains. In particular, BioHEL has obtained very important results when solving protein structure prediction problems and has won related merits, such as being placed among the best algorithms for this purpose at the Critical Assessment of Techniques for Protein Structure Prediction (CASP) in 2008 and 2010, and winning the bronze medal at the HUMIES Awards for Human-competitive results in 2007. However, there is still a need to analyse this system in a principled way to determine how the current mechanisms work together to solve larger domains and determine the aspects of the system that can be improved towards this aim. To fulfil the objective of this thesis, the work is divided in two parts. In the first part of the thesis exhaustive experimentation was carried out to determine ways in which the system could be improved. From this exhaustive analysis three main weaknesses are pointed out: a) the problem-dependancy of parameters in BioHEL's fitness function, which results in having a system difficult to set up and which requires an extensive preliminary experimentation to determine the adequate values for these parameters; b) the execution time of the learning process, which at the moment does not use any parallelisation techniques and depends on the size of the training sets; and c) the lack of global supervision over the generated solutions which comes from the usage of the Iterative Rule Learning paradigm and produces larger rule sets in which there is no guarantee of minimality or maximal generality. The second part of the thesis is focused on tackling each one of the weaknesses abovementioned to have a system capable of handling larger domains. First a heuristic approach to v set parameters within BioHEL's fitness function is developed. Second a new parallel evaluation process that runs on General Purpose Graphic Processing Units was developed. Finally, post-processing operators to tackle the generality and cardinality of the generated solutions are proposed. By means of these enhancements we managed to improve the BioHEL system to reduce both the learning and the preliminary experimentation time, increase the generality of the final solutions and make the system more accessible for end-users. Moreover, as the techniques discussed in this thesis can be easily extended to other Evolutionary Learning systems we consider them important additions to the research in this field towards tackling large scale domains. vi

M achine learning is the study of computational methods for improving performance by mechanizing the acquisition of knowledge from experience. Expert performance requires much domain-specific knowledge, and knowledge engineering has produced hundreds of AI expert systems that are now used regularly in industry. Machine learning aims to provide increasing levels of automation in the knowledge engineering process, replacing much time-consuming human activity with automatic techniques that improve accuracy or efficiency by discovering and exploiting regularities in training data. The ultimate test of machine learning is its ability to produce systems that are used regularly in industry, education, and elsewhere.

Small-sized hotels that prevail in the tourist destination of Serbia rarely use any kind of property management or intelligence systems. The issue that pervades throughout this paper is related to the ways in which they can benefit from data mining. This paper discusses data mining practical application of making predictions of future monthly in-house nights for small hotels. The selected data mining algorithms have been analyzed and compared in order to choose the optimal method for application of this case study. An empirical application of methods demonstrates that it can generate reasonably accurate forecasts and can be useful to managers in their evaluation of the future occupancy rate. Furthermore, it considers which of the four algorithms is best suited for other applications in the hospitality industry. Mali hoteli, koji zauzimaju značajno mesto na tursitičkom tržištu Srbije, retko koriste neki vid savremenih sistema za upravljanje imovinom hotela ili inteligentne sisteme. Pitanje koje se prožima kroz ovaj rad tiče se načina na koji mali hoteli mogu imati koristi od tehnika dubinske analize podataka (data mining). U ovom radu se razmatra praktična primena algoritama za dubinsku analizu podataka u cilju predviđanja mesečne popunjenosti kapaciteta hotela. Izabrani algoritmi se analiziraju i upoređuju u cilju izbora optimalnih metoda za primenu ove studije slučaja. Empirijska primena metoda pokazuje da se generišu prilično tačne prognoze, što može biti od velike pomoći rukovodstvu u proceni budućih stopa popunjenosti hotelskih kapaciteta. Takođe se razmatraju i druge mogućnosti u okviru kojih se četiri izabrana algoritma za dubinsku analizu podataka mogu primeniti u hotelskoj industriji.

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. -Users may download and print one copy of any publication from the public portal for the purpose of private study or research -You may not further distribute the material or use it for any profit-making activity or commercial gain -You may freely distribute the URL identifying the publication in the public portal If you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim.

We address the issue of on-line detection of communication problems in spoken dialogue systems. The usefulness is investigated of the sequence of system question types and the word graphs corresponding to the respective user utterances. By applying both ruleinduction and memory-based learning techniques to data obtained with a Dutch train time-table information system, the current paper demonstrates that the aforementioned features indeed lead to a method for problem detection that performs significantly above baseline. The results are interesting from a dialogue perspective since they employ features that are present in the majority of spoken dialogue systems and can be obtained with little or no computational overhead. The results are interesting from a machine learning perspective, since they show that the rule-based method performs significantly better than the memory-based method, because the former is better capable of representing interactions between features.

In this paper we study two types of machine learning techniques, rule-induction and memorybased learning, for error detection in spoken dialogue systems. The learners are trained and tested on two tasks: predicting whether the current user utterance will cause problems, and identifying whether the previous user utterance has caused a problem in the ongoing dialogue. We focus on a variety of features readily available in the majority of spoken dialogue systems: dialogue history, recognized words, and prosodic ...

To facilitate the development of speech enabled applications and services, we have been working on an example-based semantic grammar authoring tool. Previous studies have shown that the tool has not only significantly reduced the grammar development effort but also yielded grammars of better qualities. However, the tool requires extra human involvement when ambiguities exist in the process of grammar rule induction. In this paper we present an algorithm that is able to automatically resolve the segmentation ambiguities, hence acquire the language expressions for the concepts involved. Preliminary experiment results show that the Expectation-Maximization algorithm we investigated has not only eliminated the human involvement in ambiguity resolution but also improved the overall spoken language understanding accuracy.

The illegal use of electricity, defective meters, and a malfunctioning infrastructure are major causes of Non-Technical Losses (NTLs) in electric distribution systems. Although the use of supervised machine learning techniques to detect NTLs has been widely studied, further research is needed in order to address some significant challenges. (i) Given that fraudulent consumers remarkably outnumber non-fraudulent ones, the imbalanced nature of the dataset can have a major negative impact on the performance of supervised machine learning methods. (ii) Given the large number of dimensions present in the time series data used for training and testing classifiers, advanced signal processing techniques are required in order to extract the most relevant information. (iii) The effectiveness of classifiers must be evaluated using meaningful performance measures for imbalanced data. This paper proposes a framework that addresses the three previous challenges. The core of the proposed framework is the application of the Maximal Overlap Discrete Wavelet-Packet Transform (MODWPT) for feature extraction from time series data and the Random Undersampling Boosting (RUSBoost) algorithm for NTL detection. Moreover, our framework is evaluated using an extensive list of performance metrics. Experiments show that the MODWPT combined with the RUSBoost algorithm can significantly improve the quality of NTL predictions.

Sensitivity to distributional characteristics of sequential linguistic and nonlinguistic stimuli, have been shown to play a role in learning the underlying structure of these stimuli. A growing body of experimental and computational research with (artificial) grammars suggests that learners are sensitive to various distributional characteristics of their environment (Kuhl, 2004; Onnis, Monaghan, Richmond & Chater, 2005; Rohde & Plaut, 1999). We propose that, at a higher level, statistical characteristics of the full sample of stimuli on which learning is based, also affects learning. We provide a statistical model that accounts for such an effect, and experimental data with the Artificial Grammar Learning (AGL) methodology, showing that learners also are sensitive to distributional characteristics of a full sample of exemplars.

Fairness, Accountability, Transparency and Explainability have become strong requirements in most practical applications of Artificial Intelligence (AI). Fuzzy sets and systems are recognized world-wide because of their outstanding contribution to model AI systems with a good interpretability-accuracy tradeoff. Accordingly, fuzzy sets and systems are at the core of the socalled Explainable AI. ExpliClas is a software as a service which paves the way for interpretable and self-explainable intelligent systems. Namely, this software provides users with both graphical visualizations and textual explanations associated with intelligent classifiers automatically learned from data. This paper presents the new functionality of ExpliClas regarding the generation, evaluation and explanation of fuzzy decision trees along with fuzzy inference-grams. This new functionality is validated with two well-known classification datasets (i.e., Wine and Pima), but also with a real-world beer-style classifier.

The nested generalized exemplar theory accomplishes learning by storing objects in Euclidean n-space, as hyperrectangles. Classification of new data is performed by computing their distance to the nearest "generalized exemplar" or hyperrectangle. This learning method allows the combination of the distancebased classification with the axis-parallel rectangle representation employed in most of the rule-learning systems. In this paper, we propose the use of evolutionary algorithms to select the most influential hyperrectangles to obtain accurate and simple models in classification tasks. The proposal has been compared with the most representative models based on hyperrectangle learning; such as the BNGE, RISE, INNER, and SIA genetics based learning approach. Our approach is also very competitive with respect to classical rule induction algorithms such as C4.5Rules and RIPPER. The results have been contrasted through non-parametric statistical tests over multiple data sets and they indicate that our approach outperforms them in terms of accuracy requiring a lower number of hyperrectangles to be stored, thus obtaining simpler models than previous NGE approaches. Larger data sets have also been tackled with promising outcomes.

This paper presents the development of the N-Spherical Minimalist Machine Learning (MML) classifier, an innovative model within the Minimalist Machine Learning paradigm. Using N-spherical coordinates and concepts from metaheuristics and associative models, this classifier e ectively addresses challenges such as data dimensionality and class imbalance in complex datasets. Performance evaluations using the F measure and balanced accuracy demonstrate its superior e ciency and robustness compared to state-of-the-art classifiers. Statistical validation is conducted using the Friedman and Holm tests. Although currently limited to binary classification, this work highlights the potential of minimalist approaches in machine learning for classification of highly dimensional and imbalanced data. Future extensions aim to include multi-class problems and mechanisms for handling categorical data.

Discretization is the process of dividing a continuous-valued base attribute into discrete intervals, which highlight distinct patterns in the behavior of a re-lated goal attribute. In this paper, we present an in-tegrated visual framework in which several discretiza-tion ...

Acyclic conjunctive queries form a polyno-mially evaluable fragment of definite non-recursive first-order Horn clauses. Labeled graphs, a special class of relational struc-tures, provide a natural way for represent-ing chemical compounds. We propose an al-gorithm specific to ...

The goal of this research was to examine mechanisms underlying early induction-specifically, the relation between induction and categorization. Some researchers argue that even early in development, induction is based on category-membership information, whereas others argue that early induction is based primarily on similarity. Children 4 and 5 years of age participated in two types of tasks: categorization and induction. Both tasks were performed with artificial animal-like categories in which appearance was pitted against category membership. Although the children readily acquired categorymembership information and subsequently used this information in categorization tasks, they ignored category membership during the induction task, relying instead on the appearance of items. These results support the idea that early in development, induction is similarity based.

In supervised learning the inductive algorithm seeks to develop a conceptual description, or prescriptive model, from examples or objects that have been pre-classified. On the other hand, in unsupervised learning, or clustering, the task of the algorithm is to group non-classified examples into clusters of examples, building a descriptive model. Central to all of the goals of clustering is the notion of similarity (or dissimilarity) between the individual objects being clustering. The key distinction between prediction and description is that prediction has as its objective a unique variable, known as the class attribute in supervised learning, while in descriptive problems no single variable is central to the problem. In this work we propose an approach, as well as a particular implementation of this approach, were a combination of traditional clustering and inductive learning are used to find and interpret special patterns from a database in order to extract useful knowledge to represent real world domains. This approach helps the user to make explicit his/her prior knowledge relevant to the problem allowing to organize the set of objects in different clusters into a set of classes considered to be semantically interpretable by the user.

In supervised learning the inductive algorithm seeks to develop a conceptual description, or prescriptive model, from examples or objects that have been pre-classified. On the other hand, in unsupervised learning, or clustering, the task of the algorithm is to group non-classified examples into clusters of examples, building a descriptive model. Central to all of the goals of clustering is the notion of similarity (or dissimilarity) between the individual objects being clustering. The key distinction between prediction and description is that prediction has as its objective a unique variable, known as the class attribute in supervised learning, while in descriptive problems no single variable is central to the problem. In this work we propose an approach, as well as a particular implementation of this approach, were a combination of traditional clustering and inductive learning are used to find and interpret special patterns from a database in order to extract useful knowledge to re...

In this paper we explore the effects of query and database size on news story classification performance. Memory Based Reasoning (MBR) (a k-nearest neighbor method) used as the classification method. There are 360 different possible codes. Close matches to a new story are found using an already coded training database of about 87,000 stories from the Dow Jones Press Release News Wire, and a Connection-Machine Document Retrieval system (CMDRS, [Stanfill]) that supports full text queries, as the underlying match engine. By combining the codes from the near matches, new stories are coded with a recall of about 80% and precision of about 70%, as reported in [Masand]. When the query size is varied from 10 terms to more than 200 terms (matched against the full-text of documents) the recallprecision product changes from 0.53 to 0.6. While this is a significant change, we find that moderate sized queries of 40-80 terms can suffice for finding relevant matches for classification. By changing the size of database from 10,000 stories to 80,000 we found that the recall-precision product changed from 0.22 to 0.57. This shows that with our current MBR approach the database size can't be reduced significantly without compromising performance. We also find that fewer number of retrieved matches are needed with larger query and database sizes.

We live in a world submerged with more information than ever before. We express information or data mathematically and it is growing faster than ever. If the data is imperfect, out of context or otherwise contaminated, it can lead to decisions that could undermine the competitiveness of an enterprise or damage the personal lives of individuals. Therefore, knowledge representation plays an important role in dealing with many aspects of problem solving. In order to obtain complete information systems, we use a mathematical tool, namely, Rough Set Theory (RST), which was introduced by Pawlak in 1982 as a way to deal with data analysis based on approximation methods in information systems. RST is a novel approach to cope with imperfect data analysis as well. In essence, the theory extends the classical crisp set to rough set by defining lower and upper approximations for any subset of a nonempty finite universe. The theory presupposes that with every object of the universe some informat...

There are inherent open problems arising when developing and running Intelligent Environmental Decision Support Systems (IEDSS). During daily operation of IEDSS several open challenge problems appear. The uncertainty of data being processed is intrinsic to the environmental system, which is being monitored by several on-line sensors and off-line data. Thus, anomalous data values at data gathering level or even uncertain reasoning process at later levels such as in diagnosis or decision support or planning can lead the environmental process to unsafe critical operation states. At diagnosis level or even at decision support level or planning level, spatial reasoning or temporal reasoning or both aspects can influence the reasoning processes undertaken by the IEDSS. Most of Environmental systems must take into account the spatial relationships between the environmental goal area and the nearby environmental areas and the temporal relationships between the current state and the past states of the environmental system to state accurate and reliable assertions to be used within the diagnosis process or decision support process or planning process. Finally, a related issue is a crucial point: are really reliable and safe the decisions proposed by the IEDSS? Are we sure about the goodness and performance of proposed solutions? How can we ensure a correct evaluation of the IEDSS? Main goal of this paper is to analyse these four issues, review some possible approaches and techniques to cope with them, and study new trends for future research within the IEDSS field.

The exceptionally high virulence of COVID-19 and the patients' precondition seem to constitute primary factors in how pro-inflammatory cytokines production evolves during the course of an infection. We present a System Dynamics Model approach for simulating the patient reaction using two key control parameters (i) virulence, which can be "moderate" or "high" and (ii) patient precondition, which can be "healthy", "not so healthy" or "serious preconditions". In particular, we study the behaviour of Inflammatory (M1) Alveolar Macrophages, IL6 and Active Adaptive Immune system as indicators of the immune system response, together with the COVID viral load over time. The results show that it is possible to build an initial model of the system to explore the behaviour of the key attributes involved in the patient condition, virulence and response. The model suggests aspects that need further study so that it can then assist in choosing the correct immunomodulatory treatment, for instance the regime of application of an Interleukin 6 (IL-6) inhibitor (tocilizumab) that corresponds to the projected immune status of the patients. We introduce machine learning techniques to corroborate aspects of the model and propose that a dynamic model and machine learning techniques could provide a decision support tool to ICU physicians. Infectious pandemic corona-virus disease (COVID-19), caused by severe acute respiratory disease Corona-virus 2 syndrome (SARS-CoV-2) is rapidly spreading worldwide . In the case of COVID-19, a worsening has been observed from 7 to 8 days. However, this only occurs in some patients; and because of an over-reaction of the immune system (Pedersen and Ho, 2020). During this pandemic, the challenge is to diagnose those patients who are not getting worse; and thus free up space for those who need intensive care when they develop respiratory failure due to acute respiratory distress syndrome; the main cause of mortality. In a recent retrospective, multi-center study of 150 confirmed cases of COVID-19 in Wuhan, China, the authors suggest that mortality could be due to hyper-inflammatory sepsis . a

Algorithms for induction of concept descriptions from examples are important tools in the fields of machine learning and knowledge discovery in databases. This paper presents an induction algorithm, named PA3, that learns a set of ordered rules from examples. Functionally inspired in AQ and CN2, this algorithm attempts to generalize the learned rules during the induction process and introduces features that allow tuning of several important parameters. This paper describes PA3 and presents a comparison between this algorithm and CN2 on six data sets (three in the domain of stock market and three in an artificial domain).

Inclusion of domain knowledge in a process of knowledge discovery in databases is a complex but very important part of successful knowledge discovery solutions. In real-life data mining development, non-structured domain knowledge involvement in the data preparation phase and in the final interpretation/evaluation phase tends to dominate. This paper presents an experiment of direct domain knowledge integration in the algorithm that will search for interesting patterns in the data. In the context of stock market prediction work, a recent rule induction algorithm, PA3, was adapted to include domain theories directly in the internal rule development. Tests performed over several Portuguese stocks show a significant increase in prediction performance over the same process using the standard version of PA3. We believe that a similar methodology can be applied to other symbolic induction algorithms and in other working domains to improve the efficiency of prediction (or classification) in knowledge-intensive data mining tasks. DK can be included in the data mining phase through direct integration (implicit or explicit) in the data mining algorithm, or through an associated knowledge base. In the first case, specific changes to the core data mining algorithm must be performed, in order to directly represent the involved domain knowledge through a biasing of the search. In the latter case, a very tight coupling between the domain theory description in the knowledge base and the bias representation language accepted by the learner is need, eventually involving an intermediate knowledge "translator" . Anyway, both of these forms of DK integration tend to need software specifically adapted for each application case, since different kinds of domain knowledge usually involve different representations, and most data mining algorithms (and commercial data mining programs) don't allow the integration any form of DK not contained in the data. Direct integration of DK in data mining software generally intends to direct and focus the pattern search that takes place at that KDD step. This can raise another potential limitation of this technique: If badly directed, the focused search can miss some of the potentially interesting patterns that an unbiased search could find in the data . However, in spite of the limitations and potential problems, we believe that, in some cases, careful DK integration in the data mining step of a KDD process can produce significant improvements in the overall efficiency of the process. This paper presents an experiment that integrates two domain theories directly in a rule induction data mining algorithm. The domain is short-term stock market prediction, and the two theories bias the algorithm, during rule search, against a specific class of rules, and towards another. The theories are tested over five data sets that correspond to multivariate information based on daily quotes of five of the most significant stocks in the Portuguese BVLP stock exchange. The base rule induction algorithm used, PA3 , is a recent general-purpose sequential cover algorithm that combines general-to-specific and specific-to-general search to develop each rule.

A productive way to think about imagistic mental models of physical systems is as though they were sources of quasi‐empirical evidence. People depict or imagine events at those points in time when they would experiment with the world if possible. Moreover, just as they would do when observing the world, people induce patterns of behavior from the results depicted in their imaginations. These resulting patterns of behavior can then be cast into symbolic rules to simplify thinking about future problems and to reveal higher order relationships. Using simple gear problems, three experiments explored the occasions of use for, and the inductive transitions between, depictive models and number‐based rules. The first two experiments used the convergent evidence of problem‐solving latencies, hand motions, referential language and error data to document the initial use of a model, the induction of rules from the modeling results, and the fallback to a model when a rule fails. The third experi...

Rule systems have failed to attract much interest in large data analysis problems because they tend to be too simplistic to be useful or consist of too many rules for human interpretation. We present a method that constructs a hierarchical rule system, with only a small number of rules at each stage of the hierarchy. Lower levels in this hierarchy focus on outliers or areas of the feature space where only weak evidence for a rule was found in the data. Rules further up, at higher levels of the hierarchy, describe increasingly general and strongly supported aspects of the data. We demonstrate the proposed method's usefulness on several classification benchmark data sets using a fuzzy rule induction process as the underlying learning algorithm. The results demonstrate how the rule hierarchy allows to build much smaller rule systems and how the model-especially at higher levels of the hierarchy-remains interpretable. The presented method can be applied to a variety of local learning systems in a similar fashion.

Rule systems have failed to attract much interest in large data analysis problems because they tend to be too simplistic to be useful or consist of too many rules for human interpretation. We present a method that constructs a hierarchical rule system, with only a small number of rules at each stage of the hierarchy. Lower levels in this hierarchy focus on outliers or areas of the feature space where only weak evidence for a rule was found in the data. Rules further up, at higher levels of the hierarchy, describe increasingly general and strongly supported aspects of the data. We demonstrate the proposed method's usefulness on several classification benchmark data sets using a fuzzy rule induction process as the underlying learning algorithm. The results demonstrate how the rule hierarchy allows to build much smaller rule systems and how the model-especially at higher levels of the hierarchy-remains interpretable. The presented method can be applied to a variety of local learning systems in a similar fashion.

Most rule induction algorithms generate rules with simple logical conditions based on equality or inequality relations. This feature limits their ability to discover complex dependencies that may exist in data. This article presents an extension to the sequential covering rule induction algorithm that allows it to generate complex and M-of-N conditions within the premises of rules. The proposed methodology uncovers complex patterns in data that are not adequately expressed by rules with simple conditions. The novel two-phase approach efficiently generates M-of-N conditions by analysing frequent sets in previously induced simple and complex rule conditions. The presented method allows rule induction for classification, regression and survival problems. Extensive experiments on various public datasets show that the proposed method often leads to more concise rulesets compared to those using only simple conditions. Importantly, the inclusion of complex conditions and M-of-N conditions has no statistically significant negative impact on the predictive ability of the ruleset. Experimental results and a ready-to-use implementation are available in the GitHub repository. The proposed algorithm can potentially serve as a valuable tool for knowledge discovery and facilitate the interpretation of rule-based models by making them more concise.

Employee turnover is a serious concern in knowledge based organizations. When employees leave an organization, they carry with them invaluable tacit knowledge which is often the source of competitive advantage for the business. In order for an organization to continually have a higher competitive advantage over its competition, it should make it a duty to minimize employee attrition. This study identifies employee related attributes that contribute to the prediction of employees' attrition in organizations. Three hundred and nine (309) complete records of employees of one of the Higher Institutions in Nigeria who worked in and left the institution between 1978 and 2006 were used for the study. The demographic and job related records of the employee were the main data which were used to classify the employee into some predefined attrition classes. Waikato Environment for Knowledge Analysis (WEKA) and See5 for Windows were used to generate decision tree models and rule-sets. The results of the decision tree models and rule-sets generated were then used for developing a a predictive model that was used to predict new cases of employee attrition. A framework for a software tool that can implement the rules generated in this study was also proposed.

Genetic programming represents a flexible and powerful evolutionary technique in machine learning. The use of genetic programming for rule induction has generated interesting results in classification problems. This paper proposes an evolutionary approach for logical rule induction, which is applied to clinical data. Since logical rules disclose knowledge from the analyzed data, we use such a knowledge to filter features from the target dataset. The results reached by the used dataset have been very promising when used in classification tasks and compared with other methods.

This special issue of Knowledge Based Systems comprises expanded versions of the best papers submitted to the conference AI-2010, the thirtieth SGAI International Conference on Artificial Intelligence, which was held in Cambridge, England from December 14th to 16th 2010. The original papers were judged by the conference's programme committees to be the best refereed papers accepted for each of the two streams: six from the technical stream and five papers from the application stream. All of the papers included here have been re-refereed in their expanded forms.

Classification algorithms usually assume that any example in the training set should contribute equally to the classification model being generated. However, this is not always the case. This paper shows that the contribution of an example to the classification model varies according to many factors, which are application dependent, and can be estimated using what we call a credibility function. The credibility of an entity reflects how much value it aggregates to a task being performed, and here we investigate it in Automatic Document Classification, where the credibility of a document relates to its terms, authors, citations, venues, time of publication, among others. After introducing the concept of credibility in classification, we investigate how to estimate a credibility function using information regarding documents content, citations and authorship using mainly metrics previously defined in the literature. As the credibility of the content of a document can be easily mapped to any other classification problem, in a second phase we focus on content-based credibility functions. We propose a genetic programming algorithm to estimate this function based on a large set of metrics generally used to measure the strength of term-class relationship. The proposed and evolved credibility functions are then incorporated to the Naïve Bayes classifier, and applied to four text collections, namely ACM-DL, Reuters, Ohsumed, and 20 Newsgroup. The results obtained showed significant improvements in both micro-F1 and macro-F1, with gains up to 21% in Ohsumed when compared to the traditional Naïve Bayes.

James Liley Samuel R. Emerson Bilal A. Mateen Catalina A. Vallejos Louis J. M. Aslett Sebastian J. Vollmer 1 Alan Turing Institute, London, UK; 2 MRC Human Genetics Unit, Univ. of Edinburgh, UK; 3 Department of Mathematical Sciences, Durham Univ., UK; 4 Kings College Hospital, London, UK; 5 Wellcome Trust, London, UK; 6 Warwick Mathematics Institute, Univ. of Warwick, UK ⋆ Co-corresponding authors

Lists of if±then rules (i.e. ordered rule sets) are among the most expressive and intelligible representations for inductive learning algorithms. Two extreme strategies searching for such a list of rules can be distinguished: (i) local strategies primarily based on a stepby-step search for the optimal list of rules, and (ii) global strategies primarily based on a one-strike search for the optimal list of rules. Both approaches have their disadvantages. In this paper we present an intermediate strategy. A sequential covering strategy is combined with a one-strike genetic search for the next most promising rule. To achieve this, a new rule-®tness function is introduced. Experimental results on benchmark problems are presented and the performance of our intermediate approach is compared with other rule learning algorithms. Finally, GeSeCo's performance is compared to a more local strategy on a set of tasks in which the information value of individual attributes is varied.

Effective information systems require the existence of explicit process models. A completely specified process design needs to be developed in order to enact a given business process. This development is time consuming and often subjective and incomplete. We propose a method that constructs the process model from process log data, by determining the relations between process tasks. To predict these relations, we employ machine learning technique to induce rule sets. These rule sets are induced from simulated process log data generated by varying process characteristics such as noise and log size. Tests reveal that the induced rule sets have a high predictive accuracy on new data. The effects of noise and imbalance of execution priorities during the discovery of the relations between process tasks are also discussed. Knowing the causal, exclusive, and parallel relations, a process model expressed in the Petri net formalism can be built. We illustrate our approach with real world data in a case study.

Present-day healthcare witnesses a growing demand for coordination of patient care. Coordination is needed especially in those cases in which hospitals have structured healthcare into specialtyoriented units, while a substantial portion of patient care is not limited to single units. From a logistic point of view, this multidisciplinary patient care creates a tension between controlling the hospital's units, and the need for a control of the patient flow between units. A possible solution is the creation of new units in which different specialties work together for specific groups of patients. A first step in this solution is to identify the salient patient groups in need of multidisciplinary care. Grouping techniques seem to offer a solution. However, most grouping approaches in medicine are driven by a search for pathophysiological homogeneity. In this paper, we present an alternative logistic-driven grouping approach. The starting point of our approach is a database with medical cases for 3603 patients with peripheral arterial vascular (PAV) diseases. For these medical cases, six basic logistic variables (such as the number of visits to different specialist) are selected. Using these logistic variables, clustering techniques are used to group the medical cases in logistically homogeneous groups. In our approach, the quality of the resulting grouping is not measured by statistical significance, but by (i) the usefulness of the grouping for the creation of new multidisciplinary units; (ii) how well patients can be selected for treatment in the new units. Given a priori knowledge of a patient (e.g. age, diagnosis), machine learning techniques are employed to induce rules that can be used for the selection of the patients eligible for treatment in the new units. In the paper, we describe the results of the aboveproposed methodology for patients with PAV diseases. Two groupings and the accompanied classification rule sets are presented. One grouping is based on all the logistic variables, and another Artificial Intelligence in Medicine 26 (2002) 87-107

The paper presents the results of research related to the efficiency of the so-called rule quality measures which are used to evaluate the quality of rules at each stage of the rule induction. The stages of rule growing and pruning were considered along with the issue of conflict resolution which may occur during the classification. The work is the continuation of research on the efficiency of quality measures employed in sequential covering rule induction algorithm. In this paper we analyse only these quality measures (8 measures) which had been recognized as effective based on previous conducted research. The study was conducted on approximately 70 benchmark datasets related to classification, regression and survival analysis problems. In the comparisons we analyzed prognostic abilities of the induced rules as well as the complexity of the resulting rule-based data models.

This article presents GuideR, a user-guided rule induction algorithm, which overcomes the largest limitation of the existing methods-the lack of the possibility to introduce user's preferences or domain knowledge to the rule learning process. Automatic selection of attributes and attribute ranges often leads to the situation in which resulting rules do not contain interesting information. We propose an induction algorithm which takes into account user's requirements. Our method uses the sequential covering approach and is suitable for classification, regression, and survival analysis problems. The effectiveness of the algorithm in all these tasks has been verified experimentally, confirming guided rule induction to be a powerful data analysis tool.

Rule-based models are often used for data analysis as they combine interpretability with predictive power. We present RuleKit, a versatile tool for rule learning. Based on a sequential covering induction algorithm, it is suitable for classification, regression, and survival problems. The presence of a user-guided induction facilitates verifying hypotheses concerning data dependencies which are expected or of interest. The powerful and flexible experimental environment allows straightforward investigation of different induction schemes. The analysis can be performed in batch mode, through RapidMiner plug-in, or R package. A documented Java API is also provided for convenience. The software is publicly available at GitHub under GNU AGPL-3.0 license.

This paper presents an implementation of bagging techniques over the heuristic algorithm for induction of classification rules called SA Tabu Miner (Simulated Annealing and Tabu Search data miner). The goal was to achieve better predictive accuracy of the derived classification rules. Bagging (Bootstrap aggregating) is an ensemble method that has attracted a lot of attention, both experimentally, since it behaves well on noisy datasets, and theoretically, because of its simplicity. In this paper we present the experimental results of various bagging versions of the SA Tabu Miner algorithm. The SA Tabu Miner algorithm is inspired by both research on heuristic optimization algorithms and rule induction data mining concepts and principles. Several bootstrap methodologies were applied to SA Tabu Miner, including reducing repetition of instances, forcing repetition of instances not to exceed two, using different percentages of the original basic training set. Various experimental approaches and parameters yielded different results on the compared datasets.

The anomaly-based intrusion detection systems examine current system activity do find deviations from normal system activity. The present paper proposes a method for normal activity description using the Hidden Markov Models (HMM), which is tuned up using the gradient based method. The obtained model is utilized as a baseline, depicting the normal system activity. The main purpose is to distinguish the normal traces of user activity from abnormal ones using the BCJR decoding algorithm. Some results from the conducted simulation experiments are introduced as well.

In this paper a querying environment for analysis of patient clinical data is presented. The data consists of two parts: patients' pathological data and data about corresponding gene expression levels. The querying environment includes a generic algorithm for constructing decision trees, as well as algorithms for discretizing gene expression levels and for searching frequent patterns (itemsets). The algorithms are accessed by means of a query language. The language can be used to simulate various data mining algorithms, such as the one developed by Morishita et al. for Itemset Constrained Clustering.

Based on multi-dominance discernibility matrices, a non-incremental algorithm RIDDM and an incremental algorithm INRIDDM are proposed by means of Dominance-based Rough Set Approach. For the incremental algorithm, when a new object arrives, after updating one row or one column in the matrix, we could get the updated rule sets. Time complexity analysis and experimental results show that the incremental algorithm INRIDDM is superior to some other non-incremental algorithms when dealing with large data sets. This paper also explores the influence of data saturation and data concentration on rule induction algorithms. We come to conclude that the data saturation and data concentration are important for the performance analysis of one learning algorithm.

In this article a new approach to the formalization of inductive inference in terms of non-monotonic inference is proposed. Induction is characterized as closed-world reasoning from the available data, followed by an inductive jump, which consists in assuming that valid conclusions in the database (assuming closed-world) hold also in the rest of the world. This conception of induction results is adequate to characterize those inference processes that could be formalized, that is, those based in analytical procedures of pattern-matching or regularity detection in the available data. The proposed characterization formally describes the implicit deductive processes of induction and its non-monotonic nature, and could be used as an abstract model of the mental process that leads to obtaining inductive hypotheses. This proposal reduces the problem of induction automatization to that of deduction automatization. Also, it constitutes a formal framework that covers several inductive inference methods used in machine learning. Besides it formalizes inductive definitions, which are very common in science and computer science.

Healthcare systems generate a huge data collected from medical tests. Data mining is the computing process of discovering patterns in large data sets such as medical examinations. Blood diseases are not an exception; there are many test data can be collected from their patients. In this paper, we applied data mining techniques to discover the relations between blood test characteristics and blood tumor in order to predict the disease in an early stage, which can be used to enhance the curing ability. We conducted experiments in our blood test dataset using three different data mining techniques which are association rules, rule induction and deep learning. The goal of our experiments is to generate models that can distinguish patients with normal blood disease from patients who have blood tumor. We evaluated our results using different metrics applied on real data collected from Gaza European hospital in Palestine. The final results showed that association rules could give us the relationship between blood test characteristics and blood tumor. Also, it demonstrated that deep learning classifiers has the best ability to predict tumor types of blood diseases with an accuracy of 79.45%. Also, rule induction gave us an explanation of rules that describes both tumor in blood and normal hematology.

Rule-based classifier, that extract a subset of induced rules to efficiently learn/mine while preserving the discernibility information, plays a crucial role in human-explainable artificial intelligence. However, in this era of big data, rule induction on the whole datasets is computationally intensive. So far, to the best of our knowledge, no known method focusing on accelerating rule induction has been reported. This is first study to consider the acceleration technique to reduce the scale of computation in rule induction. We propose an accelerator for rule induction based on fuzzy rough theory; the accelerator can avoid redundant computation and accelerate the building of a rule classifier. First, a rule induction method based on consistence degree, called Consistence-based Value Reduction (CVR), is proposed and used as basis to accelerate. Second, we introduce a compacted search space termed Key Set, which only contains the key instances required to update the induced rule, to conduct value reduction. The monotonicity of Key Set ensures the feasibility of our accelerator. Third, a rule-induction accelerator is designed based on Key Set, and it is theoretically guaranteed to display the same results as the unaccelerated version. Specifically, the rank preservation property of Key Set ensures consistency between the rule induction achieved by the accelerator and the unaccelerated method. Finally, extensive experiments demonstrate that the proposed accelerator can perform remarkably faster than the unaccelerated rule-based classifier methods, especially on datasets with numerous instances.

To facilitate the development of speech enabled applications and services, we have been working on an example-based semantic grammar authoring tool. Previous studies have shown that the tool has not only significantly reduced the grammar development effort but also yielded grammars of better qualities. However, the tool requires extra human involvement when ambiguities exist in the process of grammar rule induction. In this paper we present an algorithm that is able to automatically resolve the segmentation ambiguities, hence acquire the language expressions for the concepts involved. Preliminary experiment results show that the Expectation-Maximization algorithm we investigated has not only eliminated the human involvement in ambiguity resolution but also improved the overall spoken language understanding accuracy.

Shopping intention prediction using decision trees. Millenium, 2(4), 13-22. m 4 14 RESUMO Introdução: O preço é um elemento negligenciado na literatura em marketing devido à complexidade da sua gestão e sensibilidade dos clientes sobre as mudanças de preços. Consequentemente, o processo de tomada de decisões de compra pode ser muito desafiador para o cliente. Objetivo: Criar um modelo para prever a intenção de compra e segmentar os clientes em uma das duas categorias, dependendo da intenção de comprar ou não. Métodos: A amostra é composta por 305 entrevistados, pessoas com idade superior a 18 anos envolvidas na compra de mantimentos para sua casa. A pesquisa foi realizada em fevereiro de 2017. Para criar um modelo, o método decision trees foi usado com os seus vários algoritmos de classificação. Resultados: Todos os modelos, exceto onde foi usado o algoritmo RandomTree, alcançaram uma taxa de classificação relativamente alta (acima dos 80%). A classificação com maior precisão foi de 84,75% com algoritmos J48 e RandomForest. Como não há diferença estatisticamente significativa entre esses dois algoritmos, os autores decidiram escolher o algoritmo J48 e criar uma árvore de decisão. Conclusões: O valor monetário e o nível de preços na loja foram as variáveis mais significativas para a classificação da intenção de compra. No futuro pretende-se comparar esse modelo com algumas outras técnicas de data mining, como redes neurais ou support vector machines, uma vez que essas técnicas alcançaram uma precisão elevada em estudos anteriores neste campo. Palavras-chaves: Modelo; Compra, Árvore de decisão.

RESUMO Introdução: O preço é um elemento negligenciado na literatura em marketing devido à complexidade da sua gestão e sensibilidade dos clientes sobre as mudanças de preços. Consequentemente, o processo de tomada de decisões de compra pode ser muito desafiador para o cliente. Objetivo: Criar um modelo para prever a intenção de compra e segmentar os clientes em uma das duas categorias, dependendo da intenção de comprar ou não. Métodos: A amostra é composta por 305 entrevistados, pessoas com idade superior a 18 anos envolvidas na compra de mantimentos para sua casa. A pesquisa foi realizada em fevereiro de 2017. Para criar um modelo, o método decision trees foi usado com os seus vários algoritmos de classificação. Resultados: Todos os modelos, exceto onde foi usado o algoritmo RandomTree, alcançaram uma taxa de classificação relativamente alta (acima dos 80%). A classificação com maior precisão foi de 84,75% com algoritmos J48 e RandomForest. Como não há diferença estatisticamente s...