Literature Review of various Fuzzy Rule based Systems

Literature Review of the Recent Trends and Applications in various Fuzzy Rule based systems

Ayush K. Varshney*, Vicenç Torra
Department of Computing sciences
Umeå University, Sweden
Emails: varshneyayush90@gmail.com*, vtorra@ieee.org

Abstract

Fuzzy rule based systems (FRBSs) is a rule-based system which uses linguistic fuzzy variables as antecedents and consequent to represent human understandable knowledge. They have been applied to various applications and areas throughout the soft computing literature. However, FRBSs suffers from many drawbacks such as uncertainty representation, high number of rules, interpretability loss, high computational time for learning etc. To overcome these issues with FRBSs, there exists many extensions of FRBSs. This paper presents an overview and literature review of recent trends on various types and prominent areas of fuzzy systems (FRBSs) namely genetic fuzzy system (GFS), hierarchical fuzzy system (HFS), neuro fuzzy system (NFS), evolving fuzzy system (eFS), FRBSs for big data, FRBSs for imbalanced data, interpretability in FRBSs and FRBSs which use cluster centroids as fuzzy rules. The review is for years 2010-2021. This paper also highlights important contributions, publication statistics and current trends in the field. The paper also addresses several open research areas which need further attention from the FRBSs research community.

Keywords: Fuzzy Systems, Genetic Fuzzy Systems, Neuro Fuzzy Systems, Hierarchical Fuzzy Systems, Evolving Fuzzy Systems, Big Data, Imbalanced Data, Cluster centroids, Soft Computing, Machine Learning.

1 Introduction

Rule-based Systems [1] are generally used to model human problem-solving activity and behavior using classical IF-THEN rules. When the antecedent(s) of the rule is satisfied, then the rule is fired. Classical approaches deal with the bivalent logic, but it fails to cover the imprecision and uncertainty

present in the knowledge representation. Fuzzy sets [2] introduced by Zadeh are widely used to handle uncertainties and imprecision. Fuzzy sets with rule-based systems leads to Fuzzy Rule Based Systems (FRBSs, see Table 1 for further abbreviation and their full name) [3], [4]. FRBSs considers fuzzy statements as antecedents and consequents, they can handle uncertainty and are more robust compared to the classical rule-based system. They use Linguistic Variables [5] to represent features in antecedents, whose values are context dependent on the membership function of the feature. Rules generated by FRBSs are human interpretable ${ }^{1}$ and can be used to understand how the system works. A typical rule in FRBSs looks like:

Rule $R_{i}: I f A_{1}$ is $x_{i 1}$ and $A_{2}$ is $x_{i 2}$ and $\ldots$ and $A_{n}$ is $x_{i n}$ then output is $O_{j}$
here, $R_{i}$ is the $i^{t h}$ rule in the rulebase; $A_{1}, A_{2}, \ldots, A_{n}$ are the features in a dataset; $x_{i 1}, x_{i 2}, \ldots x_{i n}$ are the linguisitic variables and $O_{j}$ is the output variable.

FRBSs (or fuzzy system(s)) have been used for more than 40 years in many areas of computer science and engineering, there exists 45,287 articles on fuzzy system(s) in the Scopus database (https://www.scopus.com/). Scopus has a comprehensive data coverage. In case of fuzzy systems, Scopus has the repository of papers from all the major publication venues such as IEEE Transactions on Fuzzy Sets and Systems (TFS), WCCI/IEEE Fuzzy conference, Fuzzy Sets and Systems (FS&S) and many more. Despite little fluctuation, the number of articles published in the area of fuzzy systems is constantly growing which shows the growing attention of research community in FRBSs (see Fig. 1a). Fig. 1b shows the area-wise percentage of the published articles in fuzzy systems highlighting the major area of research in FRBSs.

Table 1: Used abbreviation and their full name.

^[1]

Figure 1: (a) Number of published articles in the area of fuzzy systems throughout the years (up to 2021) (b) Area-wise categorization for the published articles.

The main objective of this article is to provide the literature review of recent trends and applications in the FRBSs during the years 2010-2021. There exists many literature review in the literature such as the following: Jang et. al [6] reviews the structure and parameter optimization for Neuro Fuzzy System (NFS) which was further considered in [7] and [8]; terminologies and usage of Genetic Fuzzy System (GFS) has been reviewed in [9]; Hierarchical Fuzzy System (HFS) and their advances were reviewed in [10] and [11]; online learning in fuzzy systems was reviewed in [12] and [13]; a literature review of three decades for FRBSs, GFS, NFS and HFS has been given in [14]; Explainability perspective of FRBSs has been reviewed in [15]. In this article, all the major extensions and areas where FRBSs have been used in the past 12 years such as FRBSs in big data, imbalanced data, cluster centroids as rule have been considered. These topics are not considered in any other review articles. This work can be considered as an extension of [14] and [15] in the sense that it identify the current trends and applications related to all the major areas of FRBSs based on the number of published articles and the citations related to them during 2010-2021. This paper has considered original research articles related to fuzzy systems and its various types written in English, published in the period 2010 to 2021.

In this paper, we have identified 8 key areas where FRBSs have contributed significantly and present a systematic literature review. These areas are Genetic/ Evolutionary Fuzzy Systems (GFS/EFS) [16], Hierarchical Fuzzy Systems (HFS) [17], Adaptive Neuro Fuzzy Inference System [18] (ANFIS/NFS), Evolving Fuzzy Systems (eFS) [19], FRBSs for Big Data [20], FRBSs for Imbalanced Datasets [21], Clusters centroids as rules in FRBSs

[22] and FRBSs for Interpretability/Explainability [23].
A GFS uses genetic/evolutionary approaches to improve the learning ability of FRBSs. A HFS solves the curse of dimensionality for FRBSs, NFS improves the approximation ability of FRBSs using neural networks and dynamic systems for streaming data is considered in eFS. FRBSs are seen as good solutions for big data and imbalanced data. In the recent years the interpretability in FRBSs has gained popularity due to high dimensional and big data and rules are initialized with cluster centroids to limit the number of rules in FRBSs. This article aims to find the answers to the following research question (RQ):

• RQ1: Which are the fundamental concepts in FRBSs and its variants? Motivation: This is needed to understand the basic knowledge regarding FRBSs and its types.
• RQ2: What are the publishing statistics associated with each type of FRBSs?

Motivation: These statistics can help the researchers to identify the direction of growth in the field. These statistics can also help in identifying the publication venues.

• RQ3: What are the trending areas for each type of FRBSs?

Motivation: Identify the trending areas can help to further enhance the FRBSs, as it is a key issue for primary as well as for secondary researchers.

• RQ4: What are the open problems in FRBSs?

Motivation: Identifying the key problems in FRBSs can help researchers in inducing the state of the art of the field.

The paper is organized as follows: Section II describes the classical FRBSs with its variants; Section III provides the literature review for recent trends in the above-mentioned areas along with the details on the selection procedures, Section IV provides current research trends along with open problems for fuzzy systems, and Section V presents the conclusion.

2 Fuzzy Rule Based Systems (FRBSs):

Classical fuzzy systems are based on Mamdani [4] approach. Its generic structure is shown in Fig. 2. The fuzzification module establishes a mapping between real-valued input data to fuzzy values based on some membership

Figure 2: General structure of Fuzzy Rule Based Systems.
function. Similarly, the defuzzification module establishes a mapping between fuzzy values to real-valued output domain. The data base contains the linguistic variables (LV) and the specific membership function associated with them. Rules are the usual way to organize knowledge in natural language. Usually, Database and Rule base are part of the Knowledge base (KB). Mamdani’s rules involves the use of LVs in antecedent and consequent. Rules are represented by the set of linguistic variables and an output associated with them. I.e., a rule can have multiple inputs and a single output. The rule base contains the set of rules for the specific application. Rule base can be represented in many ways. Rule base either has a list of rules or a decision table which is a compact representation of rules. The fuzzy inference engine infers the fuzzy output from the inputs according to the rules generated in the Rule base.

Mamdani’s FRBSs suffer from some drawbacks which were first highlighted in [24], [25] due to the use of LVs [15]:

1. Lack of flexibility due to rigid partition of input and output space.
2. It is difficult to find a fuzzy partition when input variables are mutually dependent.
3. The homogeneous partition between input and output is inefficient and does not scale well.
4. The size of a KB increases rapidly with the increase in the number of variables and linguistic terms for each variable.

There exist multiple variants of the classical fuzzy rule-based system which addresses these issues in FRBSs:

1. Mamdani FRBSs with input output scaling: Transforming the inputs and outputs to introduce more flexibility was introduced in [26]. The authors introduce a simple linear scaling function of the form below:

$$f(z)=\lambda \cdot z+c$$

Here $\lambda$ is the scaling factor and $c$ is the offset for the variable $z$. Nonlinear scaling is also possible, which may be of the form:

$$f(z)=\operatorname{sign}(z)+|z|^{\alpha}$$

Non-linear scaling factor $(\alpha)$ is used to control the sensitivity in the origin region. This way each n-dimensional rule in the FRBSs will be of the form:
IF $f\left(x_{1}\right)$ is $l_{1 i}$ and $f\left(x_{2}\right)$ is $l_{2 i}$ and $\ldots$ and $f\left(x_{n}\right)$ is $l_{n i}$ then $y$ is $Y$ where $l_{i j}$ is the $j^{\text {th }}$ linguistic variable for the $i^{\text {th }}$ feature.
2. DNF Mamdani Fuzzy rule-based system [27]: Here, each variable in the rule takes its value as a set of linguistic terms. I.e. if $X_{i}$ is a variable, and its term set is $\left\{l_{1}, l_{2}, l_{3}\right\}$ then in a rule variable $X_{i}$ can belong to the set $\left\{l_{1}, l_{2}\right\}$. The variable can belong to the set of linguistic terms in a rule. This helps in reducing the number of rules to avoid the problem of increasing size. Here, a rule may be of the form:
IF $x_{1}$ is $\left\{l_{11}, l_{12}\right\}$ and $x_{2}$ is $\left\{l_{23}\right\}$ and $\ldots$ and $X_{n}$ is $\left\{l_{n 1}, l_{n 2}\right\}$ then $y$ is $Y$
3. Takagi-Sugeno-Kang Fuzzy Rule-Based Systems: Takagi-Sugeno-Kang (TSK) [28], [29] FRBS considers a different form of rules, each rule in TSK-FRBS contains LVs as antecedents and a function of inputs as consequent. TSK FRBS models the output as a function of inputs and hence do not need a defuzzification process. This variant has been preferred in many applications where efficiency is of utmost importance. TSK-FRBS splits the input space into several fuzzy sub-spaces, based on the relationship between input and output. The main drawback of TSK-FRBS is its inability to provide interpretability for its inputoutput relationship. Here, the rule structure looks like:
IF $x_{1}$ is $l_{1 i}$ and $x_{2}$ is $l_{2 i}$ and $\ldots$ and $x_{n}$ is $l_{n i}$ then $y=p_{0}+p_{1} * x_{1}+\ldots+$ $p_{n} * x_{n}$
4. Approximate Mamdani Fuzzy Rule-based Systems: The DNF FRBS includes several items of term sets which can reduce the interpretability of the DNF FRBS. Approximate FRBSs [30] are able to obtain

better accuracy at the cost of loosing interpretability. Each rule in an approximate FRBS has its own fuzzy set instead of using linguistic terms. This approach generates semantic free rules, and has higher power of expressiveness due to the use of different fuzzy sets in each rule. It can adopt different number of rules depending on the complexity of the problem. In terms of drawbacks, approximate FRBSs suffers from the loss of interpretability, and also they can overfit the training data and perform poorly in case of unseen data. A typical rule here is of the form:

5. Multiple Input Multiple Output (MIMO) FRBSs [31]: Here, as the name suggests, there are multiple outputs in a single rule rather than a single output variable in the previous variants of FRBSs. Outputs here are considered independently of each other and computed separately. The rule structure looks like:
IF $x_{1}$ is $l_{1 i}$ and $x_{2}$ is $l_{2 i}$ and $\ldots$ and $x_{n}$ is $l_{n i}$ then $y_{1}$ is $Y_{1}$ and $\ldots y_{n}$ is $Y_{n}$
6. Fuzzy Rule Based Classification Systems (FRBCSs): Fuzzy Rule Based Classification Systems (FRBCSs) [32] is a system which uses fuzzy rules as a learning medium. In classical FRBSs, inputs are mapped to usually 1-D output but in FRBCSs inputs are mapped to one of the class labels. The rule structure looks like:
IF $x_{1}$ is $l_{1 i}$ and $x_{2}$ is $l_{2 i}$ and $\ldots$ and $x_{n}$ is $l_{n i}$ then $y=c$

3 Literature Review

In this section, literature review and recent trends for Genetic/Evolutionary Fuzzy Systems [16], Hierarchical Fuzzy Systems [17], Adaptive Neuro Fuzzy Inference System [18], FRBSs for Big Data [20], FRBSs for Imbalanced Datasets [21], Clusters centroids as rules in FRBSs [22] and FRBSs for Interpretability/Explainability [23] are given. Each topic has one subsection which considers articles in the Scopus database. Table 3 highlights the inclusion and exclusion criteria for the papers to be considered in the paper and Table 2 shows the query structure used for Keywords or Title search in

Table 2: Query structure to search for FRBSs and its type on Scopus database

Table 3: Inclusion and Exclusion criteria for literature review in Scopus database

Scopus database. In Table 2, Type of FRBSs has been used as a proxy for different types of FRBSs such as Genetic fuzzy System; and Acronym has been used as a proxy for the acronym of the type of FRBSs e.g. GFS/EFS for Genetic/Evolutionary Fuzzy System. The statistics related to each type of FRBSs are computed by finding the their name in the title or in the keywords. Areas which have seen a growing number of publication and citation have been considered for trends in each type of FRBSs. Table 5 summarizes the advantages, disadvantages, and the current trends in various types of fuzzy systems.

3.1 Genetic/Evolutionary Fuzzy Systems:

A GFS is a type of Fuzzy Rule Based System which employs evolutionary algorithms [33] such as genetic algorithm [34], and genetic programming [35] for learning purpose in the fuzzy system. Designing FRBSs can be seen as a search problem, e.g., finding the set of rules, tuning the membership parameters. GAs (EAs) are known to handle the large search space for near

Figure 3: Structure of Genetic Fuzzy System (GFS)
optimal solution with a performance measure. Apart from the ability to explore a large search space, GAs can incorporate prior knowledge, in case of fuzzy systems, prior knowledge can be the number of rules, membership functions, linguistic variables and so on.

In GFS, the first step is to identify the areas of FRBSs which can be optimized using GAs or EAs. Fig. 3 shows the areas in which GAs can be used for learning. In general, learning in FRBSs is of two types: learning the KB components and tuning the parameters. Learning KB components can be further divided into four categories namely, rule learning, rule selection, learning DB components, and learning KB simultaneously. Rule learning approaches learns the rules automatically from the data for a predefined DB. Thrift et. al [36] presents the classical proposal on tuning. The generated number of rules can be huge, rule selection selects the best set of rules, [37] presents the first contribution in the field. Learning of DB components involves learning shape of membership functions, number of fuzzy sets and other DB components. Cordon et al. [38] is the pioneer paper in the field. Learning KB components involves learning DB components and rules simultaneously, this leads to a large search space which makes learning more difficult, first such methodology was presented in [39]. Tuning of the parameters is done to improve the system’s performance. Tuning can be done by adjusting the membership function parameters, by adjusting inference systems parameters and by adjusting weights in defuzzification methods. GFS suffers from the disadvantage of genetic algorithms which is computational complexity I.e. GAs search for a solution may take significant time. In the literature, optimized exploration and exploitation can improve the running

time of GAs. Examples include an arithmetic optimization algorithm that does not need to adjust many parameters [40], pairie dog optimization [41], [42] that uses meta-heuristic animal based activities to optimize exploration and exploitation.

In the years 2010-2021, GFSs literature saw several important contributions such as in Hadavandi et al. [43]) have used GFS with Self-organizing maps clustering for next day forecasting. Elhag et al. [44] introduced a pairwise learning framework with GFS for intrusion detection. GM3M index proposed by Gacto et al. [45] is a geometric mean for three metrics accuracy, maximizing semantic interpretability, minimizing rule complexity. The authors use SPEA2 MOEA (Multi-objective Evolutionary Algorithm) to generate pareto optimal solutions for the three metrics. Methodology presented by Alcala et al. [46] deals with the problem of regression in high dimensional data, they also focus on developing simpler rules by adjusting linguistic fuzzy partitions, and in Sanz et al. [47] interval-valued fuzzy sets were used to model the linguistic labels, amplitude for IVFSs was tuned using weak ignorance theorem.

There exists very few review papers in the literature for genetic fuzzy systems. First review paper by Cordon et al. [48] talked about the GFS models, its application, trends, and open questions. A short overview of GFS models was also presented in Herrera et al. [49]. In [50], Herrera reviewed the taxonomy, GFS models, advances in the field of GFS, trends and the future areas related to GFSs. A review for learning Mamdani-type fuzzy rule-based system was provided in Cordon et al. [51]. The paper focused on tools for improving accuracy while designing interpretable GFSs, [51] focuses on aspects other than accuracy (e.g. interpretability), and on reducing the complexity of the model. Synthesis of eFS with special focus on Genetic programming based eFS was given in Koshiyama et al. [52]. Fernandez et al. [9] have presented an overview of Evolutionary fuzzy Systems, their terminologies, applications, areas where evolutionary approaches are useful and knowledge base generation.

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for GFS during the years 2010-2021. Fig. 4a shows the trend for the published articles. As it can be the seen, the number of articles are decreasing (with some fluctuations). Fig. 5a shows the area-wise contribution in the field of fuzzy systems with the field of computer science having the highest number of articles. Fig. 6a shows the number of citations for the articles published in GFS.

Recent trends in GFS

In recent years, the researchers in the field of GFSs have focused in the

Figure 4: Number of published articles during the year 2010-2021: (a) Genetic Fuzzy Systems, (b) Hierarchical Fuzzy Systems, © Neuro-Fuzzy Systems, (d) Evolving Fuzzy Systems, (e) FRBSs for Big Data, (f) Interpretability/Explainability in FRBSs, (g) FRBSs for imbalanced data, (h) Cluster centroids as rules in FRBSs.

Figure 5: Area-wise categorization for the published articles in FRBSs during 2010-2021: a) Genetic Fuzzy Systems, (b) Hierarchical Fuzzy Systems, © Neuro-Fuzzy Systems, (d) Evolving Fuzzy Systems, (e) FRBSs for Big Data, (f) Interpretability/Explainability in FRBSs, (g) FRBSs for imbalanced data, (h) Cluster centroids as rules in FRBSs.

Figure 6: Citations for the published articles during the year 2010-2021: (a) Genetic Fuzzy Systems, (b) Hierarchical Fuzzy Systems, © Neuro-Fuzzy Systems, (d) Evolving Fuzzy Systems, (e) FRBSs for Big Data, (f) Interpretability/Explainability in FRBSs, (g) FRBSs for imbalanced data, (h) Cluster centroids as rules in FRBSs.

Table 4: List of top journals/conferences with the number of articles published in various types of FRBSs between the years 2010-2021
following areas:

1. Multi-objective GFS: There has been many papers in the field of multi-objective GFS where accuracy along with other metrics such as interpretability, complexity are used as objectives. In [53], a multiobjective ant colony optimization [54] has been used to tune IT2FLS parameters for the performance of an Hexapod robot. A multi-objective eFS for intrusion detection has been proposed in [55]. In [45] SPEA2 MOEA was used to obtain pareto fronts between different objectives.
2. Hybrid GFS/eFS: Hybrid form genetic/evolutionary approaches with another methodology are used for learning. In [56], a genetic fuzzy neural network approach has been given which combines the benefit of neural networks (NN) with GAs. In [57], variants of ANFIS with several evolutionary approaches were presented for the prediction of wildfire probability.
3. Variants of fuzzy sets in GFS: There exists several fuzzy sets in the literature such as intuitionistic fuzzy sets, rough set, and type-2 fuzzy sets. Recently, significant focus has been devoted in developing other fuzzy set variants of GFS to capture more uncertainty present in the data. A type-2 evolutionary TSK fuzzy system is presented in [58]. It learns parameters and footprint-of-uncertainty from scratch. A rough-set based variant of GFS was presented in [59] for heart disease diagnosis.
4. Applications of GFS: There has been many areas where GFSs have been making its marks. E.g. the field of power engineering in [60], the

GA based fuzzy system reduces upto $46 \%$ of power consumption. Then in [61] a GA based optimization of FRBSs water pumping system has been proposed; and in [62] a system for prediction of heart disease was given.

3.2 Hierarchical Fuzzy Systems

Conventional fuzzy rule based systems suffers from the curse of dimensionality. I.e., with the increase in the dimensionality the rule base can become huge. Consider an example of a system which models data with 10 attribute where each attribute can be represented by $5-7$ linguistic variables, there are about $5^{10} \sim 7^{10}$ i.e., $10-28$ million possible number of rules. In the age of big data, number of attributes are not limited to 10 , hence conventional (flat) fuzzy systems are highly infeasible for high dimensional data. Hierarchical Fuzzy Systems were first proposed in [63] to overcome the curse of dimensionality. HFS is composed of several low-dimensional fuzzy systems in a hierarchical way. HFSs have also been proved to be Universal approximators [64]. Rules in Hierarchical fuzzy systems are grouped into modules (low-dimensional fuzzy systems) as per their roles in the system. Each module computes a partial solution which is further passed onto the next level modules. Although each module is a fuzzy system, it generates significantly less number of rules than a flat fuzzy system. However, this leads to the reduction in the interpretability of HFS as it would not be possible to interpret each intermediate HFS subsystem.

In general, there are three types of Hierarchical fuzzy systems namely, Serial, Parallel, and Cascaded Hierarchical Fuzzy System. Their structure is given in Figure 7. In serial hierarchical fuzzy systems (7a), output from the previous modules are fed as one of the inputs in the next module. Here in every stage there is only one fuzzy system. Modules have one input from the previous layer module along with one or more input variable. This process is continued until all the input variables are used. In parallel (or aggregated) hierarchical fuzzy systems (7b), the lowest level modules serve as the input for the entire structure. Output of the first level serves as the input of the second level modules. This process continues until the last module. Its output serves as the output of the system. In cascaded hierarchical fuzzy system (7c) each stage is a module which takes all the inputs as parameters. The output of stage 1 module provides the input for stage 2 module. Cascaded HFS are hybrid of FRBSs and Neural networks [65] but since it uses all of the input variables it loses the benefit of reducing the number of rules. Because of that the field has not progressed much in

Figure 7: Types of Hierarchical Fuzzy System (a) Serial (b) Parallel © Cascaded
the past years.
In the years 2010-2021, several important contributions in the field of Hierarchical Fuzzy Systems have been made. For example, Juang et al. [66] provide a particle swarm optimization method for hierarchical fuzzy systems; in Zhang et al. [67] describe a HFS optimized with GAs to develop a robust and accurate traffic prediction system in intelligent transportation problem; Fares et al. [68] present a HFS based framework for detecting water mains system; Lopez et al. [69] develop a hierarchical FRBS for classifying imbalanced data. Qu et al. [70] develop a trust evaluation system using HFS was developed for Infrastructure-as-a-service cloud.

The work done in Wang et al. [71] provides rough analysis and design of HFS. It also highlights that HFSs is a uniform and universal approximator although it can inherit the curse of dimensionality to obtain such general property. In [10], a review of HFS from the perspective of complex systems was done. The paper also highlights that if the functions are not decomposable then it may not be possible to generate HFS, but that for some functions HFS is an universal approximator. Review on the approximation capabilities of HFS was done in [72]. For any continuous function with natural hierarchy, HFS can approximate the system for the desired level of accuracy. A survey paper on HFS which presents the motivation, current trends and open problems in HFS has been presented by Di et al. [11]. A little work has been done to interpret the intermediate variables in HFS. Magdalena et al.

[73] shows that HFS only improve the interpretability when it is capable of decoupling the system into subsystems which are interpreted independently.

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for HFS during the years 2010-2021. Fig. 4b shows the trend in number of publications in the years 2010-2021. There has not been much growth in terms of research articles in the field of HFS. Fig. 5b shows the areas where HFS papers were published. Most of the papers are in the areas of Computer Science, Engineering and Mathematics. Fig. 6b shows the citations for the articles published in the years 2010-2021. The increasing trend means that the literature on this topic is increasingly used or constantly improved.

Recent trends in HFS:

In the recent years, the researchers in the field of HFS have focused in the following areas:

1. Interpretable HFS: Interpretability is one of the main reasons to advocate for the use of FRBSs. Recently many researchers are working towards making interpretable HFS. For example, in [74] the authors compare the hierarchical and serial topology of HFS in terms of complexity and interpretation based on Seesaw method, in [75] the framework to measure the interpretability with participatory user design for user specific applications was proposed.
2. EA based HFS: The structure to choose for HFS is a key problem. In cases where relationship between variables are unknown, the search space of possible HFS models is huge. EAs in general are great at finding near optimal solution in huge search spaces. In case of HFS as well, this has been an active area of research. For example, in [76] a PSO-based interval type-2 HFS was proposed for real-time travel route guidance, and in [77] a PSO based HFS for reference evapotranspiration prediction was proposed.
3. Type-2 HFS: Type-2 fuzzy sets are an extension of type-1 to handle more uncertainty present in the data. Researchers have been working towards extending type-1 HFS to the variants of type-2 HFS. E.g., in [78] a variable selection method for an interval type-2 hierarchical fuzzy system has been presented; in [79] the authors introduced a multi-agent architecture for type-2 beta hierarchical fuzzy system which considers different agents to optimize the structure, tune the parameters for improving accuracy and interpretability.

Figure 8: NFS layers, L1-Input layer; L2-membership layer; L3-rule layer; L4-Normalization; L5-Term layer; L6-Extra layer and L7-Output layer.
4. Applications of HFS: There also exists many areas where HFS finds its application. E.g. in [80], HFS has been used to navigate a mobile robot in an unstructured environment; a HFS was proposed in [81] for the diagnosis of dengue; in [82] HFS has been coupled with software engineering to propose a HFS based complexity measure.

3.3 Neuro Fuzzy System

Neuro Fuzzy Systems (NFS) were first introduced in [18]. They present the fusion of FRBSs with Artificial Neural Network (ANN). FRBSs are capable of handling the uncertainty present in the data along with providing interpretable reasoning which can be understood by humans, but FRBSs lacks the ability to learn the rules on its own. On the other hand, ANN is considered a black-box which can learn from the data while not able to provide the reasoning behind the learning. The basic idea behind NFS is to fuse the human-like reasoning ability of FRBSs and learning ability of ANN. There exists two common ways for extracting rules in NFS: cooperative NFS and concurrent NFS. In cooperative NFS, NN computes the FRBSs parameter from training data and FRBSs generates the interpretable rules; in concurrent NFS, NN work together with FRBSs continuously to create the model. NFS can be seen as either a self-adaptive or a dynamic learning system. Automatic tuning of membership function parameters, learning the structure of NFS or both from the data is done in self-adaptive NFS. In dynamic learning NFS, NFS learns parameters or structure from samples continuously rather than learning only initially. However, with the black-box nature of neural network NFS loses its interpretability.

NFS can have at most 7 layers as shown in Fig. 8, Input, Membership, rule, Normalization, Term, Extra, and Output layer. The first four layers (L1-L4) are responsible for tuning and structuring the antecedent of the rule while the others (L5-L7) are used to tune the consequent part. In Fig. 8, and $\rightarrow$ represents partially connected layers, $\rightarrow$ represents fully connected layers. The input is passed through the input layer (L1) without any manipulation. It provides input to either the membership layer (L2) or the rule layer (L3).

Input layer is partially connected to membership layer (or rule layer) because each input variable is connected only to their partitioned fuzzy sets. In the membership layer (L2), a problem specific membership function is employed which computes the membership degree corresponding to the fuzzy sets. Usually, a T-norm between the inputs from the input layer or membership layer is done in the rule layer (L3). The partial connection with the previous layers allows the rule layer to learn the structure of the rules, the number of nodes in the rule layer determines the number of rules generated by NFS. The normalization layer (L4) in NFS computes the firing strength of each rule. The term layer (L5) computes the consequent(s) of the rule, and has the same number of nodes as rule and normalization layer. The additional layer (L6) is used to map the input from L5 to a polynomial function which is not very common in NFS research community. The output layer generates the final output of the fuzzy system. In general it is a summation of all the input values from layer L6 (or L5). NFS architecture has at least 3 number of layers (L1-L3-L7 layers) and a maximum number of 7 layers (L1…L7 layers). NFS architecture with higher number of layers offer higher presence of FIS components which also leads to higher efficiency but at the same time may decrease the interpretability of NFS.

In the literature of NFS, there has been several important contributions during the years 2010-2021 such as the following ones: in Abiyev et al. [83], a type-2 NFS has been designed for the identification of time-varying systems and authors use clustering methods to equalize time varying methods; in Subramanian et al. [84], NFS have been used in the metacognitive framework to develop metacognitive FRBS; in Cervantes et al. [85] a TSK based NFS for the identification in non-linear system has been proposed; in Chen et al. [86] a hybrid version of NFS with two evolutionary algorithms for the modelling of landslide susceptibility has been proposed; in Feng et al. [87] a method to incorporate broad learning in NFS has been proposed for regression and classification problems; the work in Deng et al. [88] fuses fuzzy logic with NN in hierachical fashion to create robust classification method.

One of the first review for NFS was done in [89] where three NFS models were reviewed keeping the model simple and with the same fuzzy semantics. Following the same principle as in [89], [90] also reviews the NFS models based on NEFCON-model. In [91], a review of NFS for non-linear system identification has been presented. This review also highlights that modelling of NFS is rather a complex procedure and requires user interaction constantly. The review for various methodologies and applications for NFS during the years 2002-2012 has been done in [92]. In [93], pros and cons of the optimization techniques using derivative, non-derivative, and hybrid ap-

proaches for type-2 NFS has been presented. Recently, a compresive review of the architecture of the NFS along with the review of the optimization approaches has been presented (see [94]). There has been several articles which review the use of NFS for various applications. For example, building energy consumption in Naji et al.[95], for estimating power coefficient using NFS in Petkovic et al. [96], etc.

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for NFS during the years 2010-2021. Fig. 4c shows the trend in number of publications in the years 2010-2021. The research field is seeing increasing number of papers in the field of NFS each year. Fig. 5 c shows the areas where NFS papers were published, most of the papers are in the areas of Engineering, Computer Science and Mathematics. Fig. 6 c shows the citations for the articles published in the years 2010-2021, the increasing trend means that the literature on this topic is increasingly used or constantly improved.

Recent trends in NFS

In the recent years, the researchers in the field of NFS have focused in the following areas:

1. Explainable NFS: With the increase in the use of AI/ML in day-today life, Explainability/Interpretability is one of the key areas to improve and many researchers are also working towards making explainable/interpretable NFS. In [97] a deep type-2 FRBSs with the special focus on high dimensional data; in [98] a NFS which uses CNNs to extract features and then uses fuzzy classifiers to generate interpretable results; and many more.
2. EA based NFS: Evolutionary Algorithms present an interesting direction as meta-heuristic approach for the optimization of NFS. For example, in [99] PSO has been used for the parameter tuning of NFS to predict the concentration of benzene in the air, in [100] authors uses reptile search algorithm [42] for predicting swelling potentiality in Egyptian soil, and in [86] PSO was used in NFS for modelling landslide susceptibility.
3. Hybrid NFS: NFS is a fusion of FRBSs with NN. There has been an increasing interest for other hybrid variants of NFS. For example, in [88] the authors proposed a fused hierarchical DNN for classification in image segmentation; in [101] fuzzy c-Means clustering has been used to determine the labelling of the attributes and then NFS was employed for the classification in the traffic management.

4. Applications of NFS: There also exists many areas where NFS finds its application. E.g. prediction of the number of unique online visitors for a journal has been predicted using NFS in [102], modelling of landslide susceptibility was in [86], and the number of fuzzy rules and membership functions are computed using NFS for regression problem in $[103]$.

3.4 Evolving Fuzzy Systems

Nowadays, there is an increasing demand for time-varying systems, systems which can update themselves with the arrival of new data. Data streams are one of the main reasons to look for evolving, adaptive models. Evolving Fuzzy systems (eFS) are a type of FRBSs which can self-adapt the parameters and the structure of the rule with the incoming data stream in online mode. eFS determines whether the incoming data stream can be used to update the parameters, structure, generate a new rule or delete a rule. An on-line clustering between current data and the existing rule is used to check if a new rule will be generated or will be used to update the input space partitioning. A new rule is generated if the new data is some threshold apart from its nearest rule. A rule needs to be deleted before adding a new rule if the rule base already has a predefined maximum number of rules [104]. The rule with the lowest potential is deleted. Potential here represents how much the rule is representative of the data. Consequent of the rule is updated with the incoming data stream. Incoming data can also be an outlier. So in general the consequent is updated in a recursive fashion as done in [105]. Incremental learning is one of the common approaches for training and testing incoming data streams. Incremental learning should take care of the noise and the concept drift with the incoming data. Noise and concept drift represents the change in the relationship between the input vector and the output(s). eFS uses the evolving clustering to detect the concept drift and then update the rule antecedent, and consequent along with managing the number of rules. However, clustering is susceptible to the outliers and noise. Other robust algorithm may present interesting work direction.

The eFS can be any of the standard FRBS, HFS, NFS which incorporates incremental learning approach to evolve itself with the incoming data stream. In eFS for standard FRBSs, evolving clustering methods are used to update the antecedents; and for addition or removal of the rules in rule base. One such method was proposed in [106] for TSK-FRBSs. It updates its rule base with the new incoming data using online clustering. The evolvable extension of NFS, are called adaptive or self-evolving NFS methods. In adaptive NFS,

models learn to update the antecedent part using evolving clustering methods and simultaneously the weight for the consequent(s). With the incoming data stream, in tree like evolvable structure of FRBSs, a model from a set of neighbour is chosen if the performance of the current model is not optimal as done in [107]. In [108] an incremental learning approach for linearly evolving tree like FRBSs, which updates its leaves with sub trees based on model selection test to improve the model’s performance is presented.

There has been several important contribution in the field of eFS which have received attention by the research community during the years 20102021. For example in [106], a multi-input multi-output extension of evolving Takagi-sugeno called eTS + methodology which is recursive (suitable for data streams) was presented. It allows to shrink or expand the rule base based on the age and potential of the rule and also allows to chose variable selection for incoming data stream; a methodology to deal with concept drift and shift in on-line data stream has been presented in [109]; in [110] a self-evolving recurrent fuzzy neural network for prediction in time-varying systems has been proposed. Unlike TSK model, the methodology considers non-linear relationships between the input variables. In [111] a model that learns rule bases from scratch, and based on the incoming data stream, can easily add or remove the rules is presented. The contribution of the rule in the output is the major criteria for addition or removal of the rule. Generalized eFS model which considers projection of high dimensional input data into a single fuzzy set has been presented in [109] this results into more interpretable result. Features in this model can be removed from the rules depending on their relevance, and added in a later stage if needed. A multivariable gaussian eFS model [112] which uses multi-variate membership functions for fuzzy sets and uses recursive clustering methods for its robust behaviour has been proposed. There have not been many review articles for eFS, in [113] a systematic overview for eFS has been presented, they have broadly categorized eFS into two categories: evolvable standard FRBSs and evolvable NFS. Similarly in [14], a brief review of learning and implementation approaches for eFS has been discussed.

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for eFS during the years 2010-2021. Fig. 4d shows the trend in number of publications in the years 2010-2021. The trend for the number of papers hasn’t been consistent but overall there is an average of 45 papers in the field of eFS each year. Fig. 5d shows the areas where eFS papers were published, most of the papers are in the areas of Computer Science, Engineering and Mathematics. Fig. 6c shows the citations for the articles published in the years 2010-2021. The increasing trend for

eFS has seen some fluctuations in the recent years, but the interest in the development and use of eFS is growing.

Recent trends in eFS

In the recent years, the researchers in the field of eFS have focused in the following areas:

1. Neuro-eFS: Evolvable NFS has been one of the prominent approaches in the field of eFS in the recent years where online approaches are used to incorporate data stream for the purpose of tuning or learning the structure of NFS. E.g. in [114], a self-evolving interval type-2 NFS has been presented for system identification and control problem. The architecture present learns the optimal structure of NN with the incoming IT2 data. Similarly in [110] recurrent approaches along with online structure learning algorithms have been used.
2. Structural Updates in eFS: The purpose of Evolvable FRBS is to learn the structure, update the rule base with the incoming new data. For example, in [115] a compression layer has been used for the antecendents to eliminate the irrelevant information from the rules which results in higher generalization; in [116] a structural evolving approach to improve the accuracy-interpretability trade-off has been presented. This latter study also presents several methodologies to reduce the number of rules in the model.
3. Type-II eFS: Type-II fuzzy sets are an extension of type-I fuzzy sets whose membership grade itself is a fuzzy set. There has been an increase in interest for type-II variants of eFS in the recent years. E.g., in [114] a type-2 evolvable NFS has been proposed; in [117] an evolvable type-II eFS model which considers long-term potentiation if the incoming data stream enhances the previous knowledge and long-term depression if the new data degrades the previous knowledge to update the FRBS model.
4. Applications of eFS: There also exists many areas where eFS are applied. E.g. for online identification of 1000 different channels for Engine test benches in [118], for gene data expression in [119], for thermal modelling of power transformers in [120].

3.5 FRBSs for Big Data

The amount of data collected in today’s world is huge due to internet, mobile devices, social media and many other sources. This huge collection of data

is termed as big data. There exists several definitions of big data: some considers big data if has 3 Vs to 5 Vs [121] (Volume: huge data, Velocity: high speed of data creation, Variety: data has diversity, Veracity: Data must be reliable and Value: worth of the collected data). Some define big data as the data which can not be processed in a single machine [122]. FRBSs are known for generating interpretable models but FRBSs struggle with the scalability issue, so it does not perform well for big data. To solve this issue, researchers have been working over generating distributed, parallel models in the past decade, hence the number of publication in the field has significantly increased (see fig. 4e). The basic idea is to use distributed computing, where data is distributed among several computing nodes. At each node, the splitted data is used to build FRBS model and the output generated from each node is aggregated to get the final output as shown in Fig 9. The problem of interpretability with high dimensional data in case of big data is still a major drawback of fuzzy systems in big data.

Mapreduce by Dean et al. [123] is one of the common approaches for distributing the large data into a cluster of nodes. Mapreduce has two primary steps, Map and Reduce. First, the Map function distribute the data into nodes and computes some intermediate result then Reduce aggregates the intermediate result into the final output of the model. Chi’s FRBS [32] is compatible with the Mapreduce paradigm, the first model which uses Chi’s method with Mapreduce called CHi-FRBCS-BigData was given in [124]. Initially, fuzzy partition of each feature in the database is created based on the level of chosen granularity. Then the whole dataset is distributed among the n computing nodes. In the second stage, each of the nodes independently computes its rule base based on Chi’s or any rule learning method. In the third stage, the rule base from each node is aggregated and in case of conflict the rule with the higher rule weight is considered. The initial computed DB and final rule base results in the KB for this type of models.

There has been several important contributions to the field of fuzzy systems with Big Data which have received attention in the research community during the years 2010-2021. Del et. al [124] describe above is one example. This is about an FRBS model for the classification in big data context using the Mapreduce approach on the Hadoop platform. This work was further extended using cost-sensitive learning for imbalanced dataset in [125]. Both approaches does not lead to the same rules if Chi’s original algorithms was used. This drawback was removed in [122]. In [126] a distributed fuzzy classifier which employs frequent pattern for pruning unnecessary rules is introduced. A brief overview of Chi-FRBS approaches for big data along with granularity analysis has been given in [127]. Their results suggest that

Figure 9: Flow of FRBS with BigData
increasing granularity increases utility, but having too high granularity ( $>5$ in their experiments) can also cost loss of utility. The work in [128] highlights that fuzzy systems for big data are still in its early stage and provides a brief review on the following key areas: classification, association rule mining and clustering for big data. In [129], an overview of fuzzy techniques has been presented where fuzzy approaches have been used to handle uncertainty and for modelling purposes. They also highlight that most of the current fuzzy work in Big Data only consider volume as their definition of big data.

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for Big data in FRBSs during the years 2010-2021. Fig. 4 e shows the trend in number of publications in the years 2010-2021. The number of publications in the field haven’t been consistent. Fig. 5e shows the areas where papers in FRBS in big data were published. Most of the papers are in the areas of Computer Science, Engineering and Mathematics. Fig. 6e shows the number of citations for the articles published in the years 2010-2021. The field was introduced recently and the trend suggests the research in FRBSs for Big Data has been receiving constant attention from the research community.

Recent trends in FRBSs for Big Data

In the recent years, the researchers in the field of FRBSs for Big Data have focused in the following areas.

1. Neuro FRBS-BigData: In the present stage, neural networks is one of the key machine learning areas where progress is being made constantly. FRBS-BigData is no different. There has been several re-

cent articles which present NFS for big data such as the following. In [130], multi-layer NFS with time varying learning rate for Big data is presented, in [131] a two-stage optimization for privacy-preserving Hierarchical-NFS has been presented. In first stage, parameters are trained using distributed K-means and AO algorithm is used for coordination among nodes at higher levels.
2. Structural Updates in FRBS-BigData: Many researchers in FRBSBigData are also looking to optimize the rule complexity and the running time of the model such as in [132] the author present a support based filtering of rules instead of rule weight based filtering for the classification task in FRBS-BigData. This system results in an improved accuracy and running time. In [133] an interpretable FRBS which tries to reduce the number of rules as well as the number of features in a rule has been presented.
3. Applications of FRBS-BigData: There also exists many areas where FRBS-BigData finds its application. E.g. in [134] brain disease prediction using medical image processing has been proposed, learning model for supply chain management using ANFIS for big data has been proposed in [135].

3.6 FRBSs for Interpretability/Explainability

FRBSs are widely used because of their ability to provide human understandable knowledge in the form of linguistic rules against their black-box counterparts. FRBS models are interpretable by design which provides the answer to what will be the output of the model and why. The interpretable structure of the FRBSs can be represented in several forms such as: TSK, Mamdani and many others as defined in section 2. A model which is highly interpretable but has low utility will never be used, so interpretability alone is not sufficient. Two generic types of fuzzy modelling approaches are famous, Linguistic fuzzy modelling (LFM) where the primary objective is to have high semantic interpretabilty while Precise fuzzy modelling (PFM) focus on highly accurate fuzzy models. There are many aspects in the design of FRBS which improve the interpretability and accuracy of the model. E.g. the granularity level, the membership function and many more. Interpretability of FRBSs is not only associated to classical FRBS but also to HFS, NFS, GFS. HFS divides the large FRBS system into subsystems to reduce the number of rules. Each of the subsystem generates an intermediate variable which is used as input in the next level. Magdalena et. al [136] highlights that the

use of intermediate variables leads to the black-box FRBS. A comprehensive study on the interpretability of the HFS is yet to be done. In GFS, the evolutionary algorithms are used for the learning purpose in the KB [51]; and in NFS, the NNs are used for learning purposes [137].

There is a general belief that there exists a trade-off between accuracy and interpretability, even though it may not be necessarily true. There exists two approaches in the accuracy-interpretability trade-off. Some try to improve the accuracy of the model while maintaining a good interpretability of the system while other methods try to make the model more interpretable while maintaining high accuracy. The accuracy of an interpretable-FRBS is further improved by updating the rule structure which encompasses the granularity, choosing the MFs and their parameters, limiting the number of rules and the number of variables in each rule etc; and improving the learning process using additional components (such as GFS). Interpretability can further be improved by reducing the number of features in the rules; reducing the number of rules (one such approach is to merge close enough rules); simplifying the output of the FRBSs. These approaches focus on the sequential improvement of accuracy for LFM or improving interpretability for PFM. Some researchers also considers these in an interleaved way. E.g., in [138], first an LFM is constructed then its accuracy is improved by generating large fuzzy partitions which generally result in a huge rule base. Then rules are merged again to improve the interpretability of this model. This can be extended till the desired result is obtained. Since interpretability and accuracy are generally conflicting objectives, some researchers focus on multiobjective EA such as SPEA, NSGA-II and many other EAs. In case of big data, the rules generated in FRBS may not be highly interpretable as the number of variables in the rule may be quiet large to obtain the accurate model. Recently, the interpretability problems in FRBS for big data have also been highlighted [139]. In [140], 1-dimensional interpretable rules have been presented. A critical analysis of interpretability for Mamdani or TSK type FRBSs has been presented in [141].

There have been several important contributions in the interpretability field of FRBSs. A field that has received attention by the research community during the years 2010-2021. E.g., in [142] a constrained multi-objective GFS which uses decision trees to initialize the rulebase that results in less rules with less variables due to a reduced search space. A generic framework to evaluate and design interpretable FRBS has been given in [143]. In [144], a NFS that focuses on accuracy along with high interpretability (as defined in [145]) for non-linear systems has been given. Hagras et. al [23] presents an overview and stresses the need of understandable and explainable AI.

There has not been much work regarding interpretability of HFS and there is a lack of interpretable measures for HFS. In [146] an index for HFS has been presented. A detailed review of interpretability in FRBSs until 2010 has been done in [145]. They also highlight that there is no comprehensive measure that quantifies interpretability. Interpretability has to be seen from different characteristics such as number of rules, size of rule, granularity etc. The work in [147] highlights the open issues, new development, trends and achievement for eFS/GFS. A review for Mamdani-type FRBSs in the context of GFS has been presented in [51]. Another review which focuses on the information granule from the the perspective of interpretability-accuracy trade-off has been given in [148].

Table 4 shows top conference/journals in FRBSs and the number of articles published in them for interpretability/explainability in FRBSs during the years 2010-2021. Fig. 4f shows the trend in number of publications in the years 2010-2021. The trend for the number of publications in the field has not been consistent, although from 2014 the number of publications are increasing. Fig. 5 f shows the areas where papers in FRBS in big data were published. Most of the papers are in the areas of Computer Science, Mathematics and Engineering. Fig. 6 f shows the citations for the articles published in the years 2010-2021. The linearly increasing trend suggests the work done is being used or improved constantly.

Recent trends in interpretability/explainability in FRBSs:

In the recent years, the researchers in the field of interpretable FRBS have focused in the following areas:

1. Interpretability of FRBS in Big data: The whole idea of fuzzy systems revolves around the idea that they are not black-box models. In contrast, they follow the interpretable by design paradigm. But as highlighted in [141], it is very difficult for humans to correlate more than two variables in a rule, let alone 10,20 or more variables. In [133], an improvement for horizontal and vertical size of the rule using different filtering approaches, a density based filtering for baseline rule base, considering only the important linguistic variable for interpretable rule, heuristic rule selection for reducing the total number of rules.
2. Interpretability of Hierarchical Fuzzy Systems: The interpretability of HFS has been linked to reducing the complexity of FRBS but at the same time it uses intermediate variables, which raises the question: reducing the number of rules at the cost of intermediate variables (interpretable or not) leads to higher or lower interpretability?. In the recent years many researchers have started looking for this question.

In [136] not all HFS are interpretable, and the interpretability of the HFS depends significantly on the the choice of intermediate variables. In [146] the lack of indices to measure interpretability in HFS has been highlighted.

3.7 Other areas:

Apart from the above mentioned areas, FRBSs have also contributed in the areas discussed in this section.

3.7.1 FRBSs for imbalanced datasets

Real-world problems generally have imbalanced data. Imbalanced datasets are the datasets which have unequal distribution of the output classes. E.g., in spam detection, the number of spam emails are significantly less than the number of legitimate email. Numerous approaches have been proposed in the literature to handle imbalanced datasets. They can be broadly categorized into three categories: Data-level approaches where the training data is modified to obtain the relatively balanced dataset; algorithm level approaches where existing methodologies are modified to improve the classification for minority class; and cost-sensitive approaches where there is a higher penalty associated with the misclassification of the minority class. In the recent years, FRBSs are seen as an effective approach for classification in imbalanced settings where many researchers consider the use of synthetic minority oversampling technique (SMOTE) in data-level approaches. E.g. fernandez et. al [149] choose two neighbouring points at random and generates a synthetic point from the combination of the two; for algorithm level approaches in FRBSs, algorithmic changes such as in [150] which presents a hierarchical FRBCSs to generate fine granularity between majority and minority classes using genetic rule selection; and for cost sensitive approaches the cost are included during evaluation of the model to favor the minority classes such as in [151]. Class imbalance in big data has also become a prominent area. E.g., in [125] a cost sensitive FRBS for imbalanced big data has been considered, in [152] a rule selection method in a spark environment for imbalanced datasets in big data FRBSs has been presented. In [128], a review of eFS approaches which focuses on classification using imbalanced datasets has been presented. Table 4 shows top conference/journals in FRBSs and the number of articles published in them for imbalanced dataset in FRBSs during the years 2010-2021. Fig. 4 g shows the number of publication in the FRBSs for imbalanced data. The trend has been inconsistent but the research in this

Table 5: Advantages, Disadvantages and Applications of various types of Fuzzy systems

direction is still active and has been receiving attention from the research community as can be seen from Fig. 6g. In the the past years, much of the research has been focused over eFS or Big data for FRBS.

3.7.2 Cluster centroids as rules in FRBSs

In FRBSs, the number of rules can be quiet large. Determining the maximum number of rules can be very problem dependent. In the recent years, clustering has gained attention for applications (or areas) which requires a fixed number of rules. For a user defined number of rules c, the data is clustered into c clusters, where each cluster centroid represents a rule of the fuzzy system. There has been many clustering approaches applied to a variety of fuzzy systems. E.g., fuzzy c-Means has been used in [153] to generate hierarchical structured fuzzy rules, in [154] rule structure was initialized using the possibilistic clustering algorithm for NFS; subtractive clustering has been used in [155] to initialize rules for TSK fuzzy systems. These systems generally use a clustering algorithm (a type of c-Means clustering) to model any type of the fuzzy system. Table 4 shows top conference/journals in FRBSs and the number of articles published in them for initializing rules using cluster centroids in FRBSs during the years 2010-2021. Fig. 4h shows the number of publication in FRBSs that use cluster centroids as initial fuzzy rules. The trend has been inconsistent. In general there has been roughly 30 papers each years on the use of cluster centroids in the various types of fuzzy systems. Fig. 6 h shows the number of citations received for the papers published in the field during the years 2010-2021. The increasing trend suggests that the work is continuosly receiving attention from the research community.

4 Open problems in FRBSs:

In the past 40-50 years, FRBSs have been successfully used in various areas. Nevertheless there are still many open problems which can present opportunities for the FRBSs research community:

1. Ethical challenges: With the increase of AI in day-to-day life, ethical challenges such as privacy, fairness, interpretability with the AI models must be handled immediately. With the increase in the number of publicly available datasets, the need to avoid disclosure of sensitive information is also increasing. Very few worls in the fuzzy community

are concerned with the ethical concerns of AI. In general, there exists a trade-off between utility-interpretability, utility-privacy, utilityfairness but further study is needed to reflect upon the relationship between data privacy, fairness and interpretability. FRBSs can be a candidate model to study the relationship between interpretability, fairness and data privacy.
2. Representation of uncertainty: Fuzzy sets are used to represent the uncertainty of the data. They are the interpretable variables in FRBSs. In the current literature, the focus has been given to classical fuzzy sets and type-II fuzzy sets but there exists many other fuzzy sets such as Hesitant fuzzy sets [156], rough sets [157] and so many others. These sets provide the opportunity either enhance the uncertainty representation or consider multiple experts opinion.

Fuzzy sets use membership functions for uncertainty representation. There exists different families of membership function but there is not much work which compares the impact of different types of membership functions on the performance of FRBSs. E.g. in [141], authors have suggested that triangular and trapezoidal MFs are more suitable to XAI than Gaussian MFs. MFs also influence the interpretability of FRBS reasoning, FRBSs automated efficient choice, and partitioning for MFs can be an important factor for FRBSs.
3. Big Data: With the amount of data being produced in today’s world, researchers should consider model’s performance in the big data context. In the recent years, researchers in classical FRBSs model have started working in the field of classification for big data (e.g. [122]) but there has not been much work in other machine learning aspects. Not much focus has been given for big data in other types of FRBSs such as GFS, NFS and HFS. Genetic/evolutionary algorithms can help in parallelization and improving the learning rate for the MapReduce framework. Further study in this area can provide fast and good solutions. The complexity of NFS increases with big data which will require time consuming and costly solutions. An interesting direction can be distributed/federated approaches for hybridization of NNs and FRBSs. This approach may present robust and fast solutions. Even though the motivation for HFS is its ability to overcome the curse of dimensionality, there hasn’t been any significant work which discusses the application of HFS in the big data context.
Another big data problem which is prominently present in the FRBSs

community is the consideration of only volume for big data. I.e. big data comprises the 3-5 Vs, as explained in section 3.5. Nevertheless, in general researchers have used huge data volume as big data. Consideration of other Vs in FRBSs can provide important insights in the respective fields.
4. Evolving nature of Algorithms: In real-world applications, data is often in streaming nature. I.e. data distribution keeps changing with time. This requires continuous development of the developed model. Generally, evolving FCM is used to determine the age and the rule based on the incoming data stream. Evolvable FCM is sensitive to the outliers and noises. A more robust approach in the evolving fuzzy systems may significantly enhance its performance. Apart from this, more focus should be given for the selection of features, input, and the aggregation of rule in eFS. Although, eFS considers streaming data, none (or few) of the papers has considered the impact of high (or low) velocity of incoming data on the performance of eFS. eFS also does not provide the flexibility architecture of the rule and system e.g. Hierarchical or distributed architecture.
5. Aggregators in FRBSs: Aggregation in FRBSs can be used for merging the fuzzy rules and reduces the number of rules in the system, and to get the firing strength of a rule. In general, neighbour rules which after merging does not affect accuracy (much) are merged using an aggregation operator for each feature. The features in a rule of FRBSs are connected through a conjunction which is usually modeled through a t-norm. There exists several other aggregation operator such as OWA [158], Choquet [159] and Sugeno [160] integrals and many more. It is relevant to consider the aggregation functions that can have a significant impact on the performance of the FRBSs. Different aggregation operators on various types of FRBSs.
6. Interpretability/Explainability: Interpretability is a key aspect of FRBSs. Nevertheless in the last few years, critical analysis of fuzzy systems with respect to interpretability has gained attention from researchers. As suggested in [141], if a rule contains more than 2 antecedents it is very hard for the human mind to understand. Further study in this area regarding the loss of accuracy with 2-3 antecedents in the rule may be needed. As mentioned in Section 3.2, the interpretability of HFS needs to be further analysed. Apart from this, focus must be given for new interpretability assessment indices to deal with the complex

FRBSs structure.
7. Model optimization: Optimization of Mamdani and TSK FRBSs is still an open problem. The choice for membership functions, number of rules, defuzzifiers are still done manually, and the automation of this process is an interesting future work direction. Meta-heuristic approaches can be used for this goal.
8. Application in Robotics: Robotics becomes more and more complex and challenging with time. Fuzzy sets and systems can be used to deal with the uncertain information available to the robots. This can be very helpful for robots in their need to navigate in dynamic and unpredictable environment.
9. Deep Fuzzy multi-layer network: In the last few years, significant focus has been given to deep fuzzy neural networks. They incorporate a fuzzy layer in the DNN, and, thus, it offers interpretability to the otherwise black-box model. Due to the deep architecture along with high number of parameters to tune, DFNN has very high computational cost for big data, distributed or parallel. Most of research articles consider gradient descent to optimize the parameters but there exists many other approaches which can result in a better performing DFNN, further analysis in this area is needed.

5 Limitations

This article presents a brief overview of the developments in the different types of fuzzy systems during the years 2010-2021. As discussed above, Fuzzy Systems have been used in various areas and a huge number of papers are published every year, discussion or inclusion of this many papers is quiet difficult. So, this review article suffers from the following limitations:

1. This article provides a brief overview, key papers which received attention in research community and the current trends in the various areas of FRBSs but at the same time it does not cover all the papers, their approaches, and their classification as done in [161].
2. This article does not dive deep into the architecture and methodology of the each type of FRBSs.

3. FRBSs has been in the literature for quiet some time and literature for each type of FRBSs is huge, so coverage of each article published in each various types of FRBSs would be beyond the scope of this article.
4. Since use cases of each type of FRBSs are different, this article does not provide the comparison between each type of FRBSs.

6 Conclusion

This article provide answers to the research questions asked in Section-I. Fundamental concepts of FRBSs have been highlighted by presenting an overview for the various types of classical fuzzy rule based systems and variants of FRBSs. The six variants of classical FRBSs namely Mamdani FRBSs with input-output scaling, DNF FRBSs, TSK FRBSS, Approximate FRBSs, MIMO FRBSs and FRBCSs. The paper identifies eight variants of FRBSs which receive most of the attention from FRBSs community namely, genetic/evolutionary fuzzy system, hierarchical fuzzy systems, neuro fuzzy systems, evolving fuzzy systems, advances in FRBSs for big data, FRBSs for interpretability/explainability, FRBSs for imbalanced datasets and FRBSs that uses cluster centroids as rules. For each of the topics, a brief overview, review papers, trending papers and trending areas related to each topic during the year 2010-2021 are presented. The articles present in the scopus database have been used to get the publishing statistics related to each of the topic for RQ2. Based on the analysis of publishing statistics (in Section 3) related to each variant of FRBS, the paper suggests that the most active trend nowadays in the field of FRBSs lies in NFs, big data and interpretability of FRBSs. Significant reduction in the research community for GFS can be observed. It is also clear that hybridization of two or more types of models are gaining more and more popularity. The open research areas in FRBSs are highlighted in Section 4 which presents the potential future direction for FRBSs out of which Ethical challenges in FRBSs, other Vs of Big data, Model optimization and applications in robotics presents an interesting future direction.

References

[1] J. Durkin, “Expert system,” The Handbook of Applied Expert Systems, vol. 4, p. 15,4 .
[2] L. A. Zadeh, “Information and control,” Fuzzy sets, vol. 8, no. 3, pp. 338353, 1965 .

[3] ——, “Outline of a new approach to the analysis of complex systems and decision processes,” IEEE Transactions on systems, Man, and Cybernetics, no. 1, pp. 28-44, 1973.
[4] E. H. Mamdani, “Application of fuzzy algorithms for control of simple dynamic plant,” in Proceedings of the institution of electrical engineers, IET, vol. 121, 1974, pp. 1585-1588.
[5] L. A. Zadeh, “The concept of a linguistic variable and its application to approximate reasoning-i,” Information sciences, vol. 8, no. 3, pp. 199-249, 1975.
[6] J.-S. Jang, “Anfis: Adaptive-network-based fuzzy inference system,” IEEE transactions on systems, man, and cybernetics, vol. 23, no. 3, pp. 665-685, 1993.
[7] B. Kosko and J. C. Burgess, Neural networks and fuzzy systems, 1998.
[8] D. Karaboga and E. Kaya, “Adaptive network based fuzzy inference system (anfis) training approaches: A comprehensive survey,” Artificial Intelligence Review, vol. 52, no. 4, pp. 2263-2293, 2019.
[9] A. Fernandez, F. Herrera, O. Cordon, M. J. del Jesus, and F. Marcelloni, “Evolutionary fuzzy systems for explainable artificial intelligence: Why, when, what for, and where to?” IEEE Computational intelligence magazine, vol. 14, no. 1, pp. 69-81, 2019.
[10] V. Torra, “A review of the construction of hierarchical fuzzy systems,” International journal of intelligent systems, vol. 17, no. 5, pp. 531-543, 2002.
[11] X.-j. Z. Di Wang and J. Keane, “A survey of hierarchical fuzzy systems,” International journal of computational cognition, vol. 4, no. 1, pp. 18-29, 2006.
[12] P. P. Angelov and X. Zhou, “Evolving fuzzy-rule-based classifiers from data streams,” Ieee transactions on fuzzy systems, vol. 16, no. 6, pp. 1462-1475, 2008.
[13] D. Leite, I. Škrjanc, and F. Gomide, “An overview on evolving systems and learning from stream data,” Evolving systems, vol. 11, no. 2, pp. 181-198, 2020.
[14] V. Ojha, A. Abraham, and V. Snášel, “Heuristic design of fuzzy inference systems: A review of three decades of research,” Engineering Applications of Artificial Intelligence, vol. 85, pp. 845-864, 2019.
[15] A. Moral, C. Castiello, L. Magdalena, and C. Mencar, Explainable Fuzzy Systems. Springer, 2021.
[16] F. Herrera, M. Lozano, et al., “Adaptation of genetic algorithm parameters based on fuzzy logic controllers,” Genetic Algorithms and Soft Computing, vol. 8, no. 1996, pp. 95-125, 1996.

[17] A. E. Gegov and P. M. Frank, “Decomposition of multivariable systems for distributed fuzzy control,” Fuzzy Sets and Systems, vol. 73, no. 3, pp. 329340, 1995.
[18] J.-S. R. Jang et al., “Fuzzy modeling using generalized neural networks and kalman filter algorithm.,” in $A A A I$, vol. 91, 1991, pp. 762-767.
[19] N. Kasabov, “Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 31, no. 6, pp. 902-918, 2001.
[20] I. Robles, R. Alcalá, J. M. Benétez, and F. Herrera, “Evolutionary parallel and gradually distributed lateral tuning of fuzzy rule-based systems,” Evolutionary Intelligence, vol. 2, no. 1, pp. 5-19, 2009.
[21] R. Batuwita and V. Palade, “Fsvm-cil: Fuzzy support vector machines for class imbalance learning,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 3, pp. 558-571, 2010.
[22] R. R. Yager and D. P. Filev, “Generation of fuzzy rules by mountain clustering,” Journal of Intelligent & Fuzzy Systems, vol. 2, no. 3, pp. 209-219, 1994.
[23] H. Hagras, “Toward human-understandable, explainable ai,” Computer, vol. 51, no. 9, pp. 28-36, 2018.
[24] A. Bastian, “How to handle the flexibility of linguistic variables with applications,” International Journal of Uncertainty, Fuzziness and KnowledgeBased Systems, vol. 2, no. 04, pp. 463-484, 1994.
[25] B. Carse, T. C. Fogarty, and A. Munro, “Evolving fuzzy rule based controllers using genetic algorithms,” Fuzzy sets and systems, vol. 80, no. 3, pp. 273-293, 1996.
[26] T. J. Procyk and E. H. Mamdani, “A linguistic self-organizing process controller,” Automatica, vol. 15, no. 1, pp. 15-30, 1979.
[27] A. González, R. Pérez, and J. L. Verdegay, “Learning the structure of a fuzzy rule: A genetic approach,” Fuzzy Systems and Artificial Intelligence, vol. 3, no. 1, pp. 57-70, 1994.
[28] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE transactions on systems, man, and cybernetics, no. 1, pp. 116-132, 1985.
[29] M. Sugeno and G. Kang, “Structure identification of fuzzy model,” Fuzzy sets and systems, vol. 28, no. 1, pp. 15-33, 1988.
[30] L. Duckstein et al., Fuzzy rule-based modeling with applications to geophysical, biological, and engineering systems. CRC press, 1995, vol. 8.
[31] X.-J. Zeng and M. G. Singh, “Approximation theory of fuzzy systems-mimo case,” IEEE Transactions on Fuzzy Systems, vol. 3, no. 2, pp. 219-235, 1995.

[32] Z. Chi, H. Yan, and T. Pham, Fuzzy algorithms: with applications to image processing and pattern recognition. World Scientific, 1996, vol. 10.
[33] T. Bäck and H.-P. Schwefel, “An overview of evolutionary algorithms for parameter optimization,” Evolutionary computation, vol. 1, no. 1, pp. 1-23, 1993.
[34] J. H. Holland, Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press, 1992.
[35] J. R. Koza, “Genetic programming as a means for programming computers by natural selection,” Statistics and computing, vol. 4, no. 2, pp. 87-112, 1994.
[36] P. R. Thrift, “Fuzzy logic synthesis with genetic algorithms.,” in ICGA, 1991, pp. 509-513.
[37] H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, “Selecting fuzzy ifthen rules for classification problems using genetic algorithms,” IEEE Transactions on fuzzy systems, vol. 3, no. 3, pp. 260-270, 1995.
[38] O. Cordón, F. Herrera, and P. Villar, “Generating the knowledge base of a fuzzy rule-based system by the genetic learning of the data base,” IEEE Transactions on fuzzy systems, vol. 9, no. 4, pp. 667-674, 2001.
[39] A. Homaifar and E. McCormick, “Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms,” IEEE transactions on fuzzy systems, vol. 3, no. 2, pp. 129-139, 1995.
[40] L. Abualigah, A. Diabat, S. Mirjalili, M. Abd Elaziz, and A. H. Gandomi, “The arithmetic optimization algorithm,” Computer methods in applied mechanics and engineering, vol. 376, p. 113609, 2021.
[41] A. E. Ezugwu, J. O. Agushaka, L. Abualigah, S. Mirjalili, and A. H. Gandomi, “Prairie dog optimization algorithm,” Neural Computing and Applications, vol. 34, no. 22, pp. 20 017-20 065, 2022.
[42] L. Abualigah, M. Abd Elaziz, P. Sumari, Z. W. Geem, and A. H. Gandomi, “Reptile search algorithm (rsa): A nature-inspired meta-heuristic optimizer,” Expert Systems with Applications, vol. 191, p. 116 158, 2022.
[43] E. Hadavandi, H. Shavandi, and A. Ghanbari, “Integration of genetic fuzzy systems and artificial neural networks for stock price forecasting,” KnowledgeBased Systems, vol. 23, no. 8, pp. 800-808, 2010.
[44] S. Elhag, A. Fernández, A. Bawakid, S. Alshomrani, and F. Herrera, “On the combination of genetic fuzzy systems and pairwise learning for improving detection rates on intrusion detection systems,” Expert Systems with Applications, vol. 42, no. 1, pp. 193-202, 2015.

[45] M. J. Gacto, R. Alcalá, and F. Herrera, “Integration of an index to preserve the semantic interpretability in the multiobjective evolutionary rule selection and tuning of linguistic fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 3, pp. 515-531, 2010.
[46] R. Alcalá, M. J. Gacto, and F. Herrera, “A fast and scalable multiobjective genetic fuzzy system for linguistic fuzzy modeling in high-dimensional regression problems,” IEEE Transactions on Fuzzy Systems, vol. 19, no. 4, pp. 666-681, 2011.
[47] J. Sanz, A. Fernández, H. Bustince, and F. Herrera, “A genetic tuning to improve the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets: Degree of ignorance and lateral position,” International Journal of Approximate Reasoning, vol. 52, no. 6, pp. 751-766, 2011.
[48] O. Cordón, F. Herrera, F. Gomide, F. Hoffmann, and L. Magdalena, “Ten years of genetic fuzzy systems: Current framework and new trends,” in Proceedings joint 9th IFSA world congress and 20th NAFIPS international conference (Cat. No. 01TH8569), IEEE, vol. 3, 2001, pp. 1241-1246.
[49] F. Herrera, “Genetic fuzzy systems: Status, critical considerations and future directions,” International Journal of Computational Intelligence Research, vol. 1, no. 1, pp. 59-67, 2005.
[50] , “Genetic fuzzy systems: Taxonomy, current research trends and prospects,” Evolutionary Intelligence, vol. 1, no. 1, pp. 27-46, 2008.
[51] O. Cordón, “A historical review of evolutionary learning methods for mamdanitype fuzzy rule-based systems: Designing interpretable genetic fuzzy systems,” International journal of approximate reasoning, vol. 52, no. 6, pp. 894913, 2011.
[52] A. S. Koshiyama, R. Tanscheit, and M. M. Vellasco, “Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 9, no. 2, e1251, 2019.
[53] W.-C. Su, C.-F. Juang, and C.-M. Hsu, “Multiobjective evolutionary interpretable type-2 fuzzy systems with structure and parameter learning for hexapod robot control,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2021.
[54] M. Dorigo, G. Di Caro, and L. M. Gambardella, “Ant algorithms for discrete optimization,” Artificial life, vol. 5, no. 2, pp. 137-172, 1999.
[55] S. Elhag, A. Fernández, A. Altalhi, S. Alshomrani, and F. Herrera, “A multiobjective evolutionary fuzzy system to obtain a broad and accurate set of solutions in intrusion detection systems,” Soft computing, vol. 23, no. 4, pp. 1321-1336, 2019.

[56] P. Melin, I. Miramontes, and G. Prado-Arechiga, “A hybrid model based on modular neural networks and fuzzy systems for classification of blood pressure and hypertension risk diagnosis,” Expert Systems with Applications, vol. 107, pp. 146-164, 2018.
[57] A. Jaafari, E. K. Zenner, M. Panahi, and H. Shahabi, “Hybrid artificial intelligence models based on a neuro-fuzzy system and metaheuristic optimization algorithms for spatial prediction of wildfire probability,” Agricultural and forest meteorology, vol. 266, pp. 198-207, 2019.
[58] F. Santoso, M. A. Garratt, and S. G. Anavatti, “T2-ets-ie: A type-2 evolutionary takagi-sugeno fuzzy inference system with the information entropybased pruning technique,” IEEE Transactions on Fuzzy Systems, vol. 28, no. 10, pp. 2665-2672, 2019.
[59] G. T. Reddy, M. Reddy, K. Lakshmanna, D. S. Rajput, R. Kaluri, and G. Srivastava, “Hybrid genetic algorithm and a fuzzy logic classifier for heart disease diagnosis,” Evolutionary Intelligence, vol. 13, no. 2, pp. 185-196, 2020.
[60] M. Mohammadi, M. Abasi, and A. M. Rozbahani, “Fuzzy-ga based algorithm for optimal placement and sizing of distribution static compensator (dstatcom) for loss reduction of distribution network considering reconfiguration,” Journal of Central South University, vol. 24, pp. 245-258, 2017.
[61] A. Mohamed, A. Berzoy, and O. Mohammed, “Optimized-fuzzy mppt controller using ga for stand-alone photovoltaic water pumping system,” in IECON 2014-40th Annual Conference of the IEEE Industrial Electronics Society, IEEE, 2014, pp. 2213-2218.
[62] A. Z. Hameed, B. Ramasamy, M. A. Shahzad, and A. A. S. Bakhsh, “Efficient hybrid algorithm based on genetic with weighted fuzzy rule for developing a decision support system in prediction of heart diseases,” The Journal of Supercomputing, vol. 77, no. 9, pp. 10 117-10 137, 2021.
[63] G. Raju, J. Zhou, and R. A. Kisner, “Hierarchical fuzzy control,” International journal of control, vol. 54, no. 5, pp. 1201-1216, 1991.
[64] L.-X. Wang, “Universal approximation by hierarchical fuzzy systems,” Fuzzy sets and systems, vol. 93, no. 2, pp. 223-230, 1998.
[65] J.-C. Duan and F.-L. Chung, “Cascaded fuzzy neural network model based on syllogistic fuzzy reasoning,” IEEE Transactions on Fuzzy Systems, vol. 9, no. 2, pp. 293-306, 2001.
[66] C.-F. Juang, C.-M. Hsiao, and C.-H. Hsu, “Hierarchical cluster-based multispecies particle-swarm optimization for fuzzy-system optimization,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 1, pp. 14-26, 2009.

[67] X. Zhang, E. Onieva, A. Perallos, E. Osaba, and V. C. Lee, “Hierarchical fuzzy rule-based system optimized with genetic algorithms for short term traffic congestion prediction,” Transportation Research Part C: Emerging Technologies, vol. 43, pp. 127-142, 2014.
[68] H. Fares and T. Zayed, “Hierarchical fuzzy expert system for risk of failure of water mains,” Journal of Pipeline Systems Engineering and Practice, vol. 1, no. 1, pp. 53-62, 2010.
[69] V. López, A. Fernández, M. J. Del Jesus, and F. Herrera, “A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets,” Knowledge-Based Systems, vol. 38, pp. 85-104, 2013.
[70] C. Qu and R. Buyya, “A cloud trust evaluation system using hierarchical fuzzy inference system for service selection,” in 2014 IEEE 28th International Conference on Advanced Information Networking and Applications, IEEE, 2014, pp. 850-857.
[71] L.-X. Wang, “Analysis and design of hierarchical fuzzy systems,” IEEE Transactions on Fuzzy systems, vol. 7, no. 5, pp. 617-624, 1999.
[72] X.-J. Zeng and J. A. Keane, “Approximation capabilities of hierarchical fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 13, no. 5, pp. 659$672,2005$.
[73] L. Magdalena, “Semantic interpretability in hierarchical fuzzy systems: Creating semantically decouplable hierarchies,” Information Sciences, vol. 496, pp. 109-123, 2019.
[74] T. R. Razak, S. S. M. Fauzi, R. A. J. Gining, M. H. Ismail, and R. Maskat, “Hierarchical fuzzy systems: Interpretability and complexity,” Indonesian Journal of Electrical Engineering and Informatics (IJEEI), vol. 9, no. 2, pp. 478-489, 2021.
[75] T. R. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Toward a framework for capturing interpretability of hierarchical fuzzy systems-a participatory design approach,” IEEE Transactions on Fuzzy Systems, vol. 29, no. 5, pp. 1160-1172, 2020.
[76] M. Zouari, N. Baklouti, J. Sanchez-Medina, H. M. Kammoun, M. B. Ayed, and A. M. Alimi, “Pso-based adaptive hierarchical interval type-2 fuzzy knowledge representation system (pso-ahit2fkrs) for travel route guidance,” IEEE Transactions on Intelligent Transportation Systems, 2020.
[77] D. K. Roy, K. K. Saha, M. Kamruzzaman, S. K. Biswas, and M. A. Hossain, “Hierarchical fuzzy systems integrated with particle swarm optimization for daily reference evapotranspiration prediction: A novel approach,” Water Resources Management, vol. 35, no. 15, pp. 5383-5407, 2021.

[78] X.-J. Wei, D.-Q. Zhang, and S.-J. Huang, “A variable selection method for a hierarchical interval type-2 tsk fuzzy inference system,” Fuzzy Sets and Systems, 2021.
[79] Y. Jarraya, S. Bouaziz, H. Hagras, and A. M. Alimi, “A multi-agent architecture for the design of hierarchical interval type-2 beta fuzzy system,” IEEE Transactions on Fuzzy Systems, vol. 27, no. 6, pp. 1174-1188, 2018.
[80] N. Krichen, M. S. Masmoudi, and N. Derbel, “Autonomous omnidirectional mobile robot navigation based on hierarchical fuzzy systems,” Engineering Computations, vol. 38, no. 2, pp. 989-1023, 2021.
[81] M. Alrashoud, “Hierarchical fuzzy inference system for diagnosing dengue disease,” in 2019 IEEE International Conference on Multimedia $\mathcal{E}$ Expo Workshops (ICMEW), IEEE, 2019, pp. 31-36.
[82] T. R. Razak, J. M. Garibaldi, and C. Wagner, “A measure of structural complexity of hierarchical fuzzy systems adapted from software engineering,” in 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2019, pp. 1-7.
[83] R. H. Abiyev, O. Kaynak, T. Alshanableh, and F. Mamedov, “A type-2 neuro-fuzzy system based on clustering and gradient techniques applied to system identification and channel equalization,” Applied Soft Computing, vol. 11, no. 1, pp. 1396-1406, 2011.
[84] K. Subramanian, S. Suresh, and N. Sundararajan, “A metacognitive neurofuzzy inference system (mcfis) for sequential classification problems,” IEEE Transactions on Fuzzy Systems, vol. 21, no. 6, pp. 1080-1095, 2013.
[85] J. Cervantes, W. Yu, S. Salazar, and I. Chairez, “Takagi-sugeno dynamic neuro-fuzzy controller of uncertain nonlinear systems,” IEEE Transactions on Fuzzy Systems, vol. 25, no. 6, pp. 1601-1615, 2016.
[86] W. Chen, M. Panahi, P. Tsangaratos, H. Shahabi, I. Ilia, S. Panahi, S. Li, A. Jaafari, and B. B. Ahmad, “Applying population-based evolutionary algorithms and a neuro-fuzzy system for modeling landslide susceptibility,” Catena, vol. 172, pp. 212-231, 2019.
[87] S. Feng and C. P. Chen, “Fuzzy broad learning system: A novel neuro-fuzzy model for regression and classification,” IEEE transactions on cybernetics, vol. 50, no. 2, pp. 414-424, 2018.
[88] Y. Deng, Z. Ren, Y. Kong, F. Bao, and Q. Dai, “A hierarchical fused fuzzy deep neural network for data classification,” IEEE Transactions on Fuzzy Systems, vol. 25, no. 4, pp. 1006-1012, 2016.
[89] D. Nauck, “Neuro-fuzzy systems: Review and prospects,” in In Proceedings of Fifth European Congress on Intelligent Techniques and Soft Computing (EUFIT’97), Citeseer, 1997, pp. 1044-1053.

[90] A. Nürnberger, D. Nauck, and R. Kruse, “Neuro-fuzzy control based on the nefcon-model: Recent developments,” Soft Computing, vol. 2, no. 4, pp. 168182, 1999.
[91] R. Babuška and H. Verbruggen, “Neuro-fuzzy methods for nonlinear system identification,” Annual reviews in control, vol. 27, no. 1, pp. 73-85, 2003.
[92] S. Kar, S. Das, and P. K. Ghosh, “Applications of neuro fuzzy systems: A brief review and future outline,” Applied Soft Computing, vol. 15, pp. 243259, 2014.
[93] S. Hassan, M. A. Khanesar, E. Kayacan, J. Jaafar, and A. Khosravi, “Optimal design of adaptive type-2 neuro-fuzzy systems: A review,” Applied Soft Computing, vol. 44, pp. 134-143, 2016.
[94] N. Talpur, S. J. Abdulkadir, H. Alhussian, N. Aziz, A. Bamhdi, et al., “A comprehensive review of deep neuro-fuzzy system architectures and their optimization methods,” Neural Computing and Applications, pp. 1-39, 2022.
[95] S. Naji, S. Shamshirband, H. Basser, A. Keivani, U. J. Alengaram, M. Z. Jumaat, and D. Petković, “Application of adaptive neuro-fuzzy methodology for estimating building energy consumption,” Renewable and Sustainable Energy Reviews, vol. 53, pp. 1520-1528, 2016.
[96] D. Petković, Ž. Čojbašič, and V. Nikolić, “Adaptive neuro-fuzzy approach for wind turbine power coefficient estimation,” Renewable and Sustainable Energy Reviews, vol. 28, pp. 191-195, 2013.
[97] R. Chimatapu, H. Hagras, M. Kern, and G. Owusu, “Hybrid deep learning type-2 fuzzy logic systems for explainable ai,” in 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2020, pp. 1-6.
[98] M. Yeganejou, S. Dick, and J. Miller, “Interpretable deep convolutional fuzzy classifier,” IEEE Transactions on Fuzzy Systems, vol. 28, no. 7, pp. 14071419, 2019.
[99] H. S. Pannu, D. Singh, and A. K. Malhi, “Multi-objective particle swarm optimization-based adaptive neuro-fuzzy inference system for benzene monitoring,” Neural computing and applications, vol. 31, no. 7, pp. 2195-2205, 2019.
[100] A. El Shinawi, R. A. Ibrahim, L. Abualigah, M. Zelenakova, and M. Abd Elaziz, “Enhanced adaptive neuro-fuzzy inference system using reptile search algorithm for relating swelling potentiality using index geotechnical properties: A case study at el sherouk city, egypt,” Mathematics, vol. 9, no. 24, p. 3295, 2021.
[101] S. H. Sumit and S. Akhter, “C-means clustering and deep-neuro-fuzzy classification for road weight measurement in traffic management system,” Soft Computing, vol. 23, no. 12, pp. 4329-4340, 2019.

[102] W. F. MAHMUDY, A. P. WIBAWA, N. R. SARI, and P. HAVILUDDIN, “Genetic algorithmised neuro fuzzy system for forecasting the online journal visitors,” International Journal of Computing, 2021.
[103] C.-F. Juang and C.-D. Hsieh, “A fuzzy system constructed by rule generation and iterative linear svr for antecedent and consequent parameter optimization,” IEEE Transactions on Fuzzy Systems, vol. 20, no. 2, pp. 372-384, 2011.
[104] P. Angelov and R. Buswell, “Identification of evolving fuzzy rule-based models,” IEEE Transactions on Fuzzy Systems, vol. 10, no. 5, pp. 667-677, 2002.
[105] J.-C. de Barros and A. L. Dexter, “On-line identification of computationally undemanding evolving fuzzy models,” Fuzzy sets and systems, vol. 158, no. 18, pp. 1997-2012, 2007.
[106] P. Angelov, “Evolving takagi-sugeno fuzzy systems from streaming data (etsp),” Evolving intelligent systems: methodology and applications, p. 21, 2010.
[107] A. Shaker, R. Senge, and E. Hüllermeier, “Evolving fuzzy pattern trees for binary classification on data streams,” Information Sciences, vol. 220, pp. 34$45,2013$.
[108] A. Lemos, W. Caminhas, and F. Gomide, “Fuzzy evolving linear regression trees,” Evolving Systems, vol. 2, no. 1, pp. 1-14, 2011.
[109] E. Lughofer and P. Angelov, “Handling drifts and shifts in on-line data streams with evolving fuzzy systems,” Applied Soft Computing, vol. 11, no. 2, pp. 2057-2068, 2011.
[110] Y.-Y. Lin, J.-Y. Chang, and C.-T. Lin, “Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network,” IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 2, pp. 310-321, 2012.
[111] M. Pratama, S. G. Anavatti, P. P. Angelov, and E. Lughofer, “Panfis: A novel incremental learning machine,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 1, pp. 55-68, 2013.
[112] A. Lemos, W. Caminhas, and F. Gomide, “Multivariable gaussian evolving fuzzy modeling system,” IEEE Transactions on Fuzzy Systems, vol. 19, no. 1, pp. 91-104, 2010.
[113] R. D. Baruah and P. Angelov, “Evolving fuzzy systems for data streams: A survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 1, no. 6, pp. 461-476, 2011.
[114] C.-M. Lin, T.-L. Le, and T.-T. Huynh, “Self-evolving function-link interval type-2 fuzzy neural network for nonlinear system identification and control,” Neurocomputing, vol. 275, pp. 2239-2250, 2018.

[115] H. Huang, H.-J. Rong, Z.-X. Yang, and C.-M. Vong, “Jointly evolving and compressing fuzzy system for feature reduction and classification,” Information Sciences, vol. 579, pp. 218-230, 2021.
[116] H. H. Y. Sa’ad, N. A. M. Isa, and M. M. Ahmed, “A structural evolving approach for fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 28, no. 2, pp. 273-287, 2019.
[117] S. W. Tung, C. Quek, and C. Guan, “Et2fis: An evolving type-2 neural fuzzy inference system,” Information Sciences, vol. 220, pp. 124-148, 2013.
[118] E. Lughofer, V. Macián, C. Guardiola, and E. P. Klement, “Data-driven design of takagi-sugeno fuzzy systems for predicting nox emissions,” in Information Processing and Management of Uncertainty in Knowledge-Based Systems. Applications: 13th International Conference, IPMU 2010, Dortmund, Germany, June 28-July 2, 2010. Proceedings, Part II 13, Springer, 2010, pp. 1-10.
[119] N. K. Kasabov, Evolving connectionist systems: the knowledge engineering approach. Springer Science & Business Media, 2007.
[120] K. S. T. R. Alves and E. P. de Aguiar, “A novel rule-based evolving fuzzy system applied to the thermal modeling of power transformers,” Applied Soft Computing, vol. 112, p. 107 764, 2021.
[121] C. P. Chen and C.-Y. Zhang, “Data-intensive applications, challenges, techniques and technologies: A survey on big data,” Information sciences, vol. 275, pp. 314-347, 2014.
[122] M. Elkano, M. Galar, J. Sanz, and H. Bustince, “Chi-bd: A fuzzy rule-based classification system for big data classification problems,” Fuzzy Sets and Systems, vol. 348, pp. 75-101, 2018.
[123] J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,” Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
[124] S. del Rıéo, V. López, J. M. Bemétez, and F. Herrera, “A mapreduce approach to address big data classification problems based on the fusion of linguistic fuzzy rules,” International Journal of Computational Intelligence Systems, vol. 8, no. 3, pp. 422-437, 2015.
[125] V. López, S. Del Rıéo, J. M. Bemétez, and F. Herrera, “Cost-sensitive linguistic fuzzy rule based classification systems under the mapreduce framework for imbalanced big data,” Fuzzy Sets and Systems, vol. 258, pp. 5-38, 2015.
[126] A. Segatori, A. Bechini, P. Ducange, and F. Marcelloni, “A distributed fuzzy associative classifier for big data,” IEEE transactions on cybernetics, vol. 48, no. 9, pp. 2656-2669, 2017.
[127] A. Fernández, S. del Rıéo, A. Bawakid, and F. Herrera, “Fuzzy rule based classification systems for big data with mapreduce: Granularity analysis,” Advances in Data Analysis and Classification, vol. 11, no. 4, pp. 711-730, 2017.

[128] A. Fernandez, C. J. Carmona, M. J. del Jesus, and F. Herrera, “A view on fuzzy systems for big data: Progress and opportunities,” International Journal of Computational Intelligence Systems, vol. 9, no. sup1, pp. 69-80, 2016.
[129] H. Wang, Z. Xu, and W. Pedrycz, “An overview on the roles of fuzzy set techniques in big data processing: Trends, challenges and opportunities,” Knowledge-Based Systems, vol. 118, pp. 15-30, 2017.
[130] J. de Jesús Rubio, “Usnfis: Uniform stable neuro fuzzy inference system,” Neurocomputing, vol. 262, pp. 57-66, 2017.
[131] L. Zhang, Y. Shi, Y.-C. Chang, and C.-T. Lin, “Hierarchical fuzzy neural networks with privacy preservation for heterogeneous big data,” IEEE Transactions on Fuzzy Systems, vol. 29, no. 1, pp. 46-58, 2020.
[132] L. Íniguez, M. Galar, and A. Fernández, “Improving fuzzy rule based classification systems in big data via support-based filtering,” in 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2018, pp. 18 .
[133] F. Aghaeipoor, M. M. Javidi, and A. Fernandez, “Ifc-bd: An interpretable fuzzy classifier for boosting explainable artificial intelligence in big data,” IEEE Transactions on Fuzzy Systems, 2021.
[134] M. Hu, Y. Zhong, S. Xie, H. Lv, and Z. Lv, “Fuzzy system based medical image processing for brain disease prediction,” Frontiers in Neuroscience, vol. 15, p. $714318,2021$.
[135] S. M. H. Bamakan, N. Faregh, and A. ZareRavasan, “Di-anfis: An integrated blockchain-iot-big data-enabled framework for evaluating service supply chain performance,” Journal of Computational Design and Engineering, vol. 8, no. 2, pp. 676-690, 2021.
[136] L. Magdalena, “Do hierarchical fuzzy systems really improve interpretability?” In International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Springer, 2018, pp. 1626 .
[137] R. P. Paiva and A. Dourado, “Interpretability and learning in neuro-fuzzy systems,” Fuzzy sets and systems, vol. 147, no. 1, pp. 17-38, 2004.
[138] J. Knapp and A. Knapp, “Refine and merge: Generating small rule bases from training data,” in Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569), IEEE, vol. 1, 2001, pp. 197-202.
[139] E. Hüllermeier, “Does machine learning need fuzzy logic?” Fuzzy Sets and Systems, vol. 281, pp. 292-299, 2015.
[140] A. K. Varshney and V. Torra, “Designing distributed chi-fuzzy rule based classification system,” in 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2022, pp. 1-7.

[141] J. M. Mendel and P. P. Bonissone, “Critical thinking about explainable ai (xai) for rule-based fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 29, no. 12, pp. 3579-3593, 2021.
[142] P. Pulkkinen and H. Koivisto, “A dynamically constrained multiobjective genetic fuzzy system for regression problems,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 1, pp. 161-177, 2009.
[143] S. Guillaume and B. Charnomordic, “Learning interpretable fuzzy inference systems with fispro,” Information Sciences, vol. 181, no. 20, pp. 4409-4427, 2011.
[144] K. Cpałka, K. Łapa, A. Przybył, and M. Zalasiński, “A new method for designing neuro-fuzzy systems for nonlinear modelling with interpretability aspects,” Neurocomputing, vol. 135, pp. 203-217, 2014.
[145] M. J. Gacto, R. Alcalá, and F. Herrera, “Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures,” Information Sciences, vol. 181, no. 20, pp. 4340-4360, 2011.
[146] T. Razak, J. M. Garibaldi, C. Wagner, A. Pourabdollah, and D. Soria, “Interpretability indices for hierarchical fuzzy systems,” in 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2017, pp. 16 .
[147] E. Lughofer, “On-line assurance of interpretability criteria in evolving fuzzy systems-achievements, new concepts and open issues,” Information sciences, vol. 251, pp. 22-46, 2013.
[148] M. M. Ahmed and N. A. M. Isa, “Knowledge base to fuzzy information granule: A review from the interpretability-accuracy perspective,” Applied Soft Computing, vol. 54, pp. 121-140, 2017.
[149] A. Fernández, M. J. del Jesus, and F. Herrera, “On the influence of an adaptive inference system in fuzzy rule based classification systems for imbalanced data-sets,” Expert Systems with Applications, vol. 36, no. 6, pp. 98059812, 2009.
[150] , “Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets,” International Journal of Approximate Reasoning, vol. 50, no. 3, pp. 561-577, 2009.
[151] H. Ishibuchi and T. Yamamoto, “Rule weight specification in fuzzy rulebased classification systems,” IEEE transactions on fuzzy systems, vol. 13, no. 4, pp. 428-435, 2005.
[152] A. Fernandez, E. Almansa, and F. Herrera, “Chi-spark-rs: An spark-built evolutionary fuzzy rule selection algorithm in imbalanced classification for big data problems,” in 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2017, pp. 1-6.
[153] J. Kerr-Wilson and W. Pedrycz, “Generating a hierarchical fuzzy rule-based model,” Fuzzy Sets and Systems, vol. 381, pp. 124-139, 2020.

[154] S. D. Nguyen, Q. H. Nguyen, and S.-B. Choi, “A hybrid clustering based fuzzy structure for vibration control-part 2: An application to semi-active vehicle seat-suspension system,” Mechanical systems and signal processing, vol. 56, pp. 288-301, 2015.
[155] B. H. Pham, H. T. Ha, and L. T. Ngo, “Learning rule for tsk fuzzy logic systems using interval type-2 fuzzy subtractive clustering,” in Asia-Pacific Conference on Simulated Evolution and Learning, Springer, 2012, pp. 430439 .
[156] V. Torra, “Hesitant fuzzy sets,” International journal of intelligent systems, vol. 25, no. 6, pp. 529-539, 2010.
[157] D. Dubois and H. Prade, “Rough fuzzy sets and fuzzy rough sets,” International Journal of General System, vol. 17, no. 2-3, pp. 191-209, 1990.
[158] R. R. Yager, “On ordered weighted averaging aggregation operators in multicriteria decisionmaking,” IEEE Transactions on systems, Man, and Cybernetics, vol. 18, no. 1, pp. 183-190, 1988.
[159] G. Choquet, “Theory of capacities,” in Annales de l’institut Fourier, vol. 5, 1954, pp. 131-295.
[160] M. Sugeno, “Theory of fuzzy integrals and its applications,” Doct. Thesis, Tokyo Institute of technology, 1974.
[161] K. Shihabudheen and G. N. Pillai, “Recent advances in neuro-fuzzy system: A survey,” Knowledge-Based Systems, vol. 152, pp. 136-162, 2018.

Statements & Declarations

Acknowledgement: Authors are thankful to the editors and the reviewers for their valuable comments which have helped a lot in improving the work. This work was partially supported by the Wallenberg Al, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.

Author Contribution: Ayush K. Varshney: Conceptualization, Methodology, Formal analysis, Validation, Visualization, Writing - original draft. Vicenç Torra: Conceptualization, Formal analysis, Validation, Writing - review & editing, Resources, Supervision.

Data availability: Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Declaration of competing interest: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.