Papers by Rohan kumar Yadav
Cyclostationary Random Number Sequences for the Tsetlin Machine
Lecture Notes in Computer Science, 2022
An Interpretable Word Sense Classifier for Human Explainable Chatbot
Lecture Notes in Computer Science, 2022
Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
The state-of-the-art natural language processing models have raised the bar for excellent perform... more The state-of-the-art natural language processing models have raised the bar for excellent performance on a variety of tasks in recent years. However, concerns are rising over their primitive sensitivity to distribution biases that reside in the training and testing data. This issue hugely impacts the performance of the models when exposed to out-of-distribution and counterfactual data. The root cause seems to be that many machine learning models are prone to learn the shortcuts, modelling simple correlations rather than more fundamental and general relationships. As a result, such text classifiers tend to perform poorly when a human makes minor modifications to the data, which raises questions regarding their robustness. In this paper, we employ a rule-based architecture called Tsetlin Machine (TM) that learns both simple and complex correlations by ANDing features and their negations. As such, it generates explainable AND-rules using negated and non-negated reasoning. Here, we expl...
Computers
Large-scale pre-trained language representation and its promising performance in various downstre... more Large-scale pre-trained language representation and its promising performance in various downstream applications have become an area of interest in the field of natural language processing (NLP). There has been huge interest in further increasing the model’s size in order to outperform the best previously obtained performances. However, at some point, increasing the model’s parameters may lead to reaching its saturation point due to the limited capacity of GPU/TPU. In addition to this, such models are mostly available in English or a shared multilingual structure. Hence, in this paper, we propose a lite BERT trained on a large corpus solely in the Romanian language, which we called “A Lite Romanian BERT (ALR-BERT)”. Based on comprehensive empirical results, ALR-BERT produces models that scale far better than the original Romanian BERT. Alongside presenting the performance on downstream tasks, we detail the analysis of the training process and its parameters. We also intend to distri...
LSTM-based unidirectional DRNN for indoor landmark classification using Geomagnetic field
The unstable nature of RF signals and the need for external infrastructure inside buildings have ... more The unstable nature of RF signals and the need for external infrastructure inside buildings have limited the use of positioning techniques, such as Wi-Fi and Bluetooth fingerprinting. Compared to these techniques, the geomagnetic field exhibits a stable signal strength in the time domain.
ArXiv, 2021
Using logical clauses to represent patterns, Tsetlin machines (TMs) have recently obtained compet... more Using logical clauses to represent patterns, Tsetlin machines (TMs) have recently obtained competitive performance in terms of accuracy, memory footprint, energy, and learning speed on several benchmarks. A team of Tsetlin automata (TAs) composes each clause, thus driving the entire learning process. These are rewarded/penalized according to three local rules that optimize global behaviour. Each clause votes for or against a particular class, with classification resolved using a majority vote. In the parallel and asynchronous architecture that we propose here, every clause runs in its own thread for massive parallelism. For each training example, we keep track of the class votes obtained from the clauses in local voting tallies. The local voting tallies allow us to detach the processing of each clause from the rest of the clauses, supporting decentralized learning. Thus, rather than processing training examples one-by-one as in the original TM, the clauses access the training exampl...
Algorithms
Explainability is one of the key factors in Natural Language Processing (NLP) specially for legal... more Explainability is one of the key factors in Natural Language Processing (NLP) specially for legal documents, medical diagnosis, and clinical text. Attention mechanism has been a popular choice for such explainability recently by estimating the relative importance of input units. Recent research has revealed, however, that such processes tend to misidentify irrelevant input units when explaining them. This is due to the fact that language representation layers are initialized by pre-trained word embedding that is not context-dependent. Such a lack of context-dependent knowledge in the initial layer makes it difficult for the model to concentrate on the important aspects of input. Usually, this does not impact the performance of the model, but the explainability differs from human understanding. Hence, in this paper, we propose an ensemble method to use logic-based information from the Tsetlin Machine to embed it into the initial representation layer in the neural network to enhance t...
Distributed Word Representation in Tsetlin Machine
ArXiv, 2021
Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional log... more Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional logic. The algorithm has demonstrated competitive performance in many Natural Language Processing (NLP) tasks, including sentiment analysis, text classification, and Word Sense Disambiguation (WSD). To obtain human-level interpretability, legacy TM employs Boolean input features such as bag-of-words (BOW). However, the BOW representation makes it difficult to use any pre-trained information, for instance, word2vec and GloVe word representations. This restriction has constrained the performance of TM compared to deep neural networks (DNNs) in NLP. To reduce the performance gap, in this paper, we propose a novel way of using pretrained word representations for TM. The approach significantly enhances the TM performance and maintains interpretability at the same time. We achieve this by extracting semantically related words from pre-trained word representations as input features to the TM. Our ...
ArXiv, 2021
In this article, we introduce a novel variant of the Tsetlin machine (TM) that randomly drops cla... more In this article, we introduce a novel variant of the Tsetlin machine (TM) that randomly drops clauses, the key learning elements of a TM. In effect, TM with drop clause ignores a random selection of the clauses in each epoch, selected according to a predefined probability. In this way, additional stochasticity is introduced in the learning phase of TM. Along with producing more distinct and well-structured patterns that improve the performance, we also show that dropping clauses increases learning robustness. To explore the effects clause dropping has on accuracy, training time, and interpretability, we conduct extensive experiments on various benchmark datasets in natural language processing (NLP) (IMDb and SST2) as well as computer vision (MNIST and CIFAR10). In brief, we observe from +2% to +4% increase in accuracy and 2× to 4× faster learning. We further employ the Convolutional TM to document interpretable results on the CIFAR10 dataset. To the best of our knowledge, this is th...
This paper proposes a human-interpretable learning approach for aspect-based sentiment analysis (... more This paper proposes a human-interpretable learning approach for aspect-based sentiment analysis (ABSA), employing the recently introduced Tsetlin Machines (TMs). We attain interpretability by converting the intricate position-dependent textual semantics into binary form, mapping all the features into bag-of-words (BOWs). The binary-form BOWs are encoded so that the information on the aspect and context words are retained for sentiment classification. We further adopt the BOWs as input to the TM, enabling learning of aspect-based sentiment patterns in propositional logic. To evaluate interpretability and accuracy, we conducted experiments on two widely used ABSA datasets from SemEval 2014: Restaurant 14 and Laptop 14. The experiments show how each relevant feature takes part in conjunctive clauses that contain the context information for the corresponding aspect word, demonstrating human-level interpretability. At the same time, the obtained accuracy is on par with existing neural ne...
Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional log... more Tsetlin Machine (TM) is an interpretable pattern recognition algorithm based on propositional logic, which has demonstrated competitive performance in many Natural Language Processing (NLP) tasks, including sentiment analysis, text classification, and Word Sense Disambiguation. To obtain human-level interpretability, legacy TM employs Boolean input features such as bag-of-words (BOW). However, the BOW representation makes it difficult to use any pre-trained information, for instance, word2vec and GloVe word representations. This restriction has constrained the performance of TM compared to deep neural networks (DNNs) in NLP. To reduce the performance gap, in this paper, we propose a novel way of using pre-trained word representations for TM. The approach significantly enhances the performance and interpretability of TM. We achieve this by extracting semantically related words from pre-trained word representations as input features to the TM. Our experiments show that the accuracy of...
Indoor space classification is an important part of localization that helps in precise location e... more Indoor space classification is an important part of localization that helps in precise location extraction, which has been extensively utilized in industrial and domestic domain. There are various approaches that employ Bluetooth Low En-ergy (BLE), Wi-Fi, magnetic field, object detection, and Ultra Wide Band (UWB) for indoor space classification purposes. Many of the existing approaches need extensive pre-installed infrastructure, making the cost higher to obtain reasonable accuracy. Therefore, improvements are still required to increase the accuracy with minimum requirements of infrastructure. In this paper, we propose an approach to classify the indoor space using geomagnetic field (GMF) and radio signal strength (RSS) as the identity. The indoor space is an open big test bed divided into different indiscernible subspace. We collect GMF and RSS at each subspace and classify it using cascaded Long Short Term Memory (LSTM). The experimental results show that the accuracy is signific...
Positionless aspect based sentiment analysis using attention mechanism
Knowledge-Based Systems
Proceedings of the 13th International Conference on Agents and Artificial Intelligence
Word Sense Disambiguation (WSD) is a longstanding unresolved task in Natural Language Processing.... more Word Sense Disambiguation (WSD) is a longstanding unresolved task in Natural Language Processing. The challenge lies in the fact that words with the same spelling can have completely different senses, sometimes depending on subtle characteristics of the context. A weakness of the state-of-the-art supervised models, however, is that it can be difficult to interpret them, making it harder to check if they capture senses accurately or not. In this paper, we introduce a novel Tsetlin Machine (TM) based supervised model that distinguishes word senses by means of conjunctive clauses. The clauses are formulated based on contextual cues, represented in propositional logic. Our experiments on CoarseWSD-balanced dataset indicate that the learned word senses can be relatively effortlessly interpreted by analyzing the converged model of the TM. Additionally, the classification accuracy is higher than that of FastText-Base and similar to that of FastText-CommonCrawl.
IEEE Access
The unstable nature of radio frequency (RF) signals and the need for external infrastructure insi... more The unstable nature of radio frequency (RF) signals and the need for external infrastructure inside buildings have limited the use of positioning techniques, such as Wi-Fi and Bluetooth fingerprinting. Compared to these techniques, the geomagnetic field exhibits a stable signal strength in the time domain. However, existing magnetic positioning methods cannot perform well in a wide space because the magnetic signal is not always discernible. In this paper, we introduce deep recurrent neural networks (DRNNs) to build a model that is capable of capturing long-range dependencies in variable-length input sequences. The use of DRNNs is brought from the idea that the spatial/temporal sequence of magnetic field values around a given area will create a unique pattern over time, despite multiple locations having the same magnetic field value. Therefore, we can divide the indoor space into landmarks with magnetic field values and find the position of the user in a particular area inside the building. We present long short-term memory DRNNs for spatial/temporal sequence learning of magnetic patterns and evaluate their positioning performance on our testbed datasets. Experimental results show that our proposed models outperform other traditional positioning approaches with machine learning methods, such as support vector machine and k-nearest neighbors. INDEX TERMS Deep recurrent neural network (DRNN), fingerprinting, geomagnetic field, long shortterm memory (LSTM).
IEEE Access
Indoor positioning systems have received increasing attention because of their wide range of indo... more Indoor positioning systems have received increasing attention because of their wide range of indoor applications. However, the positioning system generally suffers from a large error in localization and has low solidity. The main approaches widely used for indoor localization are based on the inertial measurement unit (IMU), Bluetooth, Wi-Fi, and ultra-wideband. The major problem with Bluetooth-based fingerprinting is the inconsistency of the radio signal strength, and the IMU-based localization has a drift error that increases with time. To compensate for these drawbacks, in the present study, a novel positioning system with IMU sensors and Bluetooth low energy (BLE) beacon for a smartphone are introduced. The proposed trusted K nearest Bayesian estimation (TKBE) integrates BLE beacon and pedestrian dead reckoning positionings. The BLE-based positioning, using both the K-nearest neighbor (KNN) and Bayesian estimation, increases the accuracy by 25% compared with the existing KNN-based positioning, and the proposed fuzzy logic-based Kalman filter increases the accuracy by an additional 15%. The overall performance of TKBE has an error of <1 m in our experimental environments. INDEX TERMS Bayesian estimation, Bluetooth low energy (BLE), fingerprints, fuzzy-logic system, indoor positioning, K-nearest neighbor (KNN), pedestrian dead reckoning (PDR).
Uploads
Papers by Rohan kumar Yadav