Transfer Learning in Natural Language Processing (NLP)

IEEE Transactions on Cognitive Communications and Networking, 2020

Machine learning will play a major role in handling 1 the complexity of future mobile wireless networks by improving 2 network management and orchestration capabilities. Due to the 3 large number of parameters that can be monitored and config-4 ured in the network, collecting and processing high volumes of 5 data is often unfeasible or too expensive at network runtime, 6 which calls for taking resource management and service orches-7 tration decisions with only a partial view of the network status. 8 Motivated by this fact, this paper proposes a transfer learning 9 framework for reconstructing the radio map corresponding to a 10 target antenna tilt configuration by transferring the knowledge 11 acquired from another tilt configuration of the same antenna, 12 when no or very limited measurements are available from the 13 target. The performance of the framework is validated against 14 standard machine learning techniques on a data set collected 15 from a 4G commercial base stations. In most of the tested scenar-16 ios, the proposed framework achieves notable prediction accuracy 17 with respect to classical machine learning approaches, with a 18 mean absolute percentage error below 8%. 19 Index Terms-Radio map prediction, antenna tilt, transfer 20 learning. 21 I. INTRODUCTION 22 F IFTH generation wireless networks (5G) are expected to 23 improve the performance of cellular systems, achieving 24 higher data rates, reduced latency, higher reliability and sup-25 port for greater numbers of users. To achieve this, 5G resorts 26 to dense and heterogeneous deployments, coupled with higher 27 flexibility in the network access and core domains, which can 28 be dynamically managed in either a centralized or distributed 29 manner. To cope with such a complex scenario, it is foreseen 30 that machine learning tools will play a major role in enabling 31

Estimating accurate radio maps is important for various tasks in wireless communications, such as channel modeling, resource allocation, and network planning, to name a few. Due to the changes in the propagation characteristics of the wireless environments, a radio map model learned under a particular wireless environment cannot be directly used in a new wireless environment. Moreover, learning a new model for every environment requires, in general, a large amount of data and is computationally demanding. In this work, we design an effective novel data-driven transfer learning method that transfers and fine-tunes a deep neural network (DNN)-based radio map model learned from an original wireless environment to other wireless environments with a certain level of similarity, allowing the radio map to be estimated with less amount of training data. As opposed to other widely used similarity measures that do not take into account the wireless propagation characteristics, we design a dat...

Journal of Big Data

Infinite numbers of real-world applications use Machine Learning (ML) techniques to develop potentially the best data available for the users. Transfer learning (TL), one of the categories under ML, has received much attention from the research communities in the past few years. Traditional ML algorithms perform under the assumption that a model uses limited data distribution to train and test samples. These conventional methods predict target tasks undemanding and are applied to small data distribution. However, this issue conceivably is resolved using TL. TL is acknowledged for its connectivity among the additional testing and training samples resulting in faster output with efficient results. This paper contributes to the domain and scope of TL, citing situational use based on their periods and a few of its applications. The paper provides an in-depth focus on the techniques; Inductive TL, Transductive TL, Unsupervised TL, which consists of sample selection, and domain adaptation...

2019 International Conference on Computational Science and Computational Intelligence (CSCI), 2019

The Radio Access Network (RAN) is envisaged to undergo a significant transformation in the context of 5G and beyond mobile communications systems. One of the driving forces behind this transformation is the applicability of Machine Learning (ML) techniques. Taking as a reference the high-level architecture for a next generation RAN proposed by the Open RAN Alliance, this paper identifies the applicability domains where ML techniques can play a relevant role. For each domain, namely radio physical layer processing, Medium Access Control (MAC) scheduling, near-real time Radio Resource Management (RRM), RAN data analytics and RAN operational automation, the paper discusses the specific functionalities that can benefit from the application of ML and analyses the key considerations and challenges that need to be addressed when developing ML-based solutions, given the particular characteristics of the mobile radio environment.

2021

With outstanding features, Machine Learning (ML) has become the backbone of numerous applications in wireless networks. However, the conventional ML approaches face many challenges in practical implementation, such as the lack of labeled data, the constantly changing wireless environments, the long training process, and the limited capacity of wireless devices. These challenges, if not addressed, can impede the effectiveness and applicability of ML in wireless networks. To address these problems, Transfer Learning (TL) has recently emerged to be a promising solution. The core idea of TL is to leverage and synthesize distilled knowledge from similar tasks as well as from valuable experiences accumulated from the past to facilitate the learning of new problems. Doing so, TL techniques can reduce the dependence on labeled data, improve the learning speed, and enhance the ML methods' robustness to different wireless environments. This article aims to provide a comprehensive survey on applications of TL in wireless networks. Particularly, we first provide an overview of TL including formal definitions, classification, and various types of TL techniques. We then discuss diverse TL approaches proposed to address emerging issues in wireless networks. The issues include spectrum management, localization, signal recognition, security, human activity recognition and caching, which are all important to nextgeneration networks such as 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions of TL in future wireless networks.

2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021

Although the number of labeled datasets in Earth Observation (EO) is increasing, there is still a major gap between the Deep Learning (DL) classifiers designed in this field versus the models in Computer Vision. This gap is produced mainly by the number of datasets available, but also by the diversity of data. In EO, there are different sensors acquiring images, from multispectral (MS) or hyperspectral data, to SAR imagery. In this paper, we want to demonstrate how to reduce the divergence created by the diversity of data. We trained several DL architectures on Bag-of-Words from large-scale MS and SAR datasets, and then we used transfer learning on smaller ones and evaluated the results. With this method, we demonstrate that a DL architecture can be trained with any type of large-scale data, transformed into Bag-of-Words, and the trained model can be used further on other types of data, without regard on the number of channels.

2021 17th International Conference on Network and Service Management (CNSM), 2021

Recently, transfer learning (TL) has emerged as a powerful machine learning method in distributed environments. Transferring the knowledge between distributed agents helps reduce both learning time and computing costs. However, in a communication system, the advantage of TL comes with communication costs. To make an optimal decision of transfer between two agents, we try to answer three key questions: i) which information should be transferred from a source to a target?, ii) how this transferred information will be adapted to the target? and iii) when should TL be triggered to optimize the costs?. To this end, we introduce a new concept of similarity based on the Best Approximation Theory and a general transfer rule. Then, we propose a model to evaluate the feasibility and optimality of TL. We verify our proposed model in the context of the wireless channel selection problem using contextual multiarmed bandits. Experimental results show optimal TL decisions can be made, and Extra Action is an efficient technique for TL in channel selection.

2020

Pre-trained text encoders have rapidly advanced the state-of-the-art on many Natural Language Processing tasks. This paper presents the use of transfer learning methods applied to the automatic detection of codes in radiological reports in Spanish. Assigning codes to a clinical document is a popular task in NLP and in the biomedical domain. These codes can be of two types: standard classifications (e.g. ICD-10) or specific to each clinic or hospital. In this study we show a system using specific radiology clinic codes. The dataset is composed of 208,167 radiology reports labeled with 89 different codes. The corpus has been evaluated with three methods using the BERT model applied to Spanish: Multilingual BERT, BETO and XLM. The results are interesting obtaining 70% of F1-score with a pre-trained multilingual model.

Journal of Healthcare Engineering, 2022

Text Classification problem has been thoroughly studied in information retrieval problems and data mining tasks. It is beneficial in multiple tasks including medical diagnose health and care department, targeted marketing, entertainment industry, and group filtering processes. A recent innovation in both data mining and natural language processing gained the attention of researchers from all over the world to develop automated systems for text classification. NLP allows categorizing documents containing different texts. A huge amount of data is generated on social media sites through social media users. Three datasets have been used for experimental purposes including the COVID-19 fake news dataset, COVID-19 English tweet dataset, and extremist-non-extremist dataset which contain news blogs, posts, and tweets related to coronavirus and hate speech. Transfer learning approaches do not experiment on COVID-19 fake news and extremist-non-extremist datasets. Therefore, the proposed work ...

Algorithms for Intelligent Systems