Paraphrases Generation

Paraphrases represent a human's intuitive ability to understand expressions presented in various different ways. Current paraphrase evaluations of language models primarily use binary approaches, offering limited interpretability of specific text changes. Atomic paraphrase types (APT) decompose paraphrases into different linguistic changes and offer a granular view of the flexibility in linguistic expression (e.g., a shift in syntax or vocabulary used). In this study, we assess the human preferences towards ChatGPT in generating English paraphrases with ten APTs and five prompting techniques. We introduce APTY (Atomic Paraphrase TYpes), a dataset of 800 sentence-level and word-level annotations by 15 annotators. The dataset also provides a human preference ranking of paraphrases with different types that can be used to fine-tune models with RLHF and DPO methods. Our results reveal that ChatGPT and a DPO-trained LLama 7B model can generate simple APTs, such as additions and deletions, but struggle with complex structures (e.g., subordination changes). This study contributes to understanding which aspects of paraphrasing language models have already succeeded at understanding and what remains elusive. In addition, we show how our curated datasets can be used to develop language models with specific linguistic capabilities.

This book provides a range of research that presents new ideas to resolve issues regarding miscellaneous applications related to NLP. It covers topics such as data augmentation, phoneme recognition in low-resource languages, error correction detection, identification and aggregation of duplicate security findings, and exploiting social media discourse to estimate typical affective responses to music.
As data is a critical component in training high-performing deep learning models, this book begins with presenting techniques of text augmentation and paraphrase generation. It then discusses, for the sake of a better readability of texts, the use of generative byte-level transformer models for restoring spaces, punctuation, and capitalization in different languages. This is an important step useful for many applications as post-processing of noisy data and ASR outputs. The book also proposes a multi-task learning approach for multilingual phoneme recognition in low-resource languages. Dialog systems are one of the major widespread applications, but they can sometimes be prone to errors and ambiguities; that is why a chapter in this book is devoted to error correction detection and error correction, leading to disambiguation.
The book also shows how NLP presents solutions to predict music emotion values from social media comments without analysing audio signal or processing lyrics information. Finally, it discusses how to identify and aggregate the duplicate findings in the industry.
We hope this book will help the readers to gain a deeper understanding of the covered topics and apply the information to their own research.

The increased access to powerful processors has made possible significant progress in natural language processing (NLP). We find more research in NLP targeting diverse spectrum of major industries that use voice recognition, text-to-speech (TTS) solutions, speech translation, natural language understanding (NLU), and many other applications and techniques related to these areas.
This book presents the latest research related to natural language processing and speech technology and sheds light on the main topics for readers interested in this field. For TTS and automatic speech recognition, it is demonstrated how to explore transfer learning in order to generate speech in other voices from TTS of a specific language (Italian), and to improve speech recognition for non-native English.
Language resources are the cornerstone for building high-quality systems; however, some languages, as Arabic, are considered under-resourced compared to English. Thus, a new Arabic linguistic pipeline for NLP is presented to enrich the Arabic language resources and to solve common NLP issues, like word segmentation, POS tagging, and lemmatization. Arabic named entity recognition, a challenging task, has been resolved within this book using transformer-based-CRF model.
In addition, the readers of this book will discover conceptions and solutions for other NLP issues such as language modeling, question answering, dialog systems, and sentence embeddings.

This paper performs a detailed analysis on the alignment of Portuguese contractions, based on a previously aligned bilingual corpus. The alignment task was performed manually in a subset of the English-Portuguese CLUE4Translation Alignment Collection. The initial parallel corpus was pre-processed and, a decision was made as to whether the contraction should be maintained or decomposed in the alignment. Decomposition was required in the cases in which the two words that have been concatenated, i.e., the preposition and the determiner or pronoun, go in two separate translation alignment pairs (e.g., [no seio de] [a Uniao Europeia] | [within] [the European Union]). Most contractions required decomposition in contexts where they are positioned at the end of a multiword unit. On the other hand, contractions tend to be maintained when they occur in the beginning or in the middle of the multiword unit, i.e., in the frozen part of the multiword (e.g., [no que diz respeito a] | [with regard ...

This paper presents a methodology to extract a paraphrase database for the European and Brazilian varieties of Portuguese, and discusses a set of paraphrastic categories of multiwords and phrasal units, such as the compounds toda a gente vs todo o mundo "everybody" or the gerundive constructions [estar a + V-Inf] vs [ficar + V-Ger] (e.g., estive a observar vs fiquei observando "I was observing"), which are extremely relevant to high quality paraphrasing. The variants were manually aligned in the e-PACT corpus, using the CLUE-Aligner tool. The methodology, inspired in the Logos Model, focuses on a semantico-syntactic analysis of each paraphrastic unit and constitutes a subset of the Gold-CLUE-Paraphrases. 1 The construction of a larger dataset of paraphrastic contrasts among the distinct varieties of the Portuguese language is indispensable for variety adaptation, i.e., for dealing with the cultural, linguistic and stylistic differences between them, making it possible to convert texts (semi-)automatically from one variety into another, a key function in paraphrasing systems. This topic represents an interesting new line of research with valuable applications in language learning, language generation, question-answering, summarization, and machine translation, among others. The paraphrastic units are the first resource of its kind for Portuguese to become available to the scientific community for research purposes.

This paper presents a methodology to extract a paraphrase database for the European and Brazilian varieties of Portuguese, and discusses a set of paraphrastic categories of multiwords and phrasal units, such as the compounds “toda a gente” versus “todo o mundo” ‘everybody’ or the gerundive constructions [estar a + V-Inf] versus [ficar + V-Ger] (e.g., “estive a observar” | “fiquei observando” ‘I was observing’), which are extremely relevant to high quality paraphrasing. The variants were manually aligned in the e-PACT corpus, using the CLUE-Aligner tool. The methodology, inspired in the Logos Model, focuses on a semantico-syntactic analysis of each paraphrastic unit and constitutes a subset of the Gold-CLUE-Paraphrases. The construction of a larger dataset of paraphrastic contrasts among the distinct varieties of the Portuguese language is indispensable for variety adaptation, i.e., for dealing with the cultural, linguistic and stylistic differences between them, making it possible t...

This book presents a novel scientific approach to improve machine translation by paraphrasing support verb constructions with semantically equivalent verbs (e.g. make a presentation of/present). The author demonstrates that this strategy produces a positive impact in machine translation. The study is reproducible and extendable to distinct linguistic phenomena and successfully applied to different- purpose natural language processing applications. The author exemplifies how paraphrases can be efficiently employed by authoring aids to help simplify and clarify texts, presenting obvious benefits to linguistic quality assurance in text processing. While addressing and providing a solution for a specific linguistic problem, this book presents a comprehensive theoretical background and exposure of conceptual problems that will interest natural language processing professionals, linguists, translators, and students. Written in a simple language, this book will be easily understood by non-specialists in the field who have an interest in natural language.