Social Media Mining: Sentiment Analysis on Twitter Data

Scientometrics

The exponential growth of social media has brought with it an increasing propagation of hate speech and hate based propaganda. Hate speech is commonly defined as any communication that disparages a person or a group on the basis of some characteristics such as race, colour, ethnicity, gender, sexual orientation, nationality, religion. Online hate diffusion has now developed into a serious problem and this has led to a number of international initiatives being proposed, aimed at qualifying the problem and developing effective counter-measures. The aim of this paper is to analyse the knowledge structure of hate speech literature and the evolution of related topics. We apply co-word analysis methods to identify different topics treated in the field. The analysed database was downloaded from Scopus, focusing on a number of publications during the last thirty years. Topic and network analyses of literature showed that the main research topics can be divided into three areas: “general deb...

Future Internet

Social media services make it possible for an increasing number of people to express their opinion publicly. In this context, large amounts of hateful comments are published daily. The PHARM project aims at monitoring and modeling hate speech against refugees and migrants in Greece, Italy, and Spain. In this direction, a web interface for the creation and the query of a multi-source database containing hate speech-related content is implemented and evaluated. The selected sources include Twitter, YouTube, and Facebook comments and posts, as well as comments and articles from a selected list of websites. The interface allows users to search in the existing database, scrape social media using keywords, annotate records through a dedicated platform and contribute new content to the database. Furthermore, the functionality for hate speech detection and sentiment analysis of texts is provided, making use of novel methods and machine learning models. The interface can be accessed online w...

Complex & Intelligent Systems

Hate speech should be tackled and prosecuted based on how it is operationalized. However, the existing theoretical definitions of hate speech are not sufficiently fleshed out or easily operable. To overcome this inadequacy, and with the help of interdisciplinary experts, we propose an empirical definition of hate speech by providing a list of 10 hate speech indicators and the rationale behind them (the indicators refer to specific, observable, and measurable characteristics that offer a practical definition of hate speech). A preliminary exploratory examination of the structure of hate speech, with the focus on comments related to migrants (one of the most reported grounds of hate speech), revealed that two indicators in particular, denial of human rights and promoting violent behavior, occupy a central role in the network of indicators. Furthermore, we discuss the practical implications of the proposed hate speech indicators—especially (semi-)automatic detection using the latest na...

Proceedings of the 11th Forum for Information Retrieval Evaluation

With the growth of social media, the spread of hate speech is also increasing rapidly. Social media are widely used in many countries. Also Hate Speech is spreading in these countries. This brings a need for multilingual Hate Speech detection algorithms. Much research in this area is dedicated to English at the moment. The HASOC track intends to provide a platform to develop and optimize Hate Speech detection algorithms for Hindi, German and English. The dataset is collected from a Twitter archive and pre-classified by a machine learning system. HASOC has two sub-task for all three languages: task A is a binary classification problem (Hate and Not Offensive) while task B is a fine-grained classification problem for three classes (HATE) Hate speech, OFFENSIVE and PROFANITY. Overall, 252 runs were submitted by 40 teams. The performance of the best classification algorithms for task A are F1 measures of 0.51, 0.53 and 0.52 for English, Hindi, and German, respectively. For task B, the best classification algorithms achieved F1 measures of 0.26, 0.33 and 0.29 for English, Hindi, and German, respectively. This article presents the tasks and the data development as well as the results. The best performing algorithms were mainly variants of the transformer architecture BERT. However, also other systems were applied with good success.

Language Resources and Evaluation

Hate Speech in social media is a complex phenomenon, whose detection has recently gained significant traction in the Natural Language Processing community, as attested by several recent review works. Annotated corpora and benchmarks are key resources, considering the vast number of supervised approaches that have been proposed. Lexica play an important role as well for the development of hate speech detection systems. In this review, we systematically analyze the resources made available by the community at large, including their development methodology, topical focus, language coverage, and other factors. The results of our analysis highlight a heterogeneous, growing landscape, marked by several issues and venues for improvement.

Proceedings of the 32st ACM Conference on Hypertext and Social Media

Most hate speech detection research focuses on a single language, generally English, which limits their generalisability to other languages. In this paper we investigate the cross-lingual hate speech detection task, tackling the problem by adapting the hate speech resources from one language to another. We propose a cross-lingual capsule network learning model coupled with extra domain-specific lexical semantics for hate speech (CCNL-Ex). Our model achieves state-of-the-art performance on benchmark datasets from AMI@Ev-alita2018 and AMI@Ibereval2018 involving three languages: English, Spanish and Italian, outperforming state-of-the-art baselines on all six language pairs. CCS CONCEPTS • Information systems → Social networks; Data analytics; • Computing methodologies → Natural language processing; • Social and professional topics → Women.

Online Social Networks and Media

Online sexism has become an increasing concern in social media platforms as it has affected the healthy development of the Internet and can have negative effects in society. While research in the sexism detection domain is growing, most of this research focuses on English as the language and on Twitter as the platform. Our objective here is to broaden the scope of this research by considering the Chinese language on Sina Weibo. We propose the first Chinese sexism dataset -Sina Weibo Sexism Review (SWSR) dataset -, as well as a large Chinese lexicon SexHateLex made of abusive and gender-related terms. We introduce our data collection and annotation process, and provide an exploratory analysis of the dataset characteristics to validate its quality and to show how sexism is manifested in Chinese. The SWSR dataset provides labels at different levels of granularity including (i) sexism or non-sexism, (ii) sexism category and (iii) target type, which can be exploited, among others, for building computational methods to identify and investigate finer-grained gender-related abusive language. We conduct experiments for the three sexism classification tasks making use of state-of-the-art machine learning models. Our results show competitive performance, providing a benchmark for sexism detection in the Chinese language, as well as an error analysis highlighting open challenges need- * This is to indicate the corresponding author.

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022

While the rise in popularity of social media is seen as a hugely positive development, it is also accompanied by a proliferation of hate speech, which has recently become a major concern. On the one hand, hateful content creates an unsafe environment for certain members of society. On the other hand, manual moderation causes distress to content moderators, and the volume of harmful content is far beyond what human moderators can manually flag and react to. Thus, researchers in machine learning, social computing, and other areas have worked on developing tools to help automate the process. While initially studied as a text classification problem, over time, researchers realized that hate speech is multi-faceted and requires analysis of the role of linguistic expressions, context, and network structure, while using inspiration from psychology and user behavior, among others. With this in mind, we provide a holistic view of what the research community has explored so far, and what we believe are promising future research directions. CCS CONCEPTS • General and reference → Surveys and overviews; Reliability; • Human-centered computing → Empirical studies in collaborative and social computing.

Education and Information Technologies

Cyberbullying (CB) is classified as one of the severe misconducts on social media. Many CB detection systems have been developed for many natural languages to face this phenomenon. However, Arabic is one of the under-resourced languages suffering from the lack of quality datasets in many computational research areas. This paper discusses the design, construction, and evaluation of a multi-dialect, annotated Arabic Cyberbullying Corpus (ArCybC), a valuable resource for Arabic CB detection and motivation for future research directions in Arabic Natural Language Processing (NLP). The study describes the phases of ArCybC compilation. By way of illustration, it explores the corpus to discover strategies used in rendering Arabic CB tweets pulled from four Twitter groups, including gaming, sports, news, and celebrities. Based on thorough analysis, we discovered that these groups were the most susceptible to harassment and cyberbullying. The collected tweets were filtered based on a compiled harassment lexicon, which contains a list of multi-dialectical profane words in Arabic compiled from four categories: sexual, racial, physical appearance, and intelligence. To annotate ArCybC, we asked five annotators to classify 4,505 tweets into two classes manually: Offensive/non-Offensive and CB/non-CB. We conducted a rigorous comparison of different machine learning approaches applied on ArCybC to detect Arabic CB using two language models: bag-of-words (BoW) and word embedding. The experiments showed that Support Vector Machine (SVM) with word embedding achieved an accuracy rate of 86.3% and an F1-score rate of 85%. The main challenges encountered during the ArCybC construction were the scarcity of freely available Arabic CB texts and the deficiency of annotating the texts.

IOP Conference Series: Materials Science and Engineering, 2020

Information Processing & Management, 2021

This paper focuses on an important problem of detecting offensive analogy meme on online social media where the visual content and the texts/captions of the meme together make an analogy to convey the offensive information. Existing offensive meme detection solutions often ignore the implicit relation between the visual and textual contents of the meme and are insufficient to identify the offensive analogy memes. Two important challenges exist in accurately detecting the offensive analogy memes: i) it is not trivial to capture the analogy that is often implicitly conveyed by a meme; ii) it is also challenging to effectively align the complex analogy across different data modalities in a meme. To address the above challenges, we develop a deep learning based Analogy-aware Offensive Meme Detection (AOMD) framework to learn the implicit analogy from the multi-modal contents of the meme and effectively detect offensive analogy memes. We evaluate AOMD on two real-world datasets from online social media. Evaluation results show that AOMD achieves significant performance gains compared to state-of-the-art baselines by detecting offensive analogy memes more accurately.

Social, Cultural, and Behavioral Modeling, 2020

The ubiquity of social media has transformed online interactions among individuals. Despite positive effects, it has also allowed antisocial elements to unite in alternative social media environments (e.g. Gab.com) like never before. Detecting such hateful speech using automated techniques can allow social media platforms to moderate their content and prevent nefarious activities like hate speech propagation. In this work, we propose a weak supervision deep learning model that-(i) quantitatively uncover hateful users and (ii) present a novel qualitative analysis to uncover indirect hateful conversations. This model scores content on the interaction level, rather than the post or user level, and allows for characterization of users who most frequently participate in hateful conversations. We evaluate our model on 19.2M posts and show that our weak supervision model outperforms the baseline models in identifying indirect hateful interactions. We also analyze a multilayer network, constructed from two types of user interactions in Gab (quote and reply) and interaction scores from the weak supervision model as edge weights, to predict hateful users. We utilize the multilayer network embedding methods to generate features for the prediction task and we show that considering user context from multiple networks help achieving better predictions of hateful users in Gab. We receive upto 7% performance gain compared to single layer or homogeneous network embedding models.

Advances in Knowledge Discovery and Data Mining, 2021

Automated hate speech detection in social media is a challenging task that has recently gained significant traction in the data mining and Natural Language Processing community. However, most of the existing methods adopt a supervised approach that depended heavily on the annotated hate speech datasets, which are imbalanced and often lack training samples for hateful content. This paper addresses the research gaps by proposing a novel multitask learning-based model, AngryBERT, which jointly learns hate speech detection with sentiment classification and target identification as secondary relevant tasks. We conduct extensive experiments to augment three commonly-used hate speech detection datasets. Our experiment results show that AngryBERT outperforms state-of-the-art single-task-learning and multitask learning baselines. We conduct ablation studies and case studies to empirically examine the strengths and characteristics of our AngryBERT model and show that the secondary tasks are able to improve hate speech detection.

2021

Öz Sosyal medya ağlarının sayısının ve kullanımının artması beraberinde nefret söylemi içeriklerinin de daha çok paylaşılması problemini doğurmuştur. Gerek kamu otoriteleri gerekse sosyal medya ağlarının kendileri, artan nefret söylemiyle mücadele kapsamında çeşitli politikalar üretmektedir. Kullanıcılar tarafından üretilen verinin hacminin oldukça büyük olması nedeniyle nefret söylemi tespitinde otomatik sistemlere ihtiyaç duyulmaktadır. Özellikle son yıllarda başta İngilizce olmak üzere birçok dil üzerinde otomatik nefret söylemi çalışması yapılmış olmasına rağmen Türkçe üzerine kapsamlı bir çalışma henüz sunulmamıştır. Bu çalışma bu ihtiyaca karşılık vermek amacıyla yapılmıştır. Farklı hedef gruplara dair anahtar kelimelerin geçtiği 1000 adet Türkçe tweet toplanmış ve iki değerlendirici tarafından üç sınıflı (nefret söylemi, saldırgan ifade, hiçbiri) olarak ayrı ayrı etiketlenmiştir. Oluşturulan Türkçe nefret söylemi veri seti sonraki çalışmalarda kullanılabilmesi için kamuya açık olarak paylaşılmıştır. Bu veri seti üzerinde farklı özellik kümeleri ve farklı makine öğrenmesi algoritmaları kullanılarak çeşitli testler gerçekleştirilmiştir. Üç sınıflı veri seti üzerinde en yüksek performans %79,9 F-ölçüm değeri ile SMO (Sıralı Minimal Optimizasyon) algoritmasının kullanıldığı testte elde edilmiştir. Türkçe nefret söylemi tespitinde daha başarılı sonuçlar almak için veri seti boyutunun artırılması gerekirken, sunulan bu çalışmanın gelecekte yapılacak çalışmalara öncü niteliğinde olması beklenmektedir.

The eighth International Workshop of Chinese CHI

As toxic language becomes nearly pervasive online, there has been increasing interest in leveraging the advancements in natural language processing (NLP) to automatically detect and remove toxic comments. Despite fairness concerns and limited interpretability, there is currently little work for auditing these systems in particular for end users. We present our ongoing work, RECAST, an interactive tool for auditing toxicity detection models by visualizing explanations for predictions and providing alternative wordings for detected toxic speech. RECAST displays the attention of toxicity detection models on user input, and provides an intuitive system for rewording impactful language within a comment with less toxic alternative words close in embedding space. Finally we propose a larger user study of RECAST, with promising preliminary results, to validate it's effectiveness and useability with end users.

Social Network Analysis and Mining

Social Media has become a vital component of every individual's life in society opening a preferred spectrum of virtual communication which provides an individual with a freedom to express their views and thoughts. While virtual communication through social media platforms is highly desirable and has become an inevitable component, the dark side of social media is observed in form of detrimental/objectionable content. The reported detrimental contents are fake news, rumors, hate speech, aggressive, and cyberbullying which raise up as a major concern in the society. Such detrimental content is affecting person's mental health and also resulted in loss which cannot be always recovered. So, detecting and moderating such content is a prime need of time. All social media platforms including Facebook, Twitter, and YouTube have made huge investments and also framed policies to detect and moderate such detrimental content. It is of paramount importance in the first place to detect such content. After successful detection, it should be moderated. With an overflowing increase in detrimental content on social media platforms, the current manual method to identify such content will never be enough. Manual and semi-automated moderation methods have reported limited success. A fully automated detection and moderation is a need of time to come up with the alarming detrimental content on social media. Artificial Intelligence (AI) has reached across all sectors and provided solutions to almost all problems, social media content detection and moderation is not an exception. So, AI-based methods like Natural Language Processing (NLP) with Machine Learning (ML) algorithms and Deep Neural Networks is rigorously deployed for detection and moderation of detrimental content on social media platforms. While detection of such content has been receiving good attention in the research community, moderation has received less attention. This research study spans into three parts wherein the first part emphasizes on the methods to detect the detrimental components using NLP. The second section describes about methods to moderate such content. The third part summarizes all observations to provide identified research gaps, unreported problems and provide research directions.

Companion Proceedings of the Web Conference 2021, 2021

As user-generated contents thrive, so does the spread of toxic comment. Therefore, detecting toxic comment becomes an active research area, and it is often handled as a text classification task. As recent popular methods for text classification tasks, pre-trained language model-based methods are at the forefront of natural language processing, achieving state-of-the-art performance on various NLP tasks. However, there is a paucity in studies using such methods on toxic comment classification. In this work, we study how to best make use of pre-trained language model-based methods for toxic comment classification and the performances of different pretrained language models on these tasks. Our results show that, Out of the three most popular language models, i.e. BERT, RoBERTa, and XLM, BERT and RoBERTa generally outperform XLM on toxic comment classification. We also prove that using a basic linear downstream structure outperforms complex ones such as CNN and BiLSTM. What is more, we find that further fine-tuning a pretrained language model with light hyper-parameter settings brings improvements to the downstream toxic comment classification task, especially when the task has a relatively small dataset. CCS CONCEPTS • Computing methodologies → Neural networks; • Social and professional topics → User characteristics.

Applied Artificial Intelligence

The enormous amount of data being generated on the web and social media has increased the demand for detecting online hate speech. Detecting hate speech will reduce their negative impact and influence on others. A lot of effort in the Natural Language Processing (NLP) domain aimed to detect hate speech in general or detect specific hate speech such as religion, race, gender, or sexual orientation. Hate communities tend to use abbreviations, intentional spelling mistakes, and coded words in their communication to evade detection, adding more challenges to hate speech detection tasks. Thus, word representation will play an increasingly pivotal role in detecting hate speech. This paper investigates the feasibility of leveraging domain-specific word embedding in Bidirectional LSTM based deep model to automatically detect/classify hate speech. Furthermore, we investigate the use of the transfer learning language model (BERT) on hate speech problem as a binary classification task. The experiments showed that domain-specific word embedding with the Bidirec-tional LSTM based deep model achieved a 93% f1-score while BERT achieved up to 96% f1-score on a combined balanced dataset from available hate speech datasets.

Ethics and Information Technology, 2019

In this paper we explore quarantining as a more ethical method for delimiting the spread of Hate Speech via online social media platforms. Currently, companies like Facebook, Twitter, and Google generally respond reactively to such material: offensive messages that have already been posted are reviewed by human moderators if complaints from users are received. The offensive posts are only subsequently removed if the complaints are upheld; therefore, they still cause the recipients psychological harm. In addition, this approach has frequently been criticised for delimiting freedom of expression, since it requires the service providers to elaborate and implement censorship regimes. In the last few years, an emerging generation of automatic Hate Speech detection systems has started to offer new strategies for dealing with this particular kind of offensive online material. Anticipating the future efficacy of such systems, the present article advocates an approach to online Hate Speech d...

ACM Transactions on Social Computing

In multiplayer online games, players from different cultural backgrounds come together to cooperate and compete in real time. Although these games are enjoyed by billions of players globally, behavioral issues such as toxic behavior (TB) have become rampant in some games, perhaps most infamously in multiplayer online battle arena (MOBA) games, such as League of Legends and Dota 2. A crucial step in curbing TB lies in understanding its drivers and antecedents. In the present work, we contribute to this field of research by considering the regional offline cultural environment in which players live. We draw both on Hofstede's Cultural Dimension Framework and on Kordyaka et al.’s Unified Theory of Toxic Behavior (UTTB) to compare two cross-sectional samples of MOBA players: one from North America (n = 155) and one from India (n = 119). Our analysis reveals significant differences between the samples for all UTTB variables. Additional analyses also indicate the relevance of national...

ACM Computing Surveys

The proliferation of harmful content on online platforms is a major societal problem, which comes in many different forms including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self harm, and many other. Online platforms seek to moderate such content to limit societal harm, to comply with legislation, and to create a more inclusive environment for their users. Researchers have developed different methods for automatically detecting harmful content, often focusing on specific sub-problems or on narrow communities, as what is considered harmful often depends on the platform and on the context. We argue that there is currently a dichotomy between what types of harmful content online platforms seek to curb, and what research efforts there are to automatically detect such content. We thus survey existing methods as well as content moderation policies by online platforms in this light and we suggest directions for...

Neurocomputing

With the multiplication of social media platforms, which offer anonymity, easy access and online community formation and online debate, the issue of hate speech detection and tracking becomes a growing challenge to society, individual, policy-makers and researchers. Despite efforts for leveraging automatic techniques for automatic detection and monitoring, their performances are still far from satisfactory, which constantly calls for future research on the issue. This paper provides a systematic review of literature in this field, with a focus on natural language processing and deep learning technologies, highlighting the terminology, processing pipeline, core methods employed, with a focal point on deep learning architecture. From a methodological perspective, we adopt PRISMA guideline of systematic review of the last 10 years literature from ACM Digital Library and Google Scholar. In the sequel, existing surveys, limitations, and future research directions are extensively discussed. Keywords Hate speech detection Review • Systematic review • PRISMA hate speech • NLP deep learning review 6. YouTube: 'We remove content promoting violence or hatred against individuals or groups based on any of the following attributes: age, caste, disability, ethnicity, gender identity and expression, nationality, race, immigration status, religion, sex/gender, sexual orientation, victims of a major violent event and their kin, and veteran Status'. 4 To better understand the definitions, we consider different terms analysis from the above definitions. Tab. 2 summarizes key components that characterize HS in these definitions. For instance, all the definitions point out that HS has specific targets (person, group, nationality, etc.). Furthermore, most of these definitions refer to religion, gender discrimination, race, color, ethnicity, and violence. While less common criteria for measuring HS include curse, disability, property damage, age, and serious disease. 2.2 Other Related Concepts From the above definitions and contents analysis, it is clear that some elements are highly related to hate speech (e.g., racism, violence, gender discrimination, etc.). Moreover, we have found several previous works that have presented significant branches of HS. The analysis of HS's different branches helps to reach insights from different perspectives. This expects to contribute to spotting and recognizing the interrelationships among these terminologies. Here we will discuss some of the essential categories that are found relevant in most Hate Speech studies: Cyberbullying: Chen et al. [31], Dinakar et al. [37] defined the electronic form of traditional bullying, also, referred to cyberbullying, as the aggression and harassment targeted to an individual who is unable to defend himself. Dredge et al. opined that bullying is known for its repetitive act to the same individual, unlike hate speech which is more general and not necessarily intended to hurt a specific individual [41]. Racism: This category includes racial offense or tribalism, regionalism, xenophobia (especially for migrant workers) and nativism (hostility against immigrants and refugees), and any prejudice against a particular tribe, region, color, or physical posture of an individual. For instance, offending an individual because he belongs to a specific tribe, region, or country [8] Sexism, Gender discrimination: O'Brien [101] defined sexism as a prejudice or a discrimination based on a person's sex or gender. Sexism can affect anyone, although, it primarily affects women and girls. It has also been linked to stereotypes and gender roles. Matsumoto [86] stated that in many types of hate, there could be the existence of sexual harassment contents. Moreover, Jha and Mamidi [65] reported that sexism might come in two different forms: Hostile (which is an explicit negative attitude) and Benevolent (which is more subtle). Radicalization: This concept is usually referred to as a motive towards violent extremism. Radicalization and hate speeches are closely related and sometimes used equivalently. Some authors link radicalization to religious-based hate speech. Wadhwa and Bhatia [142] referred to radical groups as "cyber-extremists." Abusive language, Offensive language: The term abusive language refers to hurtful language and includes hate speech, derogatory language, and profanity. However, many researchers referred to the abusive language as offensive language [99].

Scientific Reports, 2021

Online debates are often characterised by extreme polarisation and heated discussions among users. The presence of hate speech online is becoming increasingly problematic, making necessary the development of appropriate countermeasures. In this work, we perform hate speech detection on a corpus of more than one million comments on YouTube videos through a machine learning model, trained and fine-tuned on a large set of hand-annotated data. Our analysis shows that there is no evidence of the presence of "pure haters", meant as active users posting exclusively hateful comments. Moreover, coherently with the echo chamber hypothesis, we find that users skewed towards one of the two categories of video channels (questionable, reliable) are more prone to use inappropriate, violent, or hateful language within their opponents' community. Interestingly, users loyal to reliable sources use on average a more toxic language than their counterpart. Finally, we find that the overall toxicity of the discussion increases with its length, measured both in terms of the number of comments and time. Our results show that, coherently with Godwin's law, online debates tend to degenerate towards increasingly toxic exchanges of views. Public debates on social media platforms are often heated and polarised 1-3. Back in the 90s, Mike Godwin coined a theorem, today known as Godwin's law, stating that "As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches to one". More recently, with the advent of social media, an increasing number of people is reporting exposure to online hate speech 4 , leading institutions and online platforms to investigate possible solutions and countermeasures 5. To prevent and counter the spread of hate speech online, for example, the European Commission agreed with Facebook, Microsoft, Twitter, YouTube, Instagram, Snapchat, Dailymotion, Jeuxvideo.com, and TikTok on a "Code of conduct on countering illegal hate speech online" 6. In addition to fuelling the toxicity of the online debate, hate speech may have severe offline consequences. Some researchers hypothesised a causal link between online hate and offline violence 7-9. Furthermore, there is empirical evidence that online hate may induce fear of offline repercussions 10. However, the detection and contrast of hate speech is complicated. There are still ambiguities in the very definition of hate speech, with academic and relevant stakeholders providing their own interpretations 4 , including social media companies such as Facebook 11 , Twitter 12 , and YouTube 13. We use the term "hate speech" to cover whole spectrum of language used in online debates, from normal, acceptable to the extreme, inciting violence. On the extreme end, violent speech covers all forms of expression which spread, incite, promote or justify racial hatred, xenophobia, antisemitism or other forms of hatred based on intolerance, including: intolerance expressed by aggressive nationalism and ethnocentrism, discrimination and hostility against minorities, migrants and people of immigrant origin 14. Less extreme forms of unacceptable speech include inappropriate (e.g., profanity) and offensive language (e.g., dehumanisation, offensive remarks), which is not illegal, but deteriorates public discourse and can lead to a more radicalised society. In this work, we analyse a corpus of more than one million comments on Italian YouTube videos related to COVID-19 to unveil the dynamics and trends of online hate. First, we manually annotate a large corpus of YouTube comments for hate speech, and train and fine-tune a hate speech deep learning model to accurately detect it. Then, we apply the model to the entire corpus, aiming to characterise the behaviour of users producing hate, and shed light on the (possible) relationship between the consumption of misinformation and usage of hate and toxic language. The reason for performing hate speech detection on the Italian language is twofold: First, Italy was one of the countries most affected by the COVID-19 pandemic and especially by the early application of non-pharmaceutical interventions (strict lockdown happened on March 9, 2020). Such an event, by forcing people at home, increased the internet use and was likely to exacerbate the public debate and foment hate speech against specific targets such as the government and politicians. Second, Italian is a less studied language

Natural Language Engineering, 2022

The quality of annotations in manually annotated hate speech datasets is crucial for automatic hate speech detection. This contribution focuses on the positive effects of manually annotating online comments for hate speech within the context in which the comments occur. We quantify the impact of context availability by meticulously designing an experiment: Two annotation rounds are performed, one in-context and one out-of-context, on the same English YouTube data (more than 10,000 comments), by using the same annotation schema and platform, the same highly trained annotators, and quantifying annotation quality through inter-annotator agreement. Our results show that the presence of context has a significant positive impact on the quality of the manual annotations. This positive impact is more noticeable among replies than among comments, although the former is harder to consistently annotate overall. Previous research reporting that out-of-context annotations favour assigning non-hate-speech labels is also corroborated, showing further that this tendency is especially present among comments inciting violence, a highly relevant category for hate speech research and society overall. We believe that this work will improve future annotation campaigns even beyond hate speech and motivate further research on the highly relevant questions of data annotation methodology in natural language processing, especially in the light of the current expansion of its scope of application.

Journal of Education College Wasit University

Language can be used to express positive thoughts and reflect positive perceptions of others. Language, on the other hand, might do the opposite. Hate speech is the language directed at a particular group of people. It incites hostility, violence, or discrimination against others based on a difference in race, religion, or gender, among others. People have become more interested in hate speech due to its impact on individuals and societies. It is argued that hate speech is common in American animated movies that are produced for the young and children. This renders such a phenomenon more dangerous. This study aims to answer the following questions: What are the motivations that incite hate speech in three different American animated movies? What are the pragmatic strategies used to convey it in the data under scrutiny? Which of these pragmatic strategies are the most prevalent? And has hate speech changed in terms of its motivations and pragmatic manifestation through time? This stu...

2022

The spread of online hate has become a major problem for newspapers that host comment sections. As a result, there is growing interest in using machine learning (ML) and natural language processing (NLP) for (semi-) automated abusive language detection to avoid manual comment moderation costs or having to shut down comment sections all together. However, much of the past work on abusive language detection with ML uses random train-test splitting procedures that assume an unrealistically static language environment. In this paper, we show using a new German newspaper comments dataset that a time-stratified evaluation procedure provides a more realistic measure of a classifier's performance on future data. We also show that the performance of classifiers can degrade quickly as the training data grows more outdated and language and news coverage evolve. Further, we demonstrate that the performance of classifiers trained on data from before the COVID-19 pandemic drops sharply when evaluated on COVID-era comments. Our findings suggest that when standard ML techniques are applied naively to abusive language detection, a classifier will fail to meet the advertised evaluation benchmarks in the real-world environment.