Concept-Oriented Transformers for Visual Sentiment Analysis

In this paper we study the connection between sentiment of images expressed in metadata and their visual content in the social photo sharing environment Flickr. To this end, we consider the bag-of-visual words representation as well as the color distribution of images, and make use of the SentiWordNet thesaurus to extract numerical values for their sentiment from accompanying textual metadata. We then perform a discriminative feature analysis based on information theoretic methods, and apply machine learning techniques to predict the sentiment of images. Our largescale empirical study on a set of over half a million Flickr images shows a considerable correlation between sentiment and visual features, and promising results towards estimating the polarity of sentiment in images.

Sentiment analysis of online user generated content is important for many social media analytics tasks. Researchers have largely relied on textual sentiment analysis to develop systems to predict political elections, measure economic indicators, and so on. Recently, social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Motivated by the needs in leveraging large scale yet noisy training data to solve the extremely challenging problem of image sentiment analysis, we employ Convolutional Neural Networks (CNN). We first design a suitable CNN architecture for image sentiment analysis. We obtain half a million training samples by using a baseline sentiment algorithm to label Flickr images. To make use of such noisy machine labeled data, we employ a progressive strategy to fine-tune the deep network. Furthermore, we improve the performance on Twitter images by inducing domain transfer with a small number of manually labeled Twitter images. We have conducted extensive experiments on manually labeled Twitter images. The results show that the proposed CNN can achieve better performance in image sentiment analysis than competing algorithms.

Images have become one of the most popular types of media through which users convey their emotions within online social networks. Although vast amount of research is devoted to sentiment analysis of textual data, there has been very limited work that focuses on analyzing sentiment of image data. In this work, we propose a novel visual sentiment prediction framework that performs image understanding with Convolutional Neural Networks (CNN). Specifically, the proposed sentiment prediction framework performs transfer learning from a CNN with millions of parameters, which is pre-trained on large-scale data for object recognition. Experiments conducted on two real-world datasets from Twitter and Tumblr demonstrate the effectiveness of the proposed visual sentiment analysis framework.

Visual sentiment analysis framework can predict the sentiment of an image by analyzing the image contents. Nowadays, people are uploading millions of images in social networks such as Twitter, Facebook, Google Plus, and Flickr. These images play a crucial part in expressing emotions of users in online social networks. As a result, image sentiment analysis has become important in the area of online multimedia big data research. Several research works are focusing on analyzing the sentiment of the textual contents. However, little investigation has been done to develop models that can predict sentiment of visual content. In this paper, we propose a novel visual sentiment analysis framework using transfer learning approach to predict sentiment. We use hyper-parameters learned from a very deep convolutional neural network to initialize our network model to prevent overfitting. We conduct extensive experiments on a Twitter image dataset and prove that our model achieves better performance than the current state-of-the-art.

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2021

Visual sentiment analysis is the way to automatically recognize positive and negative emotions from images, videos, graphics and stickers. To estimate the polarity of the sentiment evoked by images in terms of positive or negative sentiment, most of the state of the art works exploit the text associated with a social post provided by the user. However, such textual data is typically noisy due to the subjectivity of the user, which usually includes text useful to maximize the diffusion of the social post. This System will extract three views: visual view, subjective text view and objective text view of Flickr images and will give sentiment polarity positive, negative or neutral based on the hypothesis table. Subjective text view gives sentiment polarity using VADER (Valence Aware Dictionary and sEntiment Reasoner) and objective text view gives sentiment polarity with three convolution neural network models. This system implements VGG-16, Inception-V3 and ResNet-50 convolution neural networks with pre pre-trained ImageNet dataset. The text extracted through these three convolution networks is given to VADER as input to find sentiment polarity. This system implements visual view using a bag of visual word model with BRISK (Binary Robust Invariant Scalable Key points) descriptor. System has a training dataset of 30000 positive, negative and neutral images. All the three views’ sentiment polarity is compared. The final sentiment polarity is calculated as positive if two or more views gives positive sentiment polarity, as negative if two or more views gives negative sentiment polarity and as neutral if two or more views gives neutral sentiment polarity. If all three views give unique polarity then the polarity of the objective text view is given as output sentiment polarity.

2021 International Joint Conference on Neural Networks (IJCNN), 2021

Sentiment analysis is a research topic focused on analysing data to extract information related to the sentiment that it causes. Applications of sentiment analysis are wide, ranging from recommendation systems, and marketing to customer satisfaction. Recent approaches evaluate textual content using Machine Learning techniques that are trained over large corpora. However, as social media grown, other data types emerged in large quantities, such as images. Sentiment analysis in images has shown to be a valuable complement to textual data since it enables the inference of the underlying message polarity by creating context and connections. Multimodal sentiment analysis approaches intend to leverage information of both textual and image content to perform an evaluation. Despite recent advances, current solutions still flounder in combining both image and textual information to classify social media data, mainly due to subjectivity, inter-class homogeneity and fusion data differences. In this paper, we propose a method that combines both textual and image individual sentiment analysis into a final fused classification based on AutoML, that performs a random search to find the best model. Our method achieved state-of-the-art performance in the B-T4SA dataset, with 95.19% accuracy.

International Journal of Advanced Computer Science and Applications

Now-a-days, the increasing number of smartphones has caused the immediate sharing of photographs capturing current events on social media. The sentimental content of pictures from social events starts to be obtained from visual material, so visual sentiment analysis is a vital research topic. The research aims to reach valuable criteria to modify the visual sentiment prediction model based on event concepts and object detection techniques. In addition to adapting the approach for designing the method for predicting visual sentiments in a social network according to concept scores and measuring the performance of the model for predicting visual sentiments as accurately as possible, approach obtains a visual summary of social event images based on the visual elements that appear in the pictures which exceed sentiment-specific features. By this method, attributes (color, texture) are assigned to sentiments with discovering affective objects that are used to obtain emotions related to a picture of a social event by mapping the top predicted qualities to feelings and extracting the prevailing emotion connected with a photograph of a social event. This method is valid for a wide range of social events. This strategy also demonstrates the social event's effectiveness for a difficult social event image collection by using techniques for classifying complicated event images into sentiments, whether positive or negative.

Visual Contents such as images and video does not only contain objects, location and actions but also cues about affect, emotion and sentiment. Such information I very useful to understand visual content beyond semantic concept presence thus making it more explainable to the user. Images are the easiest medium through which people can express their emotions on social networking sites. Social media users are increasingly using images and videos to express their opinions and share their experiences. Sentiment analysis of such large scale visual content can help better extract user sentiments toward events or topics, such as those in image tweets, so that prediction of sentiment from visual content is complementary to textual sentiment analysis. Significant progress has been made with this technology, however, there is little research focus on the picture sentiments. This paper proposes a novel approach that exploits latent correlations among multiple views: visual and textual views, and a sentiment view constructed using SentiWordNet.

IEEE Transactions on Multimedia

Analysis of visual contents has always been interesting and important yet it is very challenging as well. With the increasing popularity of social grids, images are considered a very expedient way to communicate and diffusion of information among online users. To know the different patterns and different aspects of these images it is very important to first interpret these images in a simpler form. Like the textual information images also carry different levels and different types of sentiments to their spectators. Though it is quite easy to detect any type of sentiment from the text but it is very difficult to analyse sentiments from the visual images. By using the CBIR technique it would be quite easy to get the accurate image but the image with right sentiment is again a challenge. In this paper, I have presented a method which is based on psychosomatic models and web mining that can easily and automatically construct a huge set of Visual Sentiment Ontology (VSO) which comprises around 4000 ANP (Adjective Noun Pairs). I have also proposed the concept of SentiBank, a pictorial notion sensor library that can be used to sense more than 1000 ANPs in an image. These two technique, VSO and SentiBank will positively open the new doors to analyse the sentiments in an image and gives the more accurate results while accessing the images using sentiments.