Text-based Emotion Aware Recommender
2020, Computer Science & Information Technology
https://doi.org/10.5121/CSIT.2020.101009Abstract
We apply the concept of users' emotion vectors (UVECs) and movies' emotion vectors (MVECs) as building components of Emotion Aware Recommender System. We built a comparative platform that consists of five recommenders based on content-based and collaborative filtering algorithms. We employed a Tweets Affective Classifier to classify movies' emotion profiles through movie overviews. We construct MVECs from the movie emotion profiles. We track users' movie watching history to formulate UVECs by taking the average of all the MVECs from all the movies a user has watched. With the MVECs, we built an Emotion Aware Recommender as one of the comparative platforms' algorithms. We evaluated the top-N recommendation lists generated by these Recommenders and found the top-N list of Emotion Aware Recommender showed serendipity recommendations.
References (53)
- 3389 Let's Get Harry (1986) 35 3999 Vertical Limit (2000) 36 4367 Lara Croft: Tomb Raider (2001) 37 5088 Going Places (Valseuses, Les) (1974) 38 5244 Shogun Assassin (1980) 39 5540 Clash of the Titans (1981) 40 5667 Tuck Everlasting (2002) 41 6911 Jolson Story, The (1946) 42 7132 Night at the Opera, A (1935) 43 7248 Suriyothai (a.k.a. Legend of Suriyothai, The) (2001) 44 7302 Thief of Bagdad, The (1924) 45 7925 Hidden Fortress, The (Kakushi-toride no san-akunin) (1958) 46 25946 Three Musketeers, The (1948) 47 31923 Three Musketeers, The (1973) 48 43556 Annapolis (2006)
- 48394 Pan's Labyrinth (Laberinto del fauno, El) (2006) 50 50003 DOA: Dead or Alive (2006)
- 95441 Ted (2012) 54 96530 Conception (2011) 55 106920 Her (2013) 56 112175 How to Train Your Dragon 2 (2014) 57 112911 Hercules (2014) 58 115713 Ex Machina (2015) 59 115727 Crippled Avengers (Can que) (Return of the 5 Deadly Venoms) (1981) 60 122918 Guardians of the Galaxy 2 (2017) 61 131714 Last Knights (2015) 62 134775 Dragon Blade (2015) 63 147662 Return of the One-Armed Swordsman (1969) 64 149406 Kung Fu Panda 3 (2016) 65 160563 The Legend of Tarzan (2016)
- CONCLUSION Leverage on our prior work in affective computing [1] that making use of the Tweets Affective Classifier (TAC) to generate our needed movie emotion labeled dataset; we demonstrated in this paper a method to build an Emotion Aware Recommender (EAR) with intriguing results. We developed a Recommender platform using the following Recommender algorithms: Item-based Collaborative Filtering (IBCF), User-based Collaborative Filtering (UBCF), Content-based movie Genres Aware Recommender (GAR), Content-based Emotion Aware Recommender (EAR), and Content-based Multi-channel Emotion Aware Recommender (MAR). With each Recommender algorithm, we generate a top-20 recommendation list. We randomly selected user id 400 as an active user for testing. We compute the emotion profile, wvec, for the test user. Using the test user's wvec and the list of ivec from each of the top-20 list, we computed the top-5 for each top-20 list generated by the Recommenders. The top-N list made by each Recommender is unique, with few overlaps among the lists. We have a total of 100 movies in the combined top- 20 lists. We found 35 duplicated films among the top-20 recommendation lists. The top-N list made by each Recommender met its design focus. For example, GAR correctly recommended movies that meet the active test user's genre taste. EAR, on the other hand, shows intrigue results. We believe that with further investigation, we could enhance EAR to make serendipity recommendations. REFERENCES
- Leung, John Kalung, Griva, Igor & Kennedy, William G., (2020) "Making Use of Affective Features from Media Content Metadata for Better Movie Recommendation Making", Preprint at https://arxiv.org/abs/2007.00636.
- Wang, Yibo, Wang, Mingming & Xu, Wei, (2018) "A sentiment-enhanced hybrid recommender system for movie recommendation: a big data analytics framework", Wireless Communications and Mobile Computing.
- Wang, Wei & Wang, Hongwei, (2015) "Opinion-enhanced collaborative filtering for recommender systems through sentiment analysis", New Review of Hypermedia and Multimedia, Vol, 21, No. (3- 4), pp278-300.
- Kumar, Gaurav, (2018) "A multi-criteria decision making approach for recommending a product using sentiment analysis", In 2018 12th International Conference on Research Challenges in Information Science (RCIS), IEEE, pp1-6.
- Katarya, Rahul & Verma, Om Prakash, (2016) "Recent developments in affective recommender systems",Physica A: Statistical Mechanics and its Applications, Vol. 461, pp182-190.
- Mizgajski, Jan & Morzy, Miko\laj, (2019) "Affective recommender systems in online news industry: how emotions influence reading choices", User Modeling and User-Adapted Interaction, Vol. 29, Springer,pp345-379.
- Haruna, Khalid, Akmar Ismail, Maizatul, Suhendroyono, Suhendroyono, Damiasih, Damiasih, Pierewan, Adi Cilik, Chiroma, Haruna & Herawan, Tutut, (2017) "Context-aware recommender system: A review of recent developmental process and future research direction", Applied Sciences, Vol. 7 No. 12, pp1211.
- Tkalčič, Marko, Burnik, Urban, Odić, Ante, Košir, Andrej & Tasič, Jurij, (2012) "Emotion-aware recommender systems-a framework and a case study", In International Conference on ICT Innovations, Springer, pp141-150.
- Qian, Yongfeng, Zhang, Yin, Ma, Xiao, Yu, Han & Peng. Limei, (2019) "EARS: Emotion-aware recommender system based on hybrid information fusion", Information Fusion, Elsevier, Vol. 46, pp141-146.
- Orellana-Rodriguez, Claudia, Diaz-Aviles, Ernesto & Nejdl, Wolfgang, (2015) "Mining affective context in short films for emotion-aware recommendation", In Proceedings of the 26th ACM Conference on Hypertext & Social Media, pp185-194.
- Sundermann, Camila Vaccari, Domingues, Marcos Aurélio, Sinoara, Roberta Akemi, Marcacini, Ricardo Marcondes & Rezende, Solange Oliveira, (2019) "Using opinion mining in context-aware recommender systems: A systematic review", Information 10, Multidisciplinary Digital Publishing Institute, Vol. 42.
- Plutchik, Robert, (2001) "The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice", American scientist, Vol. 89, No. 4, pp344-350.
- Noroozi, Fatemeh, Kaminska, Dorota, Corneanu, Ciprian, Sapinski, Tomasz, Escalera, Sergio & Anbarjafari , Gholamreza, (2018) "Survey on emotional body gesture recognition", IEEE transactions on affective computing, IEEE.
- Qian, Yongfeng, Zhang, Yin, Ma, Xiao, Yu, Han & Peng, Limei, (2019) "EARS: Emotion-aware recommender system based on hybrid information fusion", Vol. 46, pp141-146.
- Narducci, Fedelucio, De Gemmis, Marco & Lops, Pasquale, (2015) "A general architecture for an emotion-aware content-based recommender system", In Proceedings of the 3rd Workshop on Emotions and Personality in Personalized Systems , pp3-6.
- Alhijawi, & Kilani, Yousef, (2020) "The recommender system: A survey. International Journal of Advanced Intelligence Paradigms ", Inderscience Publishers, Vol. 15, pp229-251.
- Mizgajski, Jan & Morzy, Mikolaj, (2019) ''Affective recommender systems in online news industry: how emotions influence reading choices", User Modeling and User-Adapted Interaction, Vol. 29, No. 2, pp345-379.
- Schroff, Florian, Kalenichenko, Dmitry & Philbin, James, (2015) "Facenet: A unified embedding for face recognition and clustering", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp815-823.
- Alimuin, Ryann, Dadios, Elmer, Dayao, Jonathan & Arenas, Shearyl, (2020) "Deep hypersphere embedding for real-time face recognition", Telkomnika, Vol. 18.
- Lin, Tsung-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Dollár, Piotr & Zitnick, C Lawrence, (2014) "Microsoft coco: Common objects in context", In European conference on computer vision, Springer, pp740-755.
- Kim, Dae-Hwan, (2019) "Evaluation of COCO Validation 2017 Dataset with YOLOv3", Evaluation 6,pp10356-10360.
- Klare, Brendan F, Klein, Ben, Taborsky, Emma, Blanton, Austin, Cheney, Jordan, Allen, Kristen, Grother, Patrick, Mah, Alan & Jain Anil K, (2015) "Pushing the frontiers of unconstrained face detection and recognition: Iarpa janus benchmark a", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp1931-1939.
- Nada, Hajime, Sindagi, Vishwanath A., Zhang, He & Patel, Vishal M., (2018) "Pushing the limits of unconstrained face detection: a challenge dataset and baseline results", In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp1-10.
- Özdil, Ahmet & Özbilen, Metin Mete, (2014) "A survey on comparison of face recognition algorithms", In 2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), IEEE, pp1-3.
- Wu, Yue & Ji, Qiang (2019) " Facial landmark detection: A literature survey", International Journal of Computer Vision 127, Springer, pp115-142.
- De Silva, Liyanage C, Miyasato, Tsutomu & Nakatsu, Ryohei, (1997) "Facial emotion recognition using multi-modal information", In Proceedings of ICICS 1997 International Conference on Information, Communications and Signal Processing. Theme: Trends in Information Systems Engineering and Wireless Multimedia Communications, IEEE, Vol. 1, pp397-401.
- Zhang, Jianhua, Yin, Zhong , Chen, Peng & Nichele, Stefano, (2020) "Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review", Information Fusion 59, Elsevier,pp103-126.
- Maas, Andrew L., Daly, Raymond E., Pham, Peter T., Huang, Dan, Ng, Andrew Y. & Potts, Christopher, (2011) "Learning Word Vectors for Sentiment Analysis", In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA., Association for Computational Linguistics, pp142-150.
- Ye, Zhe, Li, Fang & Baldwin, Timothy, (2018) "Encoding sentiment information into word vectors for sentiment analysis", In Proceedings of the 27th International Conference on Computational Linguistics, pp997-1007.
- Rezaeinia, Seyed Mahdi, Rahmani, Rouhollah , Ghodsi, Ali & Veisi, Hadi, (2019) "Sentiment analysis based on improved pre-trained word embeddings", Expert Systems with Applications 117, Elsevier, pp139-147.
- Ekman, Paul, (1999) "Basic emotions", Handbook of cognition and emotion, Vol. 98, No. 45-60, pp16.
- Siedlecka, Ewa & Denson, Thomas F, (2019) "Experimental methods for inducing basic emotions: A qualitative review", Emotion Review 11, SAGE Publications Sage UK, London, England, pp87-97.
- Ekman, Paul, Friesen, Wallace V. & Ellsworth, Phoebe, (2013) "Emotion in the human face: Guidelines for research and an integration of findings", Elsevier, Vol. 11.
- Matsumoto, David & Hwang, Hyi Sung C, (2019) "Culture, emotion, and expression", Cross-Cultural Psychology: Contemporary Themes and Perspectives, Wiley Online Library,pp501-515.
- Tayib, Saifulazmi & Jamaludin, Zulikha, (2016) "An algorithm to define emotions based on facial gestures as automated input in survey instrument", Advanced Science Letters, Vol. 22, No. 10, pp2889-2893.
- Li, Shan & Deng, Weihong, (2020) "Deep facial expression recognition: A survey", IEEE Transactions on Affective Computing, IEEE.
- Van't Wout, Mascha, Kahn, René S, Sanfey, Alan G. & Aleman, André, (2006) "Affective state and decision-making in the ultimatum game", Experimental brain research, Vol. 169, No. 4, pp564-568.
- Heliman, Renata M, (2018) "A new look at the ultimatum game: Relational and individual differences underlying the division of gains and losses", Nova Science Publishers.
- Toda, Masanao (1980) "Emotion and decision making", Acta Psychologica, Vol. 45, No. 1-3, pp133- 155.
- Kratzwald, Bernhard, Ilić, Suzana, Kraus, Mathias , Feuerriegel, Stefan & Prendinger, Helmut,(2018) "Deep learning for affective computing: Text-based emotion recognition in decision support", Decision Support Systems , Elsevier, Vol. 115, pp24-35.
- Sanfey, Alan G., (2007) "Social decision-making: insights from game theory and neuroscience", Science, Vol. 318 No. 5850, pp598-602.
- Gu, Ruolei, Liu, Jie & Cui, Fang, (2019) "Pain and social decision-making: New insights from the social framing effect", Brain Science Advances 5, SAGE Publications Sage UK, London, England, pp221-238.
- Blanchette, Isabelle & Richards, Anne, (2010) "The influence of affect on higher level cognition: A review of research on interpretation, judgement, decision making and reasoning", Cognition & Emotion, Vol. 24, No. 4, pp561-595.
- Evans, Jonathan St BT, (2019) "Hypothetical thinking: Dual processes in reasoning and judgement", Psychology Press.
- Wright, William F. & Bower, Gordon H., (1992) "Mood effects on subjective probability assessment", Organizational behavior and human decision processes, Vol. 52, No. 2, pp276-291.
- Diener, Ed, Oishi, Shigehiro & Tay, Louis, (2018) "Advances in subjective well-being research", Nature Human Behaviour 2, Nature Publishing Group, pp253.
- Harper, F Maxwell & Konstan, Joseph A., (2016) "The movielens datasets: History and context", ACM Transactions on Interactive Intelligent Systems (TiiS), Vol. 5, No. 4, pp19.
- TMDb, (2018) "Tmdb about", Published online at www.themoviedb.org, Accessed: 2018-05-11 17:29:37.
- Wikipedia, (2018) "Imdb", Published online at en.wikipedia.org, Accessed: 2018-05-11 17:01:41.