Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Data Storage and Analysis
Data Manager
Data Processing Manager
Results for Speaker Identification
Results for Speaker Emotion Recognition
Discussion
Conclusions
References
All Topics
Chemistry
Analytical Chemistry

A Smart Mirror for Emotion Monitoring in Home Environments

Profile image of Raimondo SchettiniRaimondo Schettini

2021, Sensors

https://doi.org/10.3390/S21227453
visibility

…

description

23 pages

link

1 file

Abstract

Smart mirrors are devices that can display any kind of information and can interact with the user using touch and voice commands. Different kinds of smart mirrors exist: general purpose, medical, fashion, and other task specific ones. General purpose smart mirrors are suitable for home environments but the exiting ones offer similar, limited functionalities. In this paper, we present a general-purpose smart mirror that integrates several functionalities, standard and advanced, to support users in their everyday life. Among the advanced functionalities are the capabilities of detecting a person’s emotions, the short- and long-term monitoring and analysis of the emotions, a double authentication protocol to preserve the privacy, and the integration of Alexa Skills to extend the applications of the smart mirrors. We exploit a deep learning technique to develop most of the smart functionalities. The effectiveness of the device is demonstrated by the performances of the implemented funct...

Related papers

Smart Doll: Emotion Recognition Using Embedded Deep Learning
Oscar Deniz

Symmetry

Computer vision and deep learning are clearly demonstrating a capability to create engaging cognitive applications and services. However, these applications have been mostly confined to powerful Graphic Processing Units (GPUs) or the cloud due to their demanding computational requirements. Cloud processing has obvious bandwidth, energy consumption and privacy issues. The Eyes of Things (EoT) is a powerful and versatile embedded computer vision platform which allows the user to develop artificial vision and deep learning applications that analyse images locally. In this article, we use the deep learning capabilities of an EoT device for a real-life facial informatics application: a doll capable of recognizing emotions, using deep learning techniques, and acting accordingly. The main impact and significance of the presented application is in showing that a toy can now do advanced processing locally, without the need of further computation in the cloud, thus reducing latency and removi...

View PDFchevron_right
Emotion Detection IoT enabled Edge-node for Citizen Security
Subhra Shankha Bhattacherjee

2019 IEEE 5th World Forum on Internet of Things (WF-IoT)

In this paper, we have proposed a novel architecture for citizen security using deep learning based edge computing device. The proposed system comprises of an end-device which can be connected to every street light pole in an area. The gateway device can be connected at a large distance from the end-devices. LoRa physical layer is used as the main communication interface which enables large distance transmission. CNN based algorithm is used to identify the expression of the person who pressed the panic button on the device to reduce false alarm. A very low complex Convolutional Neural Network (CNN) architecture has been developed, to recognize the expression on the end device itself. A data logging server is used for logging the node-id everytime it is pressed. An android application is also developed which shows the real-time location of the pressed node and provides navigation routes to reach that node.

View PDFchevron_right
Smart Mirror: The New Future
Sandeep Pesala

Journal of emerging technologies and innovative research, 2021

This paper describes the designing and implementation of a voice controlled wall mirror, called "SMART MIRROR". It is a device that works both as a mirror and an interactive display displaying multimedia content such as time, date, weather and news simultaneously. The user can interact using voice commands. The Smart Mirror consists of various functionalities like real time data and information updates, voice commands, face detection/recognition using LCD monitor, microphone and webcam. The user can interact with Smart mirror using voice commands. The Smart Mirror is based on Raspberry pi 3 B model, through facial recognition and speech recognition model, mirror can identify the user. Using Machine learning algorithms the mirror can give user recommendations to utilize his time more effectively. Users will be able to control appliances in the home using relays.

View PDFchevron_right
Magic mirror table for social-emotion alleviation in the smart home
Dwenren Tsai

IEEE Transactions on Consumer Electronics, 2012

In this paper, we propose a prototype of smart furniture for the smart home-a magic mirror table. The proposed system has a camera to capture the viewer's facial expression. By analyzing the expressions, the system is able to determine the emotion of the viewer. If the viewer is in a negative emotion, the system then speaks positive sentences and plays the viewer's favorite music to alleviate his/her emotion. The experimental results confirm that the system is able to relieve the sad mood of the viewer. In addition, the proposed system can serve as a calendar for event reminding. 1

View PDFchevron_right
AN ARTIFICIALLY INTELLIGENT IOT BASED SMART MIRROR
IJSREM JOURNAL, Rashmi Venkatesh Topannavar

IJSREM Journal, 2020

In this world everyone continues running behind the comfort and solace. Present-day man has an assorted development for a comparative reason. People are happy to get to data adequately whether or not it is through the TV or web. A Smart Mirror is proposed which is an intelligent framework that makes the client get refreshed of the everyday happenings. It will be fascinating if the Mirror perceives the client and reminds the significant gatherings on that day. The Smart Mirror gives a nearby encounter that empowers the client to just walk around and be welcomed with data. The proposed mirror framework attempts to mitigate the house part's terrible mindset by giving positive explanations and music treatment. Accordingly, from an expansive perspective, the proposed mirror framework gives a social connection between the viewer and the mirror. Index Terms-Two-way mirror, Raspberry Mirror, Haar cascade classifier.

View PDFchevron_right
ARTIFICIAL INTELLIGENCE EMBEDDED MIRROR
Tijo Sebastian, Rohith Cheloor, WARSE The World Academy of Research in Science and Engineering

International Journal of Networks and Systems, 2019

A Smart Mirror is a device that is an extended and enhanced version of conventional mirror, which serves more features than what a conventional mirror could perform. It allows the user to access and interact with various features that can be found in smart devices, like smart phones. A Smart Mirror can display text, image, videos and all kinds of data supported by a device. The main advantage of a Smart mirror will be its capability to work as a conventional mirror, as well as a Smart Home device. This device can replace a wide range of household utilities like Clocks, Calendar, External voice assistants like Amazon Echo and Google Home, Temperature monitors and even a Music player. In this paper, we discuss about the design and implementation of a Smart Mirror.

View PDFchevron_right
IRJET- Personalized Smart Mirror with User Detection
Deep Mehta, IRJET Journal

IRJET, 2021

It is a growing need for one to organize one's day in an efficient manner. One of the ways is to use a smartphone which is proving to be more destructive than productive. In order to reduce screen time on a smartphone, we can use a daily use object, a mirror, to help organize the day while not having to reach out to your smartphone for basic needs. The mirror can be used as a smart device that can offer services like displaying weather, date and time, latest updates of news headlines, user personalized daily tasks and reminders, some motivational quotes to get the day going positively with voice assistance. This use of the mirror can help the user save time and multitask by planning their day. The smart device uses Raspberry Pi as the central controller to dictate the actions to perform. When a user stands in front of this smart mirror, the controller recognizes the person and it displays the tasks that a particular user has to perform. It can also be used to drop messages for other users registered under the same smart system. The mirror allows to command the mirror using voice and choose what is to be done, e.g., Display the user tasks or display messages. The smart device comes with a mobile application that enables other users to check their tasks or messages when the mirror is unreachable.

View PDFchevron_right
NEURAL MOOD: DEEP LEARNING BASED EMOTION ANALYSIS FOR SMART SYSTEMS
Ajay Sai Ayyagari, Khushi Kannamwar

Emotion recognition importantly improves human-computer interaction, monitors mental health alongside personalizes user experiences. Neural Mood leverages deep learning techniques for analysis and interpretation of human emotions in real-time, which enables smart systems to respond dynamically based on emotional states. Emotion detection is accurate because this project uses recurrent neural networks (RNNs) and convolutional neural networks (CNNs); it also processes voice tone, facial expressions, and physiological signals. The system gives helpful perceptions to improve user engagement and well-being since its design allows it to adapt across many domains like healthcare customer service education and entertainment. NeuralMood improves emotion classification precision it is thus a powerful solution. It can use large-scale emotion datasets with advanced feature extraction methods for making this enhancement in next-generation AI-driven applications. The project also explores some of the ethical considerations. This exploration makes sure that AI deployment in emotion analysis is done responsibly.

View PDFchevron_right
Smart mirror
Shubham Chaskar

as we all know the world is changing continuously.Every abject around us is getting smarter each day with the help of trending technology. The main question comes "when everything around us is getting smarter then why not make a mirror also smart?" mirror is the object which is used only for getting dressed but not anymore . it can be used as an interactive device, which will help users to make their life easier. User can interact with mirror using voice commands, whereas mirror also displays weather , News, To-do-list ,Time, Pushup notification etc. The modules can be customized according to user needs. This smart mirror has a ability of biometric authentication of users.

View PDFchevron_right
IoT based Interactive Smart Mirror
Samana Jafri

2017

Recent Advancement in Internet of Things (IoT) technology has seen many application mainly related with automation. Modernizing home appliances have become a trend but the only thing that has not been modernized is mirror. The Mirror being the most used objects in the house but there’s been no advancement in its design and application. Our project focuses on modernizing the mirror by enhancing its functionality at the same time make it more interactive and useful in saving our precious time. A mirror’s function is to display reflection of anything that comes in front of it. We present the idea of Smart Mirror which displays real-time information like mobile notifications, email notifications, Calendar, Live weather updates, reminders, and so on. In addition to this Amazon’s Alexa services have been used so that mirror takes voice commands from user. Available Smart Mirror lacks privacy of data and makes use of PIR sensor to recognize a human being also that Smart Mirror can only be ...

View PDFchevron_right

References (80)

  1. Mack, E. Toshiba's Smart Mirror Concept Puts the Future on Display. 2014. Available online: https://newatlas.com/toshiba- smart-mirror-concept-ces-2014/30574/ (accessed on 15 October 2021).
  2. Riva, G.; Baños, R.M.; Botella, C.; Wiederhold, B.K.; Gaggioli, A. Positive technology: Using interactive technologies to promote positive functioning. Cyberpsychol. Behav. Soc. Netw. 2012, 15, 69-77. [CrossRef]
  3. Grossi, G.; Lanzarotti, R.; Napoletano, P.; Noceti, N.; Odone, F. Positive technology for elderly well-being: A review. Pattern Recognit. Lett. 2019, 137, 61-70. [CrossRef]
  4. Johri, A.; Jafri, S.; Wahi, R.N.; Pandey, D. Smart mirror: A time-saving and affordable assistant. In Proceedings of the International Conference on Computing Communication and Automation (ICCCA), Greater Noida, India, 14-15 December 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1-4.
  5. Ghazal, M.; Al Hadithy, T.; Al Khalil, Y.; Akmal, M.; Hajjdiab, H. A mobile-programmable smart mirror for ambient iot environments. In Proceedings of the International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), Prague, Czech Republic, 21-23 August 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 240-245.
  6. Sun, Y.; Geng, L.; Dan, K. Design of smart mirror based on Raspberry Pi. In Proceedings of the International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), Xiamen, China, 25-26 January 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 77-80.
  7. Athira, S.; Francis, F.; Raphel, R.; Sachin, N.; Porinchu, S.; Francis, S. Smart mirror: A novel framework for interactive display. In Proceedings of the International Conference on Circuit, Power and Computing Technologies (ICCPCT), Nagercoil, India, 18-19 March 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 1-6.
  8. García, I.C.A.; Salmón, E.R.L.; Riega, R.V.; Padilla, A.B. Implementation and customization of a smart mirror through a facial recognition authentication and a personalized news recommendation algorithm. In Proceedings of the IEEE International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Jaipur, India, 4-7 December 2017; pp. 35-39.
  9. Mohamed, A.S.A.; Wahab, M.A.; Suhaily, S.; Arasu, D.B.L. Smart mirror design powered by raspberry pi. In Proceedings of the Artificial Intelligence and Cloud Computing Conference, Tokyo, Japan, 21-23 December 2018; pp. 166-173.
  10. Ding, J.R.; Huang, C.L.; Lin, J.K.; Yang, J.F.; Wu, C.H. Interactive multimedia mirror system design. IEEE Trans. Consum. Electron. 2008, 54, 972-980. [CrossRef]
  11. Park, K.H.; Tae-Seon, K.; Heo, C.R.; Kim, M.Y.; Kim, T.K. Smart Mirror with Focus Control. U.S. Patent 10,134,370, 20 November 2018.
  12. Sarnin, S.S.; Akbar, A.; Mohamad, W.N.W.; Idris, A.; Fadzlina Naim, N.; Ya'acob, N. Maleficent mirror with alexa voice services as an internet of things implement using raspberry pi 3 model b. In Proceedings of the TENCON 2018-2018 IEEE Region 10
  13. Conference, Jeju, Korea, 28-31 October 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1202-1207.
  14. Darrell, T.; Gordon, G.; Woodfill, J.; Harville, M. A virtual mirror interface using real-time robust face tracking. In Proceedings of the International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14-16 April 1998; IEEE: Piscataway, NJ, USA, 1998; pp. 616-621.
  15. Fiala, M. Magic mirror and hand-held and wearable augmentations. In Proceedings of the 2007 IEEE Virtual Reality Conference, Charlotte, NC, USA, 10-14 March 2007.
  16. Jin, K.; Deng, X.; Huang, Z.; Chen, S. Design of the smart mirror based on raspberry pi. In Proceedings of the Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Xi'an, China, 25-27 May 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1919-1923.
  17. Ganesh, H.; Sharmila, S. IoT Based Home Automation using Smart Mirror. Int. J. Innov. Sci. Res. Technol. 2019, 4, 607-612.
  18. Strickland, E.; Hatrie, H.H.M. HomeMirror. 2016. Available online: https://github.com/HannahMitt/HomeMirror (accessed on 15 October 2021).
  19. Yeo, U.C.; Park, S.H.; Moon, J.W.; An, S.W.; Han, Y.O. Smart Mirror of Personal Environment using Voice Recognition. J. Korea Inst. Electron. Commun. Sci. 2019, 14, 199-204.
  20. D'souza, A.A.; Kaul, P.; Paul, E.; Dhuri, M. Ambient Intelligence Using Smart Mirror-Personalized Smart Mirror for Home Use. In Proceedings of the Bombay Section Signature Conference (IBSSC), Mumbai, India, 26-28 July 2018; IEEE: Piscataway, NJ, USA, 2019; pp. 1-5.
  21. Mathivanan, P.; Anbarasan, G.; Sakthivel, A.; Selvam, G. Home automation using smart mirror. In Proceedings of the International Conference on System, Computation, Automation and Networking (ICSCAN), Pondicherry, India, 29-30 March 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1-4.
  22. Bahendwar, I.; Bhardwaj, R.; Hindaria, S.; Rathod, V. Mirr-Active An Aritificially Intelligent Interactive Mirror. In Proceedings of the International Conference on Advanced Computation and Telecommunication (ICACAT), Bhopal, India, 28-29 December 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1-6.
  23. Kafi, A.; Alam, M.S.A.; Hossain, S.B. Artificially intelligent smart mirror using Raspberry Pi. Int. J. Comput. Appl. 2018, 180, 15-18.
  24. Kawale, J.; Chaudhari, P. IoT based Design of Intelligent Mirror using Raspberry Pi. In Proceedings of the International Conference for Convergence in Technology (I2CT), Pune, India, 29-31 March 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1-4.
  25. Liu, Y.; Jia, J.; Fu, J.; Ma, Y.; Huang, J.; Tong, Z. Magic mirror: A virtual fashion consultant. In Proceedings of the International Conference on Multimedia, Amsterdam, The Netherlands, 15-19 October 2016; pp. 680-683.
  26. Wang, L.; Villamil, R.; Samarasekera, S.; Kumar, R. Magic mirror: A virtual handbag shopping system. In Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA, 16-21 June 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 19-24.
  27. Saakes, D.; Yeo, H.S.; Noh, S.T.; Han, G.; Woo, W. Mirror mirror: An on-body t-shirt design system. In Proceedings of the CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7-12 May 2016; pp. 6058-6063.
  28. Nguyen, T.V.; Liu, L. Smart mirror: Intelligent makeup recommendation and synthesis. In Proceedings of the International Conference on Multimedia, Mountain View, CA, USA, 23-27 October 2017; ACM: New York, NY, USA, 2017; pp. 1253-1254.
  29. Sethukkarasi, C.; HariKrishnan, V.; PalAmutha, K.; Pitchian, R. Interactive mirror for smart home. Int. J. Adv. Intell. Syst. 2016, 9, 148-160.
  30. Andreu, Y.; Chiarugi, F.; Colantonio, S.; Giannakakis, G.; Giorgi, D.; Henriquez, P.; Kazantzaki, E.; Manousos, D.; Marias, K.; Matuszewski, B.J.; et al. Wize Mirror-a smart, multisensory cardio-metabolic risk monitoring system. Elsevier Comput. Vis. Image Underst. 2016, 148, 3-22. [CrossRef]
  31. Henriquez, P.; Matuszewski, B.J.; Andreu-Cabedo, Y.; Bastiani, L.; Colantonio, S.; Coppini, G.; D'Acunto, M.; Favilla, R.; Germanese, D.; Giorgi, D.; et al. Mirror mirror on the wall... an unobtrusive intelligent multisensory mirror for well-being status self-assessment and visualization. IEEE Trans. Multimed. 2017, 19, 1467-1481. [CrossRef]
  32. Erazo, O.; Pino, J.A.; Pino, R.; Asenjo, A.; Fernández, C. Magic mirror for neurorehabilitation of people with upper limb dysfunction using kinect. In Proceedings of the Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 6-9 January 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 2607-2615.
  33. Von Hollen, S.; Reeh, B. Smart Mirror Devices. In Proceedings of the International Conference on Innovations for Community Services, Bhubaneswar, India, 24-26 June 2019; Springer: Berlin/Heidelberg, Germany, 2018; pp. 194-204.
  34. Muneer, A.; Fati, S.M.; Fuddah, S. Smart health monitoring system using IoT based smart fitness mirror. Telkomnika 2020, 18, 317-331. [CrossRef]
  35. Cvetkoska, B.; Marina, N.; Bogatinoska, D.C.; Mitreski, Z. Smart mirror E-health assistant-Posture analyze algorithm proposed model for upright posture. In Proceedings of the International Conference on Smart Technologies, Bengaluru, India, 17 August 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 507-512.
  36. Moon, Y.B.; Oh, S.W.; Kang, H.J.; Lee, H.S.; Kim, S.J.; Bang, H.C. Smart mirror health management services based on iot platform. In Proceedings of the International Conference on Applications of Computer Engineering (ACE'15), Seoul, Korea, 5-7 September 2015; pp. 87-89.
  37. Iyer, S.R.; Basu, S.; Yadav, S.; Vijayanand, V.M.; Badrinath, K. Reasonably intelligent mirror. In Proceedings of the International Conference on Computational Systems and Information Technology for Sustainable Solutions (CSITSS), Karnataka, India, 20-22 December 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 302-306.
  38. Yang, R.P.; Liu, Z.T.; Zheng, L.D.; Wu, J.P.; Hu, C.C. Intelligent Mirror System Based on Facial Expression Recognition and Color Emotion Adaptation--iMirror. In Proceedings of the Chinese Control Conference (CCC), Wuhan, China, 25-27 July 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 3227-3232.
  39. Yu, Y.C.; You, S.D.; Tsai, D.R. Magic mirror table for social-emotion alleviation in the smart home. IEEE Trans. Consum. Electron. 2012, 58, 126-131. [CrossRef]
  40. Honnali, V.; Honnali, M.P.; Vibhuti, M.A.; Malakannavar, M.R. Design and Implementation of Magic Mirror for Social- Emotion Alleviation in the Smart Home Using GSM. Available online: http://www.kscst.iisc.ernet.in/spp/41_series/SPP41S/02 _Exhibition_Projects/234_41S_BE_1440.pdf (accessed on 6 November 2021).
  41. Hanlon, M. Philips HomeLab Creates Mirror TV. 2004. Available online: https://newatlas.com/philips-homelab-creates-mirror- tv/2003/ (accessed on 15 October 2021).
  42. Groups Inc., C. Mango Mirror. Available online: https://www.mangomirror.com/ (accessed on 15 October 2021).
  43. Hanlon, M. Ekko Smart Mirror Puts a Wealth of Information Right in Front of Your Face. 2016. Available online: https://www. digitaltrends.com/home/ekko-smart-mirror-integrates-functional-technology-everyday-object/ (accessed on 15 October 2021).
  44. Crist, R. Griffin Technology Takes a Stab at the Smart Mirror at CES 2017. 2017. Available online: https://www.cnet.com/ reviews/griffin-technology-connected-mirror-preview/ (accessed on 15 October 2021).
  45. Memomi. Memory Mirror. Available online: https://memorymirror.com/ (accessed on 15 October 2021).
  46. Carman, A. The HiMirror Plus Scanned My Face and Told Me I Have Wrinkles. 2017. Available online: https://www.theverge. com/2017/1/4/14166064/himirror-plus-scan-smart-mirror-ces-2017/ (accessed on 15 October 2021).
  47. Airnodes. Anna Smart Mirror. Available online: https://www.miroir-anna.com/ (accessed on 15 October 2021).
  48. Panasonic. Digital Concierge-Advanced Smart Mirror with IBM Watson. 2017. Available online: https://channel.panasonic. com/contents/19698/ (accessed on 15 October 2021).
  49. Tater, L.; Pranjale, S.; Lade, S.; Nimbalkar, A.; Mahalle, P. IoT based Assistive Smart Mirror with Human Emotion Recognition System. Int. J. Eng. Res. Technol. (IJERT) 2020, 9, 381-385.
  50. Bianco, S.; Celona, L.; Napoletano, P. Visual-based sentiment logging in magic smart mirrors. In Proceedings of the International Conference on Consumer Electronics-Berlin (ICCE-Berlin), Berlin, Germany, 2-5 September 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1-4.
  51. Bhosale, S.; Deshmukh, P.; Daware, M.; Pawar, R.; Boraste, P. An Industrial Purpose Smart Mirror For Mood Detection. Int. J. Emerg. Trends Technol. 2019, 6, 13044-13046.
  52. Miotto, R.; Danieletto, M.; Scelza, J.R.; Kidd, B.A.; Dudley, J.T. Reflecting health: Smart mirrors for personalized medicine. NPJ Digit. Med. 2018, 1, 62. [CrossRef]
  53. Silapasuphakornwong, P.; Uehira, K. Smart Mirror for Elderly Emotion Monitoring. In Proceedings of the Global Conference on Life Sciences and Technologies (LifeTech), Nara, Japan, 9-11 March 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 356-359.
  54. Yu, H.; Bae, J.; Choi, J.; Kim, H. LUX: Smart Mirror with Sentiment Analysis for Mental Comfort. Sensors 2021, 21, 3092. [CrossRef]
  55. CareOS. CareOS-The First Smart Health & Beauty Platform for the Bathroom. 2021. Available online: https://care-os.com/ themis/ (accessed on 15 October 2021).
  56. King, D.E. Dlib-ml: A machine learning toolkit. J. Mach. Learn. Res. 2009, 10, 1755-1758.
  57. Kazemi, V.; Sullivan, J. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23-28 June 2014; IEEE: Piscataway, NJ, USA, 2014.
  58. Bianco, S.; Celona, L.; Schettini, R. Who Is in the Crowd? Deep Face Analysis for Crowd Understanding. In Proceedings of the International Conference on Pattern Recognition, Chengdu, China, 16-18 July 2021; Springer: Berlin/Heidelberg, Germany, 2021; pp. 487-494.
  59. Mollahosseini, A.; Hasani, B.; Mahoor, M.H. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 2017, 10, 18-31. [CrossRef]
  60. Celona, L.; Bianco, S.; Schettini, R. Fine-grained face annotation using deep multi-task cnn. Sensors 2018, 18, 2666. [CrossRef] [PubMed]
  61. Liu, Z.; Luo, P.; Wang, X.; Tang, X. Deep learning face attributes in the wild. In Proceedings of the International Conference on Computer Vision, Las Condes, Chile, 11-18 December 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 3730-3738.
  62. Bianco, S.; Cereda, E.; Napoletano, P. Discriminative Deep Audio Feature Embedding for Speaker Recognition in the Wild. In Proceedings of the International Conference on Consumer Electronics-Berlin (ICCE-Berlin), Berlin, Germany, 2-5 September 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1-5.
  63. Nagrani, A.; Chung, J.S.; Zisserman, A. VoxCeleb: A large-scale speaker identification dataset. arXiv 2017, arXiv:1706.08612.
  64. Cao, H.; Cooper, D.G.; Keutmann, M.K.; Gur, R.C.; Nenkova, A.; Verma, R. Crema-d: Crowd-sourced emotional multimodal actors dataset. IEEE Trans. Affect. Comput. 2014, 5, 377-390. [CrossRef]
  65. Burkhardt, F.; Paeschke, A.; Rolfes, M.; Sendlmeier, W.F.; Weiss, B. A database of German emotional speech. In Proceedings of the European Conference on Speech Communication and Technology, Lisbon, Portugal, 4-8 September 2005.
  66. Wang, W. Machine Audition: Principles, Algorithms and Systems: Principles, Algorithms and Systems; IGI Global: Hershey, PA, USA, 2010.
  67. Pichora-Fuller, M.K.; Dupuis, K. Toronto Emotional Speech Set (TESS); Draft Version; Scholars Portal Dataverse: Toronto, ON, Canada, 2020.
  68. React Component for Avataaars. Available online: https://github.com/fangpenlin/avataaars (accessed on 15 October 2021).
  69. Sollfrank, T.; Kohnen, O.; Hilfiker, P.; Kegel, L.C.; Jokeit, H.; Brugger, P.; Loertscher, M.L.; Rey, A.; Mersch, D.; Sternagel, J.; et al. The effects of dynamic and static emotional facial expressions of humans and their avatars on the EEG: An ERP and ERD/ERS study. Front. Neurosci. 2021, 15, 459. [CrossRef] [PubMed]
  70. Fabri, M.; Elzouki, S.Y.A.; Moore, D. Emotionally expressive avatars for chatting, learning and therapeutic intervention. In Proceedings of the International Conference on Human-Computer Interaction, Beijing, China, 22-27 July 2007; Springer: Berlin/Heidelberg, Germany, 2007; pp. 275-285.
  71. Rehm, I.C.; Foenander, E.; Wallace, K.; Abbott, J.A.M.; Kyrios, M.; Thomas, N. What role can avatars play in e-mental health interventions? Exploring new models of client-therapist interaction. Front. Psychiatry 2016, 7, 186. [CrossRef] [PubMed]
  72. Python Alexa Voice Service App. Available online: https://pypi.org/project/avs/ (accessed on 15 October 2021).
  73. Livingstone, S.R.; Russo, F.A. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE 2018, 13, e0196391.
  74. Brooke, J. SUS-A quick and dirty usability scale. Usability Eval. Ind. 1996, 189, 4-7.
  75. Vakhshiteh, F.; Nickabadi, A.; Ramachandra, R. Adversarial attacks against face recognition: A comprehensive study. IEEE Access 2021, 9, 92735-92756. [CrossRef]
  76. Joshi, S.; Villalba, J.; Żelasko, P.; Moro-Velázquez, L.; Dehak, N. Study of Pre-processing Defenses against Adversarial Attacks on State-of-the-art Speaker Recognition Systems. IEEE Trans. Inf. Forensics Secur. 2021, 16, 4811-4826. [CrossRef]
  77. Chen, Z.; Gu, S.; Zhu, F.; Xu, J.; Zhao, R. Improving Facial Attribute Recognition by Group and Graph Learning. In Proceedings of the International Conference on Multimedia and Expo (ICME), Shenzhen, China, 5-9 July 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1-6.
  78. Kwon, S. Optimal feature selection based speech emotion recognition using two-stream deep convolutional neural network. Int. J. Intell. Syst. 2021, 36, 5116-5135.
  79. Kalhor, E.; Bakhtiari, B. Speaker independent feature selection for speech emotion recognition: A multi-task approach. Multimed. Tools Appl. 2021, 80, 8127-8146. [CrossRef]
  80. Bianco, S.; Cadene, R.; Celona, L.; Napoletano, P. Benchmark analysis of representative deep neural network architectures. IEEE Access 2018, 6, 64270-64277. [CrossRef]

Related papers

MirrorME: implementation of an IoT based smart mirror through facial recognition and personalized information recommendation algorithm
Uzzal Kumar Acharjee

International Journal of Information Technology

We are living in the era of the fourth industrial revolution, which also treated as 4IR or Industry 4.0. Generally, 4IR considered as the mixture of robotics, artificial intelligence (AI), quantum computing, the Internet of Things (IoT) and other frontier technologies. It is obvious that nowadays a plethora of smart devices is providing services to make the daily life of humans easier. However, in the morning most people around the globe use a traditional mirror while preparing themselves for daily tasks. The aim is to build a low-cost intelligent mirror system that can display a variety of details based on user recommendations. Therefore, in this article, Internet of Things (IoT) and AI-based smart mirror is introduced that will support the users to receive the necessary daily update of weather information, date, time, calendar, to-do list, updated news headlines, traffic updates, COVID-19 cases status and so on. Moreover, a face detection method also implemented with the smart mirror to construct the architecture more secure. Our proposed MirrorME application provides a success rate of nearly 87% in interacting with the features of face recognition and voice input. The mirror is capable of delivering multimedia facilities while maintaining high levels of security within the device.

View PDFchevron_right
Auto secure: A Device with Emotion Detection Enabled Automation and Telegram App for Home Safety
IJIRT Journal

Advancement in IOT based application has become the state-of-the art technology among the researcher due to the availability of Internet everywhere. To make the application more user friendly, web based and android based technologies have gained their importance in this cutting edge technology. In this paper, device with emotion detection enable automation using telegram app user can manage their home security. This system is very useful for reducing the cost of monitoring the movement from outside. In this paper, a real time recognition system is proposed which will handle images very quickly. The main purpose of this paper is to protect home, office by recognizing people. The proposed system is real-time, fast and has low computational cost.

View PDFchevron_right
Real-Time Speech Emotion Analysis for Smart Home Assistants
Rohit Halder

IEEE Transactions on Consumer Electronics, 2021

Artificial Intelligence (AI) based Speech Emotion Recognition (SER) has been widely used in the consumer field for control of smart home personal assistants, with many such devices on the market. However, with the increase in computational power, connectivity and the need to enable people to live in the home for longer though the use of technology, then smart home assistants that could detect human emotion will improve the communication between a user and the assistant enabling the assistant of offer more productive feedback. Thus, the aim of this work is to analyze emotional states in speech and propose a suitable method considering performance verses complexity for deployment in Consumer Electronics home products, and to present a practical live demonstration of the research. In this paper, a comprehensive approach has been introduced for the human speech-based emotion analysis. The 1-D convolutional neural network (CNN) has been implemented to learn and classify the emotions associated with human speech. The paper has been implemented on the standard datasets (emotion classification) Ryerson AudioVisual Database of Emotional Speech and Song (RAVDESS) and Toronto Emotional Speech Set database (TESS) (Young and Old). The proposed approach gives 90.48%, 95.79% and94.47% classification accuracies in the aforementioned datasets. We conclude that the 1-D CNN classification models used in speaker-independent experiments are highly effective in the automatic prediction of emotion and are ideal for deployment in smart home assistants to detect emotion.

View PDFchevron_right
Personalized Smart Mirror with User Detection
Deep Mehta

It is a growing need for one to organize one's day in an efficient manner. One of the ways is to use a smartphone which is proving to be more destructive than productive. In order to reduce screen time on a smartphone, we can use a daily use object, a mirror, to help organize the day while not having to reach out to your smartphone for basic needs. The mirror can be used as a smart device that can offer services like displaying weather, date and time, latest updates of news headlines, user personalized daily tasks and reminders, some motivational quotes to get the day going positively with voice assistance. This use of the mirror can help the user save time and multitask by planning their day. The smart device uses Raspberry Pi as the central controller to dictate the actions to perform. When a user stands in front of this smart mirror, the controller recognizes the person and it displays the tasks that a particular user has to perform. It can also be used to drop messages for other users registered under the same smart system. The mirror allows to command the mirror using voice and choose what is to be done, e.g., Display the user tasks or display messages. The smart device comes with a mobile application that enables other users to check their tasks or messages when the mirror is unreachable.

View PDFchevron_right
Alexa mirror
Soundeswaran S

In the case of college guide, the coding have been done in such a way that it can tell the details about that particular place by using a security camera as well as tell the directions for the classrooms, different blocks, labs, washrooms etc. Using Google assistant controlled display, the user can send messages to the mirror from his smart phone without using a sim card. And so this is how we will be able to achieve our main objectives by building this project.

View PDFchevron_right

Related topics

  • Computer Science
  • Human Computer Interaction
  • Analytical Chemistry
  • Usability
  • Medicine
  • Electrical and Electronic Engine...
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved