Smart mirror

Smart mirror

Chaskar Shubham D.

Abstract

as we all know the world is changing continuously.Every abject around us is getting smarter each day with the help of trending technology. The main question comes “when everything around us is getting smarter then why not make a mirror also smart?” mirror is the object which is used only for getting dressed but not anymore, it can be used as an interactive device, which will help users to make their life easier. User can interact with mirror using voice commands, whereas mirror also displays weather, News, To-do-list, Time, Pushup notification etc. The modules can be customized according to user needs. This smart mirror has a ability of biometric authentication of users.

Index Terms-Smart Mirror, IOT, Raspberry Pi3, PIR sensor

I. INTRODUCTION

The world around is constantly changing. Interactive computing, with wirelessly connected device that are being used in various day to day activities, are changing and improving the standard of the quality life. Based on this interactive computing and communication technologies, many devices products are now emerging and with this multimedia intelligence it is providing comfortable, secure and convenient personal services and making a lot of users comfortable. We have smart cities, smartphones, smart cars, and more. This fast way of life requires. The developments of home automation projects. The smart homes which will automatically close or open windows based on weather conditions outsides. This paper presents an overall review of different authors contribution in smart mirror. A smart mirror is capable to displaying time, date, weather and cloud. The many benefits of using a smart mirror it make life easier as need to look at phones every time we need to check time, weather is also reduced. The smart mirror help in developed smart house with embedded artificial intelligence finding application in industries. We look at the mirror daily and internet with it psychologically to find out how we look and how our attire is. The interactive mirror is develop the mirror with proper embedded intelligence for offering feature such as weather of city, latest updates of news and headlines and local time corresponding to location.

The smart bathroom mirror is a signature detail of futuristic movies and advertisements. Want some visual shorthand for how far technology has come? Show your protagonist brushing her teeth, while over her reflection are laid her daily schedule, weather report, news headlines, incoming messages, and more. So it’s kind of surprising that, even though the technology has existed for some years now, we haven’t seen a version of this product from any of the big names in touchscreen computers or the smart .

Fortunately, that hasn’t stopped the DIY community from forging ahead and building what the market wouldn’t provide.

One of the slickest-looking examples was just posted earlier tin 2019 by Google employee Max Braun. It’s built from a super-thin Asus monitor behind two-way glass, with a Fire TV HDMI streaming stick running a custom Android application. The screen shows widgets like time, weather and headlines, and Braun says it can be expanded to include basically anything that has a Google Now card.

Like the examples below, there’s no touchscreen capability and who would want all those fingerprints greasing up the mirror anyway? but Braun’s idea was to create a UI that presents a quickglance overview of information and doesn’t require much interaction. If he needs to know something that’s not in one of the widgets, the mirror also can access Android’s built-in voice search features. Braun hasn’t yet posted a full how-to with the complete bill of materials, but we estimate a setup like this would cost upwards of $\$ 500$, with the monitor and the glass as the main expenses.

II. WHY SMART MIRROR??

The “Internet of Things” (IoT) has the power to change our world. IoT will play an important role in the future and there is expected to be a significant amount of cash flowing through the market in the up-coming years. Over half of major new business processes and systems will incorporate IoT elements by 2020. The impact on consumers’ lives and corporate business models is rapidly increasing as the cost of instrumenting physical things with sensors and connecting them to other things devices, systems and people continue to drop.

Fig1.Trends in IOT and future of IOT
Rapid changes in IOT technology makes it a challenging task for the most experienced experts to anticipate the fu- ture of standardization in the field. For humanity, which is moderately muddled by nature, the Internet of Things is an extraordinary advancement. On the other hand, for individuals who esteem

their security, the M2M helps in interconnecting different electronic gadgets. Basically, IOT means having each electronic gadget and numerous different things-associated and associating progressively with the Internet by controlling via applications. It can be followed and observed in queries why do we need internet of thing. The thinking goes you’ll have the capacity to sort out your life better by not expecting to pay tedious regard for your life.

• Smart mirrors are straight from science fiction. They’re part of an optimistic vision of the future that imagines a world where screens and data are everywhere, ready to feed you whatever information you need at a moment’s notice.
• What’s cooler than a mirror that shows you handy information before starting your day? Adding voice controls so that you can update your calendar, play Spotify playlists, and so much more. Also, it’s really awesome and futuristic. Impress your friends, family, and pets!
• But it turns out that building your own smart mirror isn’t just science fiction - it’s actually really easy to put one together. So don’t wait around for some random Kickstarter to get its act together and sell you a smart mirror. Here’s how to build one yourself.

RELATED WORK

The Smart Mirror Represents A Natural Interface That Facilities Access To Personalized Services. This Is An Attempt To Contribute To This Design Of A Smart Mirror Like Inter- face As Well As The Smart Environment In Which The Inter- face Is Used For Interaction On The Following. Phillips Home lab Is A Test bed For Creating Perspective And Context-Aware A Home Environments Intelligent Personal Care Environment Uses An Interactive Mirror In The Bathrooms To Provide Per- socialized Care Environment Uses An Interactive Preferences. For Example, Children Can Watch Their Favorite Cartoon While Brushing Their Teeth. The Mirror Can Provide Live TV Feeds Monitor The Latest Weather And Soon. The Mirror Is A Combination Of One Or More LCD Flat Screen Display Specifically Combined With A Mirror Surface And Connected With A Central Processor To Provide The Intended.

SYSTEM ABSTRACT

The mirror is eventually a technologically augmented interaction device. The objective of designing the mirror is to provide a natural interface in the ambient home environment for accessing various services such as location based weather, time, calendar etc. as well as provide access to YouTube, Music, maps etc. The project includes downloading the Raspbian operating system based on Debian and extracting the image on SD
card, inserting the card in the Raspberry Pi SD slot an then performing the required steps.

We plan to deliver a working prototype i.e. design and development of a futuristic Smart Mirror on Raspberry Pi 3 for the ambient home environment as well as for commercial environment

Most people have mirrors at home, so the concept of a smart mirror that you can interact with is attractive and can be fantasized by anyone. At times no one has time to read the newspaper or switch on the TV right in the morning to check the news headlines or the weather forecast. If a mirror serves to this purpose, one can imagine the amount of time it will save and be of such a great use. The device was to look like a regular mirror but would have a screen inside. The project which would collect real world machine data such as location based latest news and headlines, weather reports, and as well as show us the local time. The data would be transmitted from the machine and would be managed in a central database. Authors have also worked on including Artificial Intelligence in the Smart Mirror wherein a Voice enabled assistant will cater to the needs of the user.

Fig. component diagram

A. Image module

The image control module is mainly consist of one way mirror, driver board ,display.
It is the basic system for human and computer interaction.it shows the daily information by accessing the internet.

B. Raspberry pi

Raspi is the microcontroller which is used for smart mirror to perform the opetation.it is the mini module of a computer CPU.raspi helps to perform various operation of CP on small scale.

C. Sensors & H/W

Smart mirror uses various types of seniors/hardware component’s like microphone, speakers, camera module etc.

SOFTWARE DESIGN OF SYSTEM

The relevant property work of development platform should be done prior to software design.
1.use as SD card above class 4 ,because the cards speed directly affects on the performance of raspi.
2.download the system image file for raspi.
3.using eicher write tystem image of SD card.
4.configer the SD card connect the keyboard and mouse& HDMI cable and other hardware devices.
5.install magic mirror framework from GitHub.
6.selcect appropriate modules according to personalized interest.
7.configer the required setting of magic mirror.

SYSTEM OVERVIEW

The system developed by Lin Li gives us a complete control over the services and functions oy voice commands. The system is divided in 3 different subsystems as follows.

1.physical structure.
2.smart mirror framework.
3.cloud services.

1.The physical system of smart mirror contens all the hardware related components and sensors including wodden fream which helps to enclose all the setup and give natural look to system.the hardware components inclide display monitor,raspi,speakers,microphone,one way mirror,camera module,wires etc.
2.smart mirror freamwork includes smart mirror software which was developed usinh python programming language. it contains various modules which handels individual function of system including weather,city,time and date,news feeds,multimedia display.mirror displays white character over a black background for increased visibility from other side of mirror.
3.cloud services:-system supports two cloud service Amazon voice service (AVS - Alexa), Amazon web services (AWS), and Google custom search engine. Amazon voice service is used to provide the system’s voice assistant capabilities in addition to the added Alexa skills.Amazon web services is used to create a special voice instructions from the voice assistant. Finally, Google custom search is used in creating a customized search engine to provide multimedia content based on voice requests by the user.

Fig.amezon alexa
Amazon Alexa Voice Service (AVS): AVS is a platform that allows developers to integrate “Alexa” into their projects. With an AVS enabled product, users can stream media, get updates on weather, traffic, and news, as well as ask general knowledge questions. Alexa is triggered when a user says the wake word, “Alexa”. Once the device detects the wake word, it begins to listen for a request. The request is then relayed to cloud based voice service. It is thus a requirement to have a functioning Internet connection. The cloud-based voice service interprets the request and sends a command back to the device or in this case, the smart mirror system.
custom skills - Skills are commands that Alexa can process. They are essentially applications but tailored specifically for AVS. Alexa provides a wide range of custom skills that are readily available to users of Alexa enabled devices. Perhaps the most impressive capability of AVS is that it allows the creation of custom skills that can be used to improve the richness of user experience. The number of existing custom skills runs into the tens of thousands

Fig. amazon custom skills

Fig. amazon custom skill creation
The author has created a new skill called “smart mirror” This name represents what the user say to indicate that they want to interact with the skill. The skill is created by visiting the web link https://developer.amazon.com/alexa-skills-kit and selecting to create a new Alexa skill and listing the intents for the new skill. Authors have created a skill name Smart mirror and an intent is created called " get Time" and a sample phrase like “tell me the time”. If user says “alexa tell mi the time” alexa voice service will invoke the smart mirror skill and it would also recognize users intension for requesting the skill “getTime”. There are various aspects while considering voice commands like:

A. voice based commands

The system supports interaction between user and system. Users can control all functions using voice commands, including registration, authentication, searching, and signing out. The implementation features three primary technologies: Google’s
"Speech-to-Text [12], Google’s "Custom Search Engine [13], and Amazon’s “Alexa Voice Service (AVS)”.

1) Google Speech-to-Text Conversion:

it enables developers to convert audio to text by applying powerful neural network models in an easy way. The API recognizes 120 languages and variants to support a global user base. It can be used to enable voice command-and-control, and more by processing real-time streaming or pre-recorded audio.

Speech to text is implemented by “wake word detection”,“pass phrase detection” .voice instructions are given to system by using microphone.each phrease is recorded and given to google speech to text conversion.google converts the necessiry command and customized search engine will produce the result.

Fig.google text to speech engine

Wake Word Detection:

Like other voice-controlled systems, authors implements the use of a wake word. A wake word is a word or phrase that is sayed by the user to notify the system that he is ready to give an instruction. The system, upon detection of the wake word, begins to wait for instructions.

SYSTEM DESIGN AND IMPLEMENTATION

1. Facial Recognition: facial recognition is an approach of identifying an individual. There are various facial recognition algorithm, the generally work by comparing selected facial features from given images with face in database. OpenCV library. OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning library. It supports face detection, face training, and face prediction. Face detection is the process of identifying faces in a picture. Face training is the process of studying several pictures of the same person to learn the features specific to the individual. This process creates a database of the known users. Face prediction is the process of identifying a known or pre-registered individual in a new image. With the support of OpenCV, two modules:

Luser registration.

2.user authentication.

- User registration:

User has to register for accesing the service user needs to take pictures of face using raspi camera.if the face is identified in
fream this picture is saved in database.
All the pictures saved are loded into array and passed to recognizer .the recognizer creates an instance of opencv loal binary pattern histogram class. LBP is a method for extracting a useful pattern from an image.this method compares the pixel with neighbor pixels this operation extracts the unique face features from all the pictures and saves them to a model file, which is known as the face database

- User authentication:

When a registered user wants to log into the system, he needs to be authenticated. The authentication procedure includes both face detection and face prediction. The authentication mechanism uses the Raspberry camera to obtain live images of the user. A recognizer object is created. The live images are passed to the recognizer and the prediction method is called. This method compares the image with the model file created earlier and returns the unique id of the user (from the model file) whose facial features are closest to the current image. The prediction model also returns an integer value called the confidence which represents how close the image is to the identified user. 0 means perfect match.

Fig.face authentication

2. Voice Recognition: Voice recognition is a method to Identificat any person on the basis of voice commands . voice commands have a advantage of preventing user data leak.if the facial dta is been stolen anyone can access the system with the help of images. However,voice authentication helps to create a voice impression of the user which is not easy to get stolen. Authors have usec voice recognition function was implemented using third party API of voiceit. voiceit system uses three different phases for recognization.

• capturing
• digitalizing
• methametical model comparison voice recognisation consist of the modules mainly:
• user enrollment
• user authentication

user enrollment: in this module the first step is user creation using ‘create_user’ method of voiceIt API.to enroll user the sytem turns on microphone and user has to say the selected pass phrase .once the phrase is detected it is saves as a wave file and represents the user voiceprint.the voice print is passed to create enrollment method of VoiceIt API to create enrollment of user.overall five enrollment must be done to each user for higher accuracy.
User authentication: when the weak word is said system strats to listen voice phrase of user.the phrease is recored in wave file and passed as aperameter to ‘authenticate user’ method. A respone is send in the form of JSON file indicating wether the authentication is succesfull or not. A response ‘SUC’ represents success of authentication and ‘ATF’ indicates authentication failure.which is determind by a confidance value between 1-100 where 100 means perfect match.

SMART MIRROR MODULES

WEATHER: displays weather and location information. The data is obtained from two APIs, namely openweathermap .

Fig. weather module

TIME & DATE: shows how time and date information is retrieved from the computer system.

Fig.time and event calender
NEWS: This section shows news headlines pulled from news stations. A news feed is displayed every five seconds.

SOS Alarm om Viktigt meddelande via sms

Från den 1 juli kommer du att kunna få VMA, Viktiga meddelanden till allmänheten, även via ett sms om något allvarligt händer i området där du befinner dig.

From: SOS alarm, knisinformation.se, MSSt

Fig. news feeds

TO-DO-LIST:it displays the user shedules and do do list which are costomised by user.

Fig. to-do-list

MULTIMEDIA CONTENCT:

This is used to display multimedia content like videos, music ,images etc.

Fig .Multimedia content module

EVOLUTION METHODS

Series of experiments has been done by authors in order to find appropriate values for some important system variable’s like frame rate, camera resolution, face recognition threshold, voice recognition confidence threshold.

Frame rate refers to no of frame extracted from smart mirror camera and displayed. To find appropriate frame rate and resolution values, the authors enrolled a new user and measured the average confidence achieved for various frame rates and resolution values during authentication.

Fig.Confidence comparison based on difference frame rates
An optimal value, as can be observed across all graphs, would be to select a frame rate of 50 and a resolution of $480 \times 360$. This is the combination that obtains the lowest average confidence value, 25 , while keeping frame processing time low.

confidence value for face recognition:

the authors used a room with a constant lighting and asked 10 candidates to attempt logging into the system. Among the candidates, candidate 1 was preregistered on the system to serve as the control setup. Thus, a lower confidence value for candidate 1 was expected during authentication.

Fig Average confidence levels of 10 candidates on face recognition
candidate 1 achieved the lowest average confidence of 25 . Candidate 2 had the second lowest average confidence of 38 . Finally authors decided, a confidence value of 28 to serve as the required threshold for successful facial authentication of the system.

Confidence for voice recognition: to find confidence value authors turned off facial recognition and 10 different candidates are said to use the system where 5 candidates are registered and 5 are new. All of the candidates knew the pass phrase and each of them attempted five times. The confidence value of each candidate was calculated based on the average confidence prediction of the voice recognition.

Fig. Average confidence levels of 10 candidates on voice recognition
Out of 10 five registered candidates (candidates 6 to 10) have much higher confidence value than that of the unregistered users (candidates 1 to 5). After repeating the experiments among a number of candidates, a confidence value of 83 was selected to serve as the required threshold for successful voice authentication of the system. Based on the threshold values authors have conducted another experiment to verify the authentication accuracy.
Authors have created an account on Voicelt API and created different customized phases and users ask to authenticate system.

Fig. pass phrases of Voicelt API

different users are asked to access the system by saying pass phrases. It includes Registered user and unregistered users each of them have given a 10 attempts to log into system.the experiment shows:

voice authentication has high accuracy level. The unregistered user was never authenticated to log in. As a comparison, except for one time on the first pass phrase, the system always succeeded to recognize the registered user and authorize him to $\log$ in. Similar results were also found through the experiments conducted on other users.

CONCLUSION

This paper is presented in order to review the latest technology trends in smart mirror.in todays trending technology everything has been made smart and getting smarter day by day. The “smart mirror” was introduced in 2016 and day by day it is improving in terms of performance and services. Smart mirror has improved our standards of living by making our life style easier. The smart mirror is getting personalized the main problem come in picture of security. Security is the main function while dealing with personalized IOT platform device hence the authors tried to improve the security of the system by adding facial recognition and voice authentication features in smart mirror in order to full fill the requirements of security of users data. The experimental figures shows the accuracy and the confidence of the system has achieved near about security consurens f the system in todays world it can be considered as a best fitted solution invented now a days.in the future there can be various biometric authentication can be added in the system like fingerprint authentication ect.

This product has great potential mostly in luxury markets due to current high costs. As of today, do-it-yourself electronic hobbyists produce most smart mirrors, aside from a few small companies. From a small survey, I concluded that it is extremely interesting to people and that they would be interested in purchasing one for their own home. Smart mirrors can be produced quite easily depending on how complex one wants to make it. This has given me great motivation to continue building smart mirrors for friends and family…

REFERENCES

1. Lin Li “Voice Controlled Smart Mirror with Multifactor Authentication” lilina@psamu.edu 978-1-5386-59595/18/$31.00 2018 IEEE.
2. Kun Jin “Design of the smart Mirror based on Raspberry Pi” Jinkunhg@163.com 978-1-5386-1803-5/18/$31.00 @IEEE
3. R.P. I. Foundation, “Teach, learn, and make with raspberry pi,” Raspberry Pi. [Online]. Available: https://www.raspberrypi.org/
4. A. Orr, “Use python to build a raspberry pi-powered home security camera,” 2018. [Online]. Available:

https://medium.com/@andyorr/ use-python-to-build-a-raspberry-pi-powered-home-security-camera-tom-50-84ab7e344e2d
5. M. Patkar, “6 best raspberry pi smart mirror projects we’ve seen so far,” 2016. [Online]Available:https://www.makeuseof.com/tag/ 6-best-raspberry-pi-smart-mirror-projects-weve-seen

$\qquad$