Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Headset Design
System Design
Results
Painting Recognition Results
Conclusions
References
All Topics
Chemistry
Analytical Chemistry

Automatic Museum Audio Guide

Profile image of Oscar DenizOscar Deniz

Sensors

https://doi.org/10.3390/S20030779
visibility

…

description

25 pages

link

1 file

Abstract

An automatic “museum audio guide” is presented as a new type of audio guide for museums. The device consists of a headset equipped with a camera that captures exhibit pictures and the eyes of things computer vision device (EoT). The EoT board is capable of recognizing artworks using features from accelerated segment test (FAST) keypoints and a random forest classifier, and is able to be used for an entire day without the need to recharge the batteries. In addition, an application logic has been implemented, which allows for a special highly-efficient behavior upon recognition of the painting. Two different use case scenarios have been implemented. The main testing was performed with a piloting phase in a real world museum. Results show that the system keeps its promises regarding its main benefit, which is simplicity of use and the user’s preference of the proposed system over traditional audioguides.

Related papers

Initial Evaluation of the Potential of Smartphone Stereo-Vision in Museum Visits
lidia fiorini

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019

The recent introduction of new technologies such as augmented reality, machine learning and the worldwide spread of mobile devices provided with imaging, navigation sensors and high computational power can be exploited in order to drammatically change the museum visit experience. Differently from the traditional use of museum docents or audio guides, the introduction of digital technologies already proved to be useful in order to improve the interest of the visitor thanks to the increased interaction and involvement, reached also by means of visual effects and animations. Actually, the availability of 3D representations, augmented reality and navigation abilities directly on the visitor's device can lead to a personalized visit, enabling the visitor to have an experience tailored on his/her needs. In this framework, this paper aims at investigating the potentialities of smartphone stereo-vision to improve the geometric information about the artworks available on the visitor's device. More specifically, in this work smartphone stereo-vision will used as a 3D model generation tool in a 3D artwork recognition system based on a neural network classifier.

View PDFchevron_right
Smart Guide – an approach to the Smart Museum using Android
IRJET Journal
View PDFchevron_right
Museum 360: Automated Guiding System For A Museum
Dhishan Dhammearatchi

2016

During past few years Indoor navigation systems have been developed in many places such as Airports, Hospitals, Shopping malls, Companies and Museums. Museum 360 is a navigation system design specially for Museums. The system is developed based on Bluetooth Low Energy (BLE) Beacons technology instead of using Radio Frequency Identification (RFID), Near Field Communication (NFC) and Wireless Fidelity (Wi-Fi) technologies. Rather than traditional museum navigation systems, the implemented solution has a new narrating system for navigation, provide descriptions about the items verbally, providing suggestions to visitors what to visit by analyzing past history and also visitors can collect a soft copy of the tour. Museum 360 has two main components, Android application and the server. The system was developed according to the prototype life cycle methodology. When the system is completed it will benefits both of the visitors and museum staff. Visitors can be able to explore the museum a...

View PDFchevron_right
PhoneGuide: Adaptive Image Classification for Mobile Museum Guidance
Oliver Bimber

2011 International Symposium on Ubiquitous Virtual Reality, 2011

This paper summarizes the various components of our mobile museum guidance system PhoneGuide. It explains how practically viable object recognition rates can be achieved under realistic conditions using adaptive image classification.

View PDFchevron_right
A Ubiquitous Interactive Museum Guide
Yo-Ping Huang

Lecture Notes in Computer Science, 2008

This paper proposes a PDA-based guide and recommendation system. The intelligent guide provides an alternative to the current audio-based guide tools. Techniques from both data mining and extension theory and RFID technology are deployed to provide better interaction between exhibitions and visitors.

View PDFchevron_right
Understanding art exhibitions: from audioguides to multimedia companions
Augusto Celentano

DMS 2009, Int. Conf. on …

View PDFchevron_right
Rfid-Based Audio and Textual Guide for Museums
Cesar Llorente

2013

Deployment of technologies that alleviates the effects of human fatigue and monotony can enhance the experience of visitors in the museums. Audio recordings can be played back repeatedly without deterioration brought about by fatigue and boredom. Such technologies based on Radio Frequency ID tags can be used to identify or tag an artifact. A recording or multimedia presentation can be associated to a particular tag. This presentation can be activated or played back in a portable device when it detects the tag. The portable device is carried by the tour guide. In this study, a portable device is developed to scan or detect an RF tag and play back an audio recording describing the artifact. It utilizes the NIOSII softcore processor and implemented on Altera Corporation's DE2 FPGA Development Board using Quatus II SOPC Builder. Tests showed that the system is able to detect the tag placed in the artifact and played back the corresponding wav audio recording with 100% accuracy.

View PDFchevron_right
Artwork Identification in a Museum Environment: A Quantitative Evaluation of Factors Affecting Identification Accuracy
Zenonas Theodosiou

Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection, 2021

The ability to identify the artworks that a museum visitor is looking at, using first-person images seamlessly captured by wearable cameras can be used as a means for invoking applications that provide information about the exhibits, and provide information about visitors' activities. As part of our efforts to optimize the artwork recognition accuracy of an artwork identification system under development, an investigation aiming to determine the effect of different conditions on the artwork recognition accuracy in a gallery/exhibition environment is presented. Through the controlled introduction of different distractors in a virtual museum environment, it is feasible to assess the effect on the recognition performance of different conditions. The results of the experiment are important for improving the robustness of artwork recognition systems, and at the same time the conclusions of this work can provide specific guidelines to curators, museum professionals and visitors, that will enable the efficient identification of artworks, using images captured with wearable cameras in a museum environment.

View PDFchevron_right
Adaptive Multimodal Perception for a Virtual Museum Guide
Martin Luerssen

… '08), 18

View PDFchevron_right
Mobile Phone-Enabled Museum Guidance with Adaptive Classification
Oliver Bimber

IEEE Computer Graphics and Applications, 2008

We present an adaptive museum guidance system called PhoneGuide. It uses camera-equipped mobile phones for on-device object recognition in ad-hoc sensor networks and provides location and object aware multimedia content to museum visitors.

View PDFchevron_right

References (66)

  1. Hu, F. Classification and Regression Trees, 1st ed.; CRC Press: Boca Raton, FL, USA, 2013, doi:10.1201/b15552.
  2. Commission, E. Report from the Workshop on Cyber-Physical Systems: Uplifting Europe's Innovation Capacity. 2013. Available online: https://ec.europa.eu/digital-single-market/en/news/report-workshop- cyber-physical-systems-uplifting-europe's-innovation-capacity (accessed on 31 January 2020).
  3. Szeliski, R. Computer Vision: Algorithms and Applications; Springer Science & Business Media: London, UK, 2010.
  4. Belbachir, A.N. Smart Cameras; Springer: London, UK, 2010.
  5. BDTI. Implementing Vision Capabilities in Embedded Systems. Available online: https://www.bdti.com/ MyBDTI/pubs/BDTI_ESC_Boston_Embedded_Vision.pdf (accessed on 31 January 2020).
  6. Kisačanin, B.; Bhattacharyya, S.S.; Chai, S. Embedded Computer Vision; Springer International Publishing: Cham, Switzerland, 2009.
  7. Bailey, D. Design for Embedded Image Processing on FPGAs; John Wiley & Sons Asia Pte Ltd.: Singapore, 2011.
  8. Akyildiz, I.F.; Melodia, T.; Chowdhury, K.R. A survey on wireless multimedia sensor networks. Comput. Netw. 2007, 51, 921-960. [CrossRef]
  9. Farooq, M.O.; Kunz, T. Wireless multimedia sensor networks testbeds and state-of-the-art hardware: A survey. In Communication and Networking, Proceedings of the International Conference on Future Generation Communication and Networking, Jeju Island, Korea, 8-10 December 2011; Springer: Berlin, Heidelberg, Germany, 2011, pp. 1-14.
  10. Almalkawi, I.T.; Guerrero Zapata, M.; Al-Karaki, J.N.; Morillo-Pozo, J. Wireless multimedia sensor networks: Current trends and future directions. Sensors 2010, 10, 6662-6717. [CrossRef] [PubMed]
  11. Soro, S.; Heinzelman, W. A Survey of Visual Sensor Networks. Available online: https://www.hindawi. com/journals/am/2009/640386/ (accessed on 30 January 2020).
  12. Fernández-Berni, J.; Carmona-Galán, R.; Rodríguez-Vázquez, Á. Vision-enabled WSN nodes: State of the art. In Low-Power Smart Imagers for Vision-Enabled Sensor Networks; Springer: Berlin/Heidelberg, Germany, 2012; pp. 5-20.
  13. Tavli, B.; Bicakci, K.; Zilan, R.; Barcelo-Ordinas, J.M. A survey of visual sensor network platforms. Multimedia Tools Appl. 2012, 60, 689-726. [CrossRef]
  14. Chen, P.; Ahammad, P.; Boyer, C.; Huang, S.I.; Lin, L.; Lobaton, E.; Meingast, M.; Oh, S.; Wang, S.; Yan, P.; et al. CITRIC: A low-bandwidth wireless camera network platform. In Proceedings of the 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras, Stanford, CA, USA, 7-11 September 2008; pp. 1-10.
  15. Hengstler, S.; Prashanth, D.; Fong, S.; Aghajan, H. MeshEye: A hybrid-resolution smart camera mote for applications in distributed intelligent surveillance. In Proceedings of the 6th International Conference on Information Processing in Sensor Networks, Cambridge, MA, USA, 25-27 April 2007; pp. 360-369.
  16. Kerhet, A.; Magno, M.; Leonardi, F.; Boni, A.; Benini, L. A low-power wireless video sensor node for distributed object detection. J. Real-Time Image Process. 2007, 2, 331-342. [CrossRef]
  17. Kleihorst, R.; Abbo, A.; Schueler, B.; Danilin, A. Camera mote with a high-performance parallel processor for real-time frame-based video processing. In Proceedings of the 2007 IEEE Conference on Advanced Video and Signal Based Surveillance, London, UK, 5-7 September 2007; pp. 69-74.
  18. Feng, W.C.; Code, B.; Kaiser, E.; Shea, M.; Feng, W.; Bavoil, L. Panoptes: A scalable architecture for video sensor networking applications. ACM Multimedia 2003, 1, 151-167.
  19. Boice, J.; Lu, X.; Margi, C.; Stanek, G.; Zhang, G.; Manduchi, R.; Obraczka, K. Meerkats: A Power-Aware, Self-Managing Wireless Camera Network For Wide Area Monitoring. Available online: http://users.soe. ucsc.edu/~manduchi/papers/meerkats-dsc06-final.pdf (accessed on 30 January 2020).
  20. Murovec, B.; Perš, J.; Kenk, V.S.; Kovačič, S. Towards commoditized smart-camera design. J. Syst. Archit. 2013, 59, 847-858. [CrossRef]
  21. Qualcomm, Snapdragon. Available online: http://www.qualcomm.com/snapdragon (accessed on 30 January 2020).
  22. Deniz, O. EoT Project. Available online: http://eyesofthings.eu (accessed on 30 January 2020).
  23. Deniz, O.; Vallez, N.; Espinosa-Aranda, J.L.; Rico-Saavedra, J.M.; Parra-Patino, J.; Bueno, G.; Moloney, D.; Dehghani, A.; Dunne, A.; Pagani, A.; et al. Eyes of Things. Sensors 2017, 17, 1173. doi:10.3390/s17051173. [CrossRef] [PubMed]
  24. Wacker, P.; Kreutz, K.; Heller, F.; Borchers, J.O. Maps and Location: Acceptance of Modern Interaction Techniques for Audio Guides. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7-12 May 2016; pp. 1067-1071.
  25. Kenteris, M.; Gavalas, D.; Economou, D. Electronic mobile guides: A survey. Pers. Ubiquitous Comput. 2011, 15, 97-111. [CrossRef]
  26. Abowd, G.D.; Atkeson, C.G.; Hong, J.; Long, S.; Kooper, R.; Pinkerton, M. Cyberguide: A mobile context-aware tour guide. Wireless Netw. 1997, 3, 421-433. [CrossRef]
  27. Kim, D.; Seo, D.; Yoo, B.; Ko, H. Development and Evaluation of Mobile Tour Guide Using Wearable and Hand-Held Devices. In Proceedings of the International Conference on Human-Computer Interaction, Toronto, ON, Canada, 17-22 July 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 285-296.
  28. Sikora, M.; Russo, M.; Ðerek, J.; Jurčević, A. Soundscape of an Archaeological Site Recreated with Audio Augmented Reality. ACM Trans. Multimedia Comput. Commu. Appl. 2018, 14, 74. [CrossRef]
  29. Lee, G.A.; Dünser, A.; Kim, S.; Billinghurst, M. CityViewAR: A mobile outdoor AR application for city visualization. In Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (ISMAR-AMH), Altanta, GA, USA, 5-8 November 2012; pp. 57-64.
  30. DüNser, A.; Billinghurst, M.; Wen, J.; Lehtinen, V.; Nurminen, A. Exploring the use of handheld AR for outdoor navigation. Comput. Graphics 2012, 36, 1084-1095. [CrossRef]
  31. Baldauf, M.; Fröhlich, P.; Hutter, S. KIBITZER: A wearable system for eye-gaze-based mobile urban exploration. In Proceedings of the 1st Augmented Human International Conference, Megève, France, 2-3 April 2010; pp. 1-5.
  32. Szymczak, D.; Rassmus-Gröhn, K.; Magnusson, C.; Hedvall, P.O. A real-world study of an audio-tactile tourist guide. In Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, San Francsico, CA, USA, 21-24 September 2012, pp. 335-344.
  33. Lim, J.H.; Li, Y.; You, Y.; Chevallet, J.P. Scene Recognition with Camera Phones for Tourist Information Access. In Proceedings of the 2007 IEEE International Conference on Multimedia and Expo, Beijing, China, 2-5 July 2007; pp. 100-103.
  34. Skoryukina, N.; Nikolaev, D.P.; Arlazarov, V.V. 2D art recognition in uncontrolled conditions using one-shot learning. In Proceedings of the International Conference on Machine Vision, Amsterdam, The Netherlands, 1 October 2019; p. 110412E.
  35. Fasel, B.; Gool, L.V. Interactive Museum Guide: Accurate Retrieval of Object Descriptions. In Adaptive Multimedia Retrieval; Springer: Berlin/Heidelberg, Germany, 2006; pp. 179-191.
  36. Temmermans, F.; Jansen, B.; Deklerck, R.; Schelkens, P.; Cornelis, J. The mobile Museum guide: Artwork recognition with eigenpaintings and SURF. In Proceedings of the 12th International Workshop on Image Analysis for Multimedia Interactive Services, Delft, The Netherlands, 13-15 April 2011.
  37. Greci, L. An Augmented Reality Guide for Religious Museum. In Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Lecce, Italy, 15-18 June 2016; Springer: Berlin/Heidelberg, Germany, 2016, pp. 280-289.
  38. Raptis, G.E.; Katsini, C.P.; Chrysikos, T. CHISTA: Cultural Heritage Information Storage and reTrieval Application. In Proceedings of the 6th EuroMed Conference, Nicosia, Cyprus, 29 October-3 November 2018; pp. 163-170.
  39. Ali, S.; Koleva, B.; Bedwell, B.; Benford, S. Deepening Visitor Engagement with Museum Exhibits through Hand-crafted Visual Markers. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18), Hong Kong, China, 9-13 June 2018; pp. 523-534.
  40. Ng, K.H.; Huang, H.; O'Malley, C. Treasure codes: Augmenting learning from physical museum exhibits through treasure hunting. Pers. Ubiquitous Comput. 2018, 22, 739-750. [CrossRef]
  41. Wein, L. Visual recognition in museum guide apps: Do visitors want it? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada, 26 April 2014; pp. 635-638.
  42. Ruf, B.; Kokiopoulou, E.; Detyniecki, M. Mobile Museum Guide Based on Fast SIFT Recognition. Adaptive Multimedia Retrieval. Identifying, Summarizing, and Recommending Image and Music; Detyniecki, M., Leiner, U., Nürnberger, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 170-183.
  43. Serubugo, S.; Skantarova, D.; Nielsen, L.; Kraus, M. Comparison of Wearable Optical See-through and Handheld Devices as Platform for an Augmented Reality Museum Guide. In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; SCITEPRESS Digital Library: Berlin, Germany, 2017; pp. 179-186. doi:10.5220/0006093901790186. [CrossRef]
  44. Altwaijry, H.; Moghimi, M.; Belongie, S. Recognizing locations with google glass: A case study. In Proceedings of the IEEE winter conference on applications of computer vision, Steamboat Springs, CO, USA, 24-26 March 2014; pp. 167-174.
  45. Yanai, K.; Tanno, R.; Okamoto, K. Efficient mobile implementation of a cnn-based object recognition system. In Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, The Netherlands, October 2016; pp. 362-366.
  46. Seidenari, L.; Baecchi, C.; Uricchio, T.; Ferracani, A.; Bertini, M.; Bimbo, A.D. Deep artwork detection and retrieval for automatic context-aware audio guides. ACM Trans. Multimedia Comput. Commun. Appl. 2017, 13, 35. [CrossRef]
  47. Seidenari, L.; Baecchi, C.; Uricchio, T.; Ferracani, A.; Bertini, M.; Del Bimbo, A. Wearable systems for improving tourist experience. In Multimodal Behavior Analysis in the Wild; Elsevier: Amsterdam, The Netherlands, 2019; pp. 171-197.
  48. Crystalsound Audio Guide. Available online: https://crystal-sound.com/en/audio-guide (accessed on 30 January 2020).
  49. Locatify. Available online: https://locatify.com/ (accessed on 30 January 2020).
  50. Copernicus Guide. Available online: http://www.copernicus-guide.com/en/index-museum.html (accessed on 30 January 2020).
  51. xamoom Museum Guide. Available online: https://xamoom.com/museum/ (accessed on 30 January 2020).
  52. Orpheo Touch. Available online: https://orpheogroup.com/us/products/visioguide/orpheo-touch (accessed on 30 January 2020).
  53. Headphone Weight. Available online: https://www.headphonezone.in/pages/headphone-weight (accessed on 30 January 2020).
  54. OASIS Standards-MQTT v3.1.1. Available online: https://www.oasis-open.org/standards (accessed on 30 January 2020).
  55. Espinosa-Aranda, J.L.; Vállez, N.; Sanchez-Bueno, C.; Aguado-Araujo, D.; García, G.B.; Déniz-Suárez, O. Pulga, a tiny open-source MQTT broker for flexible and secure IoT deployments. In Proceedings of the 2015 IEEE Conference on Communications and Network Security (CNS), Florence, Italy, 28-30 September 2015, pp. 690-694.
  56. Monteiro, D.M.; Rodrigues, J.J.P.C.; Lloret, J. A secure NFC application for credit transfer among mobile phones. In Proceedings of the 2012 International Conference on Computer, Information and Telecommunication Systems (CITS), Amman, Jordan, 13 May 2012; pp. 1-5. doi:10.1109/CITS.2012.6220369. [CrossRef]
  57. Lepetit, V.; Pilet, J.; Fua, P. Point matching as a classification problem for fast and robust object pose estimation. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 27 June-2 July 2004; Volume 2, pp. II-244-II-250. doi:10.1109/CVPR.2004.1315170.
  58. Espinosa-Aranda, J.; Vallez, N.; Rico-Saavedra, J.; Parra-Patino, J.; Bueno, G.; Sorci, M.; Moloney, D.; Pena, D.; Deniz, O. Smart Doll: Emotion Recognition Using Embedded Deep Learning. Symmetry 2018, 10, 387. doi:10.3390/sym10090387. [CrossRef]
  59. Sanderson, C.; Paliwal, K.K. Fast features for face authentication under illumination direction changes. Pattern Recognit. Lett. 2003, 24, 2409-2419. doi:10.1016/S0167-8655(03)00070-9. [CrossRef]
  60. Rosten, E.; Porter, R.; Drummond, T. Faster and better: A machine learning approach to corner detection. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 32, 105-119. [CrossRef] [PubMed]
  61. Tareen, S.A.K.; Saleem, Z. A comparative analysis of sift, surf, kaze, akaze, orb, and brisk. In Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 30 November 2018; pp. 1-10.
  62. Svetnik, V.; Liaw, A.; Tong, C.; Culberson, J.C.; Sheridan, R.P.; Feuston, B.P. Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. J. Chem. Inf. Comput. Sci. 2003, 43, 1947-1958. doi:10.1021/ci034160g. [CrossRef] [PubMed]
  63. Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees; Wadsworth and Brooks: Monterey, CA, USA, 1984.
  64. Bosch, A.; Zisserman, A.; Munoz, X. Image classification using random forests and ferns. In Proceedings of the 2007 IEEE 11th international conference on computer vision, Rio de Janeiro, Brazil, 14-20 October 2007; pp. 1-8.
  65. Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 1981, 24, 381-395. doi:10.1145/358669.358692. [CrossRef]
  66. Nvidia Developer Blogs: NVIDIA R Jetson TM TX1 Supercomputer-on-Module Drives Next Wave of Autonomous Machines. Available online: https://devblogs.nvidia.com/ (accessed on 30 January 2020).

Related papers

Interactive museum guide: Fast and robust recognition of museum objects
Beat Fasel

2006

In this paper, we describe the application of the novel SURF (Speeded Up Robust Features) algorithm [1] for the recognition of objects of art. For this purpose, we developed a prototype of a mobile interactive museum guide consisting of a tablet PC that features a touchscreen and a webcam. This guide recognises objects in museums based on images taken by the visitor. Using different image sets of real museum objects, we demonstrate that both the object recognition performance as well as the speed of the SURF algorithm surpasses the results obtained with SIFT, its main contender.

View PDFchevron_right
Smart Assistant for Museum's Objects Navigation (SAMsON)
Aditya Arie Nugraha, Ary Setijadi Prihatmanto

In this paper, we propose the idea of Smart Assistant for Museum’s Objects Navigation (SAMsON). SAMsON is a mobile multimodal spoken dialogue system that will be used by museum’s visitors for exploring the museum and learning about its collection. SAMsON has three main features: (1) Collection Browser, (2) Object Recognizer, and (3) Virtual Guide. Collection Browser provides information about the museum, its floor maps, and its collections. Visitors also can search certain object using the browser that will give the visitor a brief description of the object and inform about its location in the museum. Object Recognizer is used when visitors want to know the detail of certain object. Visitor only need to capture the image of the object and SAMsON will provides information on the mobile device screen using augmented reality (AR) technology. Both Collection Browser and Object Recognizer are also use speech, beside graphical user-interface (GUI), to deliver the information to visitors. ...

View PDFchevron_right
A Smart Assistant for Visual Recognition of Painted Scenes
Roberto Giaconia

2021

Nowadays, smart devices allow people to easily interact with the surrounding environment thanks to existing communication infrastructures, i.e., 3G/4G/5G or WiFi. In the context of a smart museum, data shared by visitors can be used to provide innovative services aimed to improve their cultural experience. In this paper, we consider as case study the painted wooden ceiling of the Sala Magna of Palazzo Chiaramonte in Palermo, Italy and we present an intelligent system that visitors can use to automatically get a description of the scenes they are interested in by simply pointing their smartphones to them. As compared to traditional applications, this system completely eliminates the need for indoor positioning technologies, which are unfeasible in many scenarios as they can only be employed when museum items are physically distinguishable. Experimental analysis aimed to evaluate the performance of the system in terms of accuracy of the recognition process, and the obtained results sh...

View PDFchevron_right
Deep Artwork Detection and Retrieval for Automatic Context-Aware Audio Guides
Andrea Ferracani

ACM Transactions on Multimedia Computing, Communications, and Applications, 2017

In this article, we address the problem of creating a smart audio guide that adapts to the actions and interests of museum visitors. As an autonomous agent, our guide perceives the context and is able to interact with users in an appropriate fashion. To do so, it understands what the visitor is looking at, if the visitor is moving inside the museum hall, or if he or she is talking with a friend. The guide performs automatic recognition of artworks, and it provides configurable interface features to improve the user experience and the fruition of multimedia materials through semi-automatic interaction. Our smart audio guide is backed by a computer vision system capable of working in real time on a mobile device, coupled with audio and motion sensors. We propose the use of a compact Convolutional Neural Network (CNN) that performs object classification and localization. Using the same CNN features computed for these tasks, we perform also robust artwork recognition. To improve the rec...

View PDFchevron_right
Interactive museum guide
Beat Fasel

2005

In this paper, we describe the prototype of an interactive museum guide. It runs on a tablet PC that features a touchscreen, a webcam and a Bluetooth receiver. This guide recognises objects on display in museums based on images of the latter which are taken directly by the visitor. Furthermore, the computer can determine the visitor's location by receiving signals emitted from Bluetooth senders in the museum, so called BTnodes. This information is used to reduce the search space for the extraction of relevant objects. Hence, the recognition accuracy is increased and the search time reduced. Moreover, this information can be used to indicate the user's current location in the museum. The prototype has been demonstrated to visitors of the Swiss National Museum in Zurich.

View PDFchevron_right

Related topics

  • Analytical Chemistry
  • Electrical and Electronic Engine...

    • Cited by

    VISCOUNTH: A Large-Scale Multilingual Visual Question Answering Dataset for Cultural Heritage
    Ludovica Marinucci, Luana Bulla

    ACM Transactions on Multimedia Computing, Communications, and Applications, 2023

    Visual question answering has recently been settled as a fundamental multi-modal reasoning task of artificial intelligence that allows users to get information about visual content by asking questions in natural language. In the cultural heritage domain this task can contribute to assist visitors in museums and cultural sites, thus increasing engagement. However, the development of visual question answering models for cultural heritage is prevented by the lack of suitable large-scale datasets. To meet this demand, we built a large-scale heterogeneous and multilingual (Italian and English) dataset for cultural heritage that comprises approximately 500K Italian cultural assets and 6.5M question-answer pairs. We propose a novel formulation of the task that requires reasoning over both the visual content and an associated natural language description, and present baselines for this task. Results show that the current state of the art is reasonably effective, but still far from satisfactory, therefore further research is this area is recommended. Nonetheless, we also present a holistic baseline to address visual and contextual questions and foster future research on the topic.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved