Academia.eduAcademia.edu

Outline

Ontology For Multimedia Applications

Abstract

This paper provides an overview of the contents of a tutorial on the subject by one of the authors at WI-2013 Conference. The domination of multimedia contents on the web in recent times has motivated research in their semantic analysis. This tutorial aims to provide a critical overview of the technology, and focuses on application of ontologies for multimedia applications. It establishes the need for a fundamentally different approach for a representation and reasoning scheme with ontologies for semantic interpretation of multimedia contents. It introduces a new ontology representation scheme that enables reasoning with uncertain media properties of concepts in a domain context and a language "Multimedia Web Ontology Language" (MOWL) to support the representation scheme. We discuss the approaches to semantic modeling and ontology learning with specific reference to the probabilistic framework of MOWL. We present a couple of illustrative application examples. Further, we discuss the issues of distributed multimedia information systems and how the new ontology representation scheme can create semantic interoperability across heterogeneous multimedia data sources.

References (49)

  1. Pingdom, "Internet 2012 in numbers," http://royal.pingdom.com/2013/01/16/internet-2012- in-numbers/, 2013.
  2. H. Wang, L.-T. Chia, and S. Liu, "Image retrieval ++-web image retrieval with an enhanced multi- modality ontology," Multimedia Tools Appl., vol. 39, no. 2, pp. 189-215, Sep. 2008. [Online]. Available: http://dx.doi.org/10.1007/s11042-008-0202-7
  3. M. Rabbath, P. Sandhaus, and S. Boll, "Automatic creation of photo books from stories in social media," ACM Trans. Multimedia Comput. Commun. Appl., vol. 7S, no. 1, pp. 27:1-27:18, Nov. 2011. [Online]. Available: http://doi.acm.org/10.1145/2037676.2037684
  4. O. Conlan, I. O'Keeffe, and S. Tallon, "Combining adap- tive hypermedia techniques and ontology reasoning to produce dynamic personalized news services," in Proceed- ings of the 4th international conference on Adaptive Hyperme- dia and Adaptive Web-Based Systems, ser. AH'06. Berlin, Heidelberg: Springer-Verlag, 2006, pp. 81-90.
  5. R. Bansal, R. Kumaran, D. Mahajan, A. Khurdiya, L. Dey, and H. Ghosh, "Twipix: a web magazine curated from social media," in Proceedings of the 20th ACM international conference on Multimedia, ser. MM '12. New York, NY, USA: ACM, 2012, pp. 1355-1356. [Online]. Available: http://doi.acm.org/10.1145/2393347.2396482
  6. M. Hatala, L. Kalantari, R. Wakkary, and K. Newby, "Ontology and rule based retrieval of sound objects in augmented audio reality system for museum visitors," in Proceedings of the 2004 ACM symposium on Applied computing, ser. SAC '04. New York, NY, USA: ACM, 2004, pp. 1045-1050.
  7. A. Mallik, S. Chaudhury, and H. Ghosh, "Nrityakosha: Preserving the intangible heritage of indian classical dance," J. Comput. Cult. Herit., vol. 4, no. 3, pp. 11:1-11:25, Dec. 2011. [Online]. Available: http://doi.acm.org/10.1145/2069276.2069280
  8. A. Mallik, H. Ghosh, S. Chaudhury, and G. Harit, "Mowl: An ontology representation language for web based mul- timedia applications," ACM Transactions on Multimedia Computing, Communications and Applications, (In press).
  9. T. Berners-Lee, J. Hendler, and O. Lassila, "The semantic web," Scientific American, May 2001.
  10. T. R. Gruber, "A translation approach to portable ontolo- gies," Knowledge Acquisition, vol. 5, no. 2, pp. 199-220, 1993.
  11. F. van Harmelen, V. Lifschitz, and B. Porter, Eds., Hand- book of Knowledge Representation. Elsevier, 2008.
  12. "Owl web ontology language guide," http://www.w3.org/TR/owl-guide/, 2004.
  13. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, "Query by image and video content: The qbic system," Computer, vol. 28, no. 9, pp. 23-32, Sep. 1995. [Online]. Available: http://dx.doi.org/10.1109/2.410146
  14. J. R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. Jain, and C. fe Shu, "The virage image search engine: An open framework for image manage- ment," in Proceedings of the SPIE Conference on Visual communication and Image Processing, 1996, pp. 76-87.
  15. Y. Liu, D. Zhang, G. Lu, and W.-Y. Ma, "A survey of content-based image retrieval with high-level semantics," Pattern Recogn., vol. 40, no. 1, pp. 262-282, Jan. 2007. [Online]. Available: http://dx.doi.org/10.1016/j.patcog.2006.04.045
  16. J. Sivic, "Efcient visual search of videos cast as text retrieval," IEEE Trans. on PAMI, vol. 31, no. 4, pp. 591- 605, April 2009.
  17. J. Fan, Y. Gao, H. Luo, and G. Xu, "Statistical modeling and conceptualization of natural images," Pattern Recogn., vol. 38, no. 6, pp. 865-885, Jun. 2005. [Online]. Available: http://dx.doi.org/10.1016/j.patcog.2004.07.011
  18. B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas, and L. Fei-Fei, "Human action recognition by learning bases of action attributes and parts," in Proceedings of the 2011 International Conference on Computer Vision, ser. ICCV '11. Washington, DC, USA: IEEE Computer Society, 2011, pp. 1331-1338. [Online]. Available: http://dx.doi.org/10.1109/ICCV.2011.6126386
  19. M. Doerr, "The cidoc conceptual reference mod- ule: an ontological approach to semantic in- teroperability of metadata," AI Mag., vol. 24, no. 3, pp. 75-92, Sep. 2003. [Online]. Available: http://dl.acm.org/citation.cfm?id=958671.958678
  20. A. Gangemi, N. Guarino, C. Masolo, A. Oltramari, and L. Schneider, "Sweetening ontologies with dolce," in Proceedings of the 13th International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, ser. EKAW '02. London, UK, UK: Springer-Verlag, 2002, pp. 166-181. [Online]. Available: http://dl.acm.org/citation.cfm?id=645362.650863
  21. H. Xu, X. Zhou, M. Wang, Y. Xiang, and B. Shi, "Exploring flickr's related tags for semantic annotation of web images," in of the ACM International Conference on Image and Video Retrieval, ser. CIVR '09. New York, NY, USA: ACM, 2009, pp. 46:1-46:8. [Online]. Available: http://doi.acm.org/10.1145/1646396.1646450
  22. F. Nack, J. v. Ossenbruggen, and L. Hardman, "That obscure object of desire: Multimedia metadata on the web, part 2," IEEE MultiMedia, vol. 12, no. 1, pp. 54-63, Jan. 2005. [Online]. Available: http://dx.doi.org/10.1109/MMUL.2005.12
  23. J. R. Smith, "Largescale concept ontology for multime- dia," IEEE Multimedia Magazine, vol. 13, no. 3, pp. 86-91, July-September 2006.
  24. S. Dasiopoulou, V. Tzouvaras, I. Kompatsiaris, and M. G. Strintzis, "Enquiring mpeg-7 based multimedia ontologies," Multimedia Tools Appl., vol. 46, no. 2-3, pp. 331-370, Jan. 2010. [Online]. Available: http://dx.doi.org/10.1007/s11042-009-0387-4
  25. C. Tsinaraki, P. Polydoros, and S. Christodoulakis, "Interoperability support between mpeg-7/21 and owl in ds-mirf," IEEE Trans. on Knowl. and Data Eng., vol. 19, no. 2, pp. 219-232, Feb. 2007. [Online]. Available: http://dx.doi.org/10.1109/TKDE.2007.33
  26. J. Hunter, "Enhancing the semantic interoperability of multimedia through a core ontology," IEEE Trans. Cir. and Sys. for Video Technol., vol. 13, no. 1, pp. 49-58, Jan. 2003. [Online]. Available: http://dx.doi.org/10.1109/TCSVT.2002.808088
  27. R. Arndt, R. Troncy, S. Staab, L. Hardman, and M. Vacura, "Comm: designing a well-founded multimedia ontology for the web," in Proceedings of the 6th international The semantic web and 2nd Asian conference on Asian semantic web conference, ser. ISWC'07/ASWC'07. Berlin, Heidelberg: Springer-Verlag, 2007, pp. 30-43. [Online]. Available: http://dl.acm.org/citation.cfm?id=1785162.1785166
  28. K. Dalakleidi, S. Dasiopoulou, G. Stoilos, V. Tzouvaras, G. Stamou, and Y. Kompatsiaris, "Knowledge-driven multimedia information extraction and ontology evolution," in Knowledge-Driven Multimedia Information Extraction and Ontology Evolution, G. Paliouras, C. D. Spyropoulos, and G. Tsatsaronis, Eds. Berlin, Heidelberg: Springer-Verlag, 2011, ch. Semantic representation of multimedia content, pp. 18-49. [Online]. Available: http://dl.acm.org/citation.cfm?id=2001069.2001071
  29. M. Bertini, A. D. Bimbo, G. Serra, C. Torniai, R. Cucchiara, C. Grana, and R. Vezzani, "Dynamic pictorially enriched ontologies for digital video libraries," IEEE MultiMedia, vol. 16, no. 2, pp. 42-51, Apr. 2009. [Online]. Available: http://dx.doi.org/10.1109/MMUL.2009.25
  30. H. Kangassalo, "Conceptual level user interfaces to data bases and information systems," in Advances in information modelling and knowledge bases, H. Jaakkola, H. Kangassalo, and S. Ohsuga, Eds. IOS Press, 1991, pp. 66-90.
  31. S. S. Wattamwar and H. Ghosh, "Spatio-temporal query for multimedia databases," in Proceedings of the 2nd ACM workshop on Multimedia semantics, ser. MS '08. New York, NY, USA: ACM, 2008, pp. 48-55. [Online]. Available: http://doi.acm.org/10.1145/1460676.1460686
  32. F. Wu and D. S. Weld, "Automatically refining the wikipedia infobox ontology," in Proceedings of the 17th international conference on World Wide Web, ser. WWW '08. New York, NY, USA: ACM, 2008, pp. 635-644. [Online].
  33. Available: http://doi.acm.org/10.1145/1367497.1367583
  34. H. Ghosh, P. Poornachander, A. Mallik, and S. Chaudhury, "Learning ontology for personalized video retrieval," in Workshop on multimedia information retrieval on The many faces of multimedia semantics, ser. MS '07. New York, NY, USA: ACM, 2007, pp. 39-46. [Online]. Available: http://doi.acm.org/10.1145/1290067.1290075
  35. A. Mallik and S. Chaudhury, "Acquisition of multimedia ontology: an application in preservation of cultural her- itage," IJMIR, vol. 1, no. 4, pp. 249-262, 2012.
  36. L. Schneider, "Designing foundational ontologies -the object-centered high-level reference ontology ochre as a case study," in in Conceptual Modeling 2003, 22nd Interna- tional Conference on Conceptual Modeling, I.-Y. Springer, 2003, pp. 91-104.
  37. S. Liu, J. Feng, Z. Song, T. Zhang, H. Lu, C. Xu, and S. Yan, "Hi, magic closet, tell me what to wear!" in Proceedings the 20th ACM international conference on Multimedia, ser. MM '12. New York, NY, USA: ACM, 2012, pp. 619-628.
  38. D. Vogiatzis, D. Pierrakos, G. Paliouras, S. Jenkyn-Jones, and B. J. H. H. A. Possen, "Expert and community based style advice," Expert Systems with Applications: An Interna- tional Journal, vol. 39, no. 12, pp. 10 647-10 655, September 2012.
  39. S. Ajmani, H. Ghosh, A. Mallik, and S. Chaudhury, "An ontology based personalized garment recommendation system," in Workshop on Personalization, Recommender Sys- tems and Social Media, November 2013.
  40. B. Kentner, Color me a season: A complete guide to finding your best colors and how to use them. Ken Kra Publishers, 1979.
  41. Y. T. F. Yueh, D. K. W. Chiu, H.-f. Leung, and P. C. K. Hung, "A virtual travel agent system for m-tourism with semantic web service based design and implementation," in Proceedings of the 21st International Conference on Advanced Networking and Applications, ser. AINA '07. Washington, DC, USA: IEEE Computer Society, 2007, pp. [Online]. Available: http://dx.doi.org/10.1109/AINA.2007.25
  42. H. Kosch, R. Slota, L. B ösz örményi, J. Kitowski, J. Otfinowski, and P. W ójcik, "A distributed medical information system for multimedia data -the first year's experience of the parmed project," in Proceedings of the 8th International Conference on High-Performance Computing and Networking, ser. HPCN Europe 2000. London, UK, UK: Springer-Verlag, 2000, pp. 543-546. [Online]. Available: http://dl.acm.org/citation.cfm?id=645564.658465
  43. X. Pan, T. Schiffer, M. Schr öttner, R. Berndt, M. Hecher, S. Havemann, and D. W. Fellner, "An enhanced dis- tributed repository for working with 3d assets in cultural heritage," in Proceedings of the 4th international conference on Progress in Cultural Heritage Preservation, ser. EuroMed'12. Berlin, Heidelberg: Springer-Verlag, 2012, pp. 349-358.
  44. E. Shakshuki, M. Kamel, and H. Ghenniwa, "A multi-agent system architecture for information gathering," in Proceedings of the 11th International Workshop on Database and Expert Systems Applications, ser. DEXA '00. Washington, DC, USA: IEEE Computer Society, 2000, pp. 732-. [Online]. Available: http://dl.acm.org/citation.cfm?id=789091.790428
  45. H. Ghosh and S. Chaudhury, "Distributed and reactive query planning in r-magic: An agent-based multimedia retrieval system," IEEE Trans. on Knowl. and Data Eng., vol. 16, no. 9, pp. 1082-1095, Sep. 2004. [Online]. Available: http://dx.doi.org/10.1109/TKDE.2004.40
  46. V. Ermolayev and M. Davidovsky, "Agent-based ontology alignment: basics, applications, theoretical foundations, and demonstration," in Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics, ser. WIMS '12. New York, NY, USA: ACM, 2012, pp. 3:1-3:12. [Online]. Available: http://doi.acm.org/10.1145/2254129.2254136
  47. B. Maiti, H. Ghosh, and S. Chaudhury, "A framework for ontology specification and integration for multimedia applications," in Proceedings of Knowledge Based Computer System (KBCS 2004), December 2004.
  48. V. Mezaris, A. Scherp, R. Jain, M. Kankanhalli, H. Zhou, J. Zhang, L. Wang, and Z. Zhang, "Modeling and representing events in multimedia," in Proceedings of the 19th ACM international conference on Multimedia, ser. MM '11. New York, NY, USA: ACM, 2011, pp. 613-614. [Online]. Available: http://doi.acm.org/10.1145/2072298.2072391
  49. P. Appan and H. Sundaram, "Networked multimedia event exploration," in Proceedings of the 12th annual ACM international conference on Multimedia, ser. MULTIMEDIA