The Future of VR and AR Interfaces
2001, GMD–Forschungszentrum Informationstechnik GmbH
Abstract
What can Virtual Reality and Augmented Reality technologies do for us? Can they help us make better use of computers? Will they ever improve the quality of our personal lives? What kind of immersive interfaces could re-shape the way we communicate with computers? How far along are we on that road, and what remains to be done? This is only a small selection of the questions discussed at the workshop on the future of VR and AR interfaces at IEEE Virtual Reality 2001. Main areas of focus were conversational user interfaces, natural ...
References (93)
- J. Brederson, M. Ikits, C. Johnson, and C. Hansen. The visual haptic workbench. In Proc. Fifth PHANToM Users Group Workshop (PUG), 2000.
- M. Ikits, J. Brederson, C. Hansen, and J. Hollerbach. An improved calibration framework for electromagnetic tracking de-vices. In Proc. IEEE Virtual Reality, 2001 (to appear).
- C. Lin and R. Loftin, VR User Interface: Closed World Interaction, VRST'2000, pp. 153-159.
- C. Lin and R. Loftin, Interaction with Geoscience Data in an Immersive Environment, VR'2000, pp. 55-62.
- K. Hinckley, J. Tullio, R. Pausch, D. Proffitt, N. Kassel, Usability Analysis of 3D Rotation Techniques, Proceedings of UIST'97, 1997, pp. 1 -10.
- K. Hinckley, R. Pausch, J. Goble, N. Kassel, A Survey of Design Issues in Spatial Input, Proceedings of UIST'94, 1994, pp. 213 -222
- Billinghurst, M. "Shared Space: An Augmented Reality Approach for Computer Supported Cooperative Work". Virtual Realty. 1998. Vol. 3, No. Springer.
- Broll, W. "Interacting in distributed collaborative environment" IEEE VRAIS' 95. p.148-155. IEEE Computer Society Press, 1995.
- Broll, W. et al "The Virtual Round Table -a Collaborative Augmented Multi-User Environment". ACM Collaborative Virtual Environments, 2000, pp. 39-46.
- Dumas, C. et al "Spin: a 3d Interface for Cooperative Work". Virtual Reality. 1999. Vol. 4, No.1. Springer. pp. 15-25.
- Kiyokawa, K. Takemura, H. Yokoya, N. "Seamless Design for 3D object Creation". IEEE Multimedia. Jan 2000.
- D. Margery, B. Arnaldi, N. Plouzeau. A General Framework for Cooperative Manipulation in Virtual Environments. Virtual Environments '99, M. Gervautz, A. Hildebrand, D. Schmalstieg (eds.), Springer, pages 169-178, 1999.
- Noma, Miyasato,"Cooperative Object Manipulation in Virtual Space using Virtual Physics", ASME-DSC-Vol.61, pp.101-106,1997
- Ohshima, T. et al. "AR2 Hockey: A case study of Figure 3: Proposed framework collaborative augmented reality". IEEE VR' 1998. pp 268-275.
- Szalavari, Z., et al. "Studierstube: An environment for collaboration in augmented reality". Virtual Realty. 1998. Vol. 3, No.1. Springer. pp. 37-48.
- Vuillème, A. et alii, "Nonverbal Communication Interface for Collaborative Virtual Environments". Virtual Realty. 1999. Vol. 4, No.1. Springer. pp. 49-59.
- Bernhard Jung, Stefan Kopp, Marc Latoschik, Timo Sowa, and Ipke Wachsmuth. Virtuelles Konstruieren mit Gestik und Sprache. Künstliche Intelligenz, 2/00:5_11, 2000.
- Stefan Kopp and Ipke Wachsmuth. A knowledge-based approach for lifelike gesture animation. In W. Horn, editor, ECAI 2000 - Proceedings of the 14th European Conference on Artificial Intelligence, pages 663_667, Amsterdam,2000.
- Marc Erich Latoschik. Multimodale Interaktion in Virtueller Realität am Beispiel der virtuellen Konstruktion. PhD thesis, Technische Fakultät, Universität Bielefeld, 2001.
- J.-L. Nespoulous and A.R. Lecours. Gestures: Nature and function. In J.-L. Nespoulous, P. Rerron, and A.R. Lecours, editors, The Biological Foundations of Gestures: Motor and Semiotic Aspects. Lawrence Erlbaum Associates, Hillsday N.J., 1986.
- Henrik Tramberend. A distributed virtual reality framework. In Virtual Reality, 1999. [6] http://www.techfak.uni- bielefeld.de/techfak/ags/wbski/wbski_engl.html. References
- R. T. Azuma. A survey of Augmented Reality. Presence: Teleoperators and Virtual Environment, 6(4), pp.355-385, Aug. 1997
- G. Welch, G. Bishop, SCAAT: Incremental Tracking with Incomplete Information, Proceedings of Siggraph97, Computer Graphics, pp. 333-344
- YoungKwan Cho, Scalable Fiducial-Tracking Augmented Reality, Ph.D. Dissertation, Computer Science Department, University of Southern California, January 1999
- J. Park, B. Jiang, and U. Neumann. "Vision- based Pose Computation: Robust and Accurate Augmented Reality Tracking," IEEE International Workshop on Augmented Reality, Oct. 1999, pp. 3-12
- Y. Bar-Shalom X. Li, Estimation and Tracking: Principles, Techniques, and Software, Artech House, Norwood MA, 1998
- U. Neumann, S. You, "Natural Feature Tracking for Augmented Reality," IEEE Transactions on Multimedia, Vol. 1, No. 1, pp. 53-64, March 1999.
- U. Neumann and A. Majoros, "Cognitive, Performance, and Systems Issues for Augmented References
- Azuma, R., A Survey of Augmented Reality. Presence: Teleoperatore and Virtual Environments, 1997. 6(4): pp. 355-385.
- Bajura, M., Fuchs, H., Ohbuchi, R., Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery Within the Patient. Proceedings of SIGGRAPH ™92. 1992. ACM. pp. 203-210.
- Butterworth, J., Davidson, A., Hench, S., Olano, T., 3DM: a three dimensional modeler using a head-mounted display. Proceedings of Symposium on Interactive 3D graphics. 1992. ACM. pp. 135-138.
- Kato, H., Billinghurst, M., Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System, Proc. of 2nd Int. Workshop on Augmented Reality, pp.85-94 (1999). Proceedings of 2nd Int. Workshop on Augmented Reality. 1999. pp. 85-94.
- Mapes, D., Moshell, J., A Two-Handed Interface for Object Manipulation in Virtual Environments. Presence: Teleoperators and Virtual Environments, 1995. 4(4): pp. 403-416.
- Milgram, P., Takemura, H., Utsumi, A., Kishino, F., Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum. Proceedings of Telemanipulator and Telepresence Technologies. 1994. SPIE.
- Poupyrev, I., Berry, R., Kurumisawa, J., Nakao, K., Billinghurst, M., et al., Augmented Groove: Collaborative Jamming in Augmented Reality. Proceedings of SIGGRAPH'2000 CA and A. 2000. ACM. pp. 77.
- Rekimoto, J., Ayatsuka, Y., Hayashi, K., Augmentable reality: Situated communication through physical and digital spaces. Proceedings of ISWC'98. 1998. IEEE.
- Schmalstieg, D., Fuhrmann, A., Hesina, G., Bridging multiple user interface dimensions with augmented reality systems. Proc. of ISAR'2000. 2000. IEEE. pp. 20-29.
- Ullmer, B., Ishii, H., The metaDesk: Models and Prototypes for Tangible User Interfaces. Proceedings of UIST'97. 1997. ACM. pp. 223- 232. A video clip is available at http://www.cs.cmu.edu/~desney/Tiles/
- Reference
- Aish R., "3D input for CAAD systems", Computer-Aided Design, 11 (2):66-70, Mar. 1979.
- Arthur P. and Passini R., Wayfinding: People, Signs, and Architecture, Toronto: McGraw-Hill Ryerson 1992.
- Baird J. C., "Studies of the Cognitive Representation of Spatial Relations", Journal of Experimental Psychology: General, 108 (1): 90- 91, 1979.
- Colle H. A. and Reid G. B., "The Room Effect: Metric Spatial Knowledge of Local and Separated Regions", Presence: Teleoperators and Figure 1: Virtual (left) and physical (right) overviews of the CMP Virtual Environments, 7 (2): 116-129, April 1998.
- Darken R. P. and Allard T., "Spatial Orientation and Wayfinding in Large-Scale Virtual Spaces II", Presence: Teleoperators and Virtual Environments, 8 (6): iii-vii, Dec. 1999.
- Frazer J. H., "Use of Simplified Three - Dimensional Computer Input Devices to Encourage Public Participation in Design", Computer Aided Design 82, Conference Proceedings, Butterworth Scientific, 143-151, 1982.
- Frazer J. H., An Evolutionary Architecture, Architectural Association 1995.
- Golledge R. G., "Cognition of Physical and Built Environments." In T. Garling and G. W. Evans (eds.), Environment, Cognition and Action: An Integrated Approach, New York: Oxford UP 1991.
- Howard J. H. JR. and Kerst S. M., "Memory and Perception of Cartographic Information for Familiar and Unfamiliar Environments", Human Factors, 23 (4): 495-504, 1981.
- Liu L., Gauthier L. and Gauthier S., "Spatial Disorientation in Persons with Early Senile Dementia of the Alzheimer Type", The American Journal of Occupational Therapy, 45 (1): 67-74, Jan. 1991.
- Patrick E. and Cosgrove D. et al, "Using a Large Projection Screen as an Alternative to Head- Mounted Displays for Virtual Environments", CHI 2000, 478-485, April 2000.
- Ruddle R. A. and Payne S. J. et al, "Navigating Large-Scale 'Desk-Top' Virtual Buildings: Effects of Orientation Aids and Familiarity", Presence: Teleoperators and Virtual Environments, 7 (2): 179-193, April 1998.
- Ullmer B. and Ishii H., "Emerging Frameworks for Tangible User Interfaces", IBM System Journal, 39(3-4): 915-931, 2000.
- Sutphen S., Sharlin E., Watson B. and Frazer J., "Reviving a Tangible Interface Affording 3D Spatial Interaction", WCGS 2000 (Western Computer Graphics Symposium), Panorama, British Columbia, Canada, March 2000.
- R. Sharma, V.I. Pavlovic, and T.S.Huang, Toward multimodal human-computer interface, In Proceedings of the IEEE, 86(5): 853-869, May 1998.
- Binnie, C.D., Recording the brain's electrical activity, Electrical Engineering and Epilepsy: A Successful Partnership (Ref. No. 1998/444), IEE Colloquium on, 1998 Page(s): 3/1 -3/3
- Michael Scherg, FOCUS source imaging: New perspectives on digital EEG review, Division of Biomagnetism, Clinic of Neurology, University of Heidelberg
- Zhong Zhang; Kawabata, H.; Zhi-Qiang Liu, EEG analysis using fast wavelet transform, Systems, Man, and Cybernetics, 2000 IEEE International Conference on , Volume: 4 , 2000, Page(s): 2959 -2964
- Herrera, R.E.; Sclabassi, R.J.; Mingui Sun; Dahl, R.E.; Ryan, N.D., Single trial visual event- related potential EEG analysis using the wavelet transform, BMES/EMBS Conference, 1999. Proceedings of the First Joint , Volume: 2 , 1999, Page(s): 947 vol.2
- Amir B. Geva, Dan H. Kerem, Brain State Identification and Forecasting of Acute Pathology Using Unsupervised Fuzzy Clustering of EEG Temporal Patterns, Fuzzy and Neuro- fuzzy Systems in Medicine, CRC Press LLC References
- M. Black and Y. Yacoob. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motions. IJCV, 25(1), 1997, 23-48.
- I. Haritaoglu, D. Harwood, and L. Davis, W4S: A real time system for detecting and tracking people in 2.5D. ECCV, 1998, 877-892.
- S. X. Ju, M. Black,and Y. Yacoob. Cardboard people: A parameterized model of articulated image motion. in Proc. Int. Conference on Face and Gesture, Vermont, 1996, 561-567.
- Y. Yacoob and L. Davis, Y. Yacoob and L.S. Davis, Learned Models for Estimation of Rigid and Articulated Human Motion from Stationary or Moving Camera, Int. Journal of Computer Vision, 36(1), 2000, 5-30.
- Bruce, V. and Green, P. (1989), Visual Perception: Physiology, Psychology and Ecology. Lawrence Erlbaum Associates, London.
- Campbell, L.W., Becker, D.A., Azarbayejani, A., Bobick, A., and Pentland, A. (1996). Invariant features for 3-D gesture recognition. MIT Media Laboratory Perceptual Computing Section, Technical Report no. 379.
- Cassell, J., Pelachaud, C. Badler, N. et . al (1994) Animated Conversation: Rule-based Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational [4] Agents. Computer Graphics Proceedings, pp.413-420.
- Chellapa, R., Wilson, C.L. and Sirohey, S. (1995). Human and machine recognition of faces: a survey. Proceedings of the IEEE, Vol. 83, No.5, pp. 705-740
- Cohen, M. and Massaro, D. (1993) Modeling Coarticulation in Synthetic Visual Speech,
- Models and Techniques in Computer Animation, Springer Verlag, pp. 139-156.
- Cootes, T.F., et al. (1992). Training models of shape from sets of examples. Proceedings of the British Machine Vision Conference.
- Cootes, T.F., Edwards, G.J. and Taylor, C.J. (1998). Proceedings of the European Confe rence on Computer Vision, Burkhardt, H. and Neumann, B. (Eds.)., Vol. 2, pp. 484-498, Springer-Verlag.
- Eisert, P. and Girod, B. (1998). Analyzing facial expressions for virtual conferencing. IEEE Computer Graphics and Applications, Vol. 18, No. 5, pp. 70-78.
- Ekman, P. and Friesen, W.V. (1975). Unmasking the Face: A Guide to Recognizing Emotions from Facial Expressions, Englewood Cliffs, New Jersey; Prentice Hall, Inc.
- Ekman, P. and Friesen, W.V. (1978), Manual for the Facial Action Coding System. Consulting Psychologists Press, Inc. Palo Alto, CA.
- Lam, K.M. and Yang, H. (1996). ``Locating and extracting the eye in human face images''.
- Pattern Recognition, Vol. 29, No. 5, pp. 771- 779.
- Ohzu, H. and Habara, K. (1996). Behind the scenes of virtual reality: vision and motion. Proceedings of the IEEE, Vol. 84, No. 5, pp. 782-798.
- Oliver, N., Pentland, A. and Berard, F. (1997). Lafter: lips and face real time tracker. MIT, Media Laboratory Perceptual Computing Section, Technical report no. 396.
- Parke, F.I. and Waters, K. (1996). Computer Facial Animation. A K Peters.
- Pentland, A.P. (1996). Smart rooms. Scientific American. April 1996, pp. 54-62.
- Rios, H.V. and Peña, J. (1998). Computer Vision interaction for Virtual Reality, In Progress in Artificial Intelligence, Helder Coelho (Ed.), Lecture Notes in Artificial Intelligence, Subseries of Lecture Notes in Computer Science, No. 1484, pp. 262-273. Springer- Verlag.
- Russel, J. (1994). Is There Universal Recognition of Emotion From Facial Expression?. A Review of Cross-Cultural Studies Psychological Bulletin 115(1) 102-141.
- Schwartz, E.I. (1995). A face of one´s own. Discover the world of Science. Vol. 16, No. 2, pp. 78-87.
- Starner, T., Weaver, J., Pentland, A. (1998). Real-time American sign language recognition using desk and wearable computer based video. MIT, Media Laboratory Perceptual Computing Section, Technical report no. 466.
- Sucar, L.E. and Gillies, D.F. (1994). Probabilistic reasoning in high-level vision. Image and Vision Computing, Vol. 12, No. 1, pp.42-60.
- Terzopoulos, D. And Szeliski, R. (1992). Tracking with Kalman Snakes. In Active Vision, Blake, A. and Yuille, A. (Eds.), MIT Press.
- Waters, K. (1987). A Muscle Model for Animating Three Dimensional Facial Expression. Computer Graphics Proceedings Vol. 21, No 4 , pp.17-23.
- Waters, K. and Levergood, T.(1994) An Automatic Lip-Synchronization Algorithm for Synthetic Faces. Proceedings of Multimedia 94, ACM, pp149-156.
- Tulving, E. Memory and Consciousness. Canadian Psychologist, 26, pp 1-12, (1985).
- Mania, K., Chalmers, A., Troscianko, T., Hawkes, R. The Effects of Levels of Immersion on Memory and Presence in Virtual Environments: A Reality Centered Approach, to appear in Cyberpsychology and Behavior journal, issue 4.2.
- L. Rosenblum, "Virtual and Augmented Reality 2020", IEEE Computer Graphics and Applications, 20(1), 2000, pp. 38-39.
- F. P. Brooks, Jr. "What's Real About Virtual Reality?", IEEE Computer Graphics and Applications, 19(6), 1999, pp. 16-27.
- B. MacIntyre and S. Feiner, "Future Multimedia User Interfaces", Multimedia Systems, 4(5), 1996, pp.150-168.