Vision-Based Markerless Gaming Interface

2009

VideoPlace-like vision-driven user interface using "optical flow" measurements and elastic labeled silhouette .The optical flow not only detects the movements but also gives us an estimate of the direction and the speed of the movement. The proposed representation is based on a self -organizing system designed to learn to recognize both the characteristic features of the image and their spatial relationship without needs of initializations or special settings. The positions of the units composing the system allow extracting information about the position and the dynamics of the observed figure. Reported results show how it is possible to identify the skeleton (legs and torso) of the walking subject using four units. It can be observed that the low-resolution skeleton formed by the four units correctly tracks the walking pattern of the two legs, while the upper segment remains centered on the subject body.

Lecture Notes in Computer Science, 2006

Mobile virtual environments, with real-time 3D and 2D graphics, are now possible on smart phone and other camera-enabled devices. Using computer vision, the camera sensor can be treated as an input modality in applications by analyzing the incoming live video. We present our tracking algorithm and several mobile virtual environment and gaming prototypes including: a 3D first person shooter, a 2D puzzle game and a simple action game. Camera-based interaction provides a user experience that is not possible through traditional means, and maximizes the use of the limited display size.

2009

This paper describes a command interface for games based on hand gestures defined by postures, movement and location. The large variety of gestures thus possible increases usability by allowing a better match between gesture and action. The system uses computer vision requiring no sensors or markers on the user or background. The analysis of requirements for games, the architecture and implementation are discussed, as well as the results of several tests to evaluate how well each requirement is met.

Proceedings of the 4th International ICST Conference on Simulation Tools and Techniques, 2011

With no doubt, advanced human-computer interaction technologies represent one of the key selling points of all the most popular new gaming consoles. Players, in fact, can now interact realistically with their gaming environments, not suffering anymore from that lack of authenticity that was caused by the use of a simple joysticks or mice. Such incredible result has been achieved by means of new advanced hardware specialized in the recognition of players' gestures. In this work, we show that the same objective can be attained utilizing an off-the-shelf webcam and a robust gesture recognition software system. In particular, our novel gesture recognition system, which is based on artificial vision algorithms, leads to a fast and robust interpretation of hand gestures. This type of technology has proven to be particularly efficient for immersive gaming experiences. The efficacy was demonstrated through a game, which has been enjoyed by hundreds of visitors at the 2010 Shanghai World Expo.

2008

We present a framework that allows for a straightforward development of multi-camera controlled interactive video games. Compared to traditional gaming input devices, cameras provide players with many degrees of freedom and a natural kind of interaction. The use of cameras can even obsolete the need for special clothing or other tracking devices. This partly accounted for the success of the currently popular single-camera video games like the Sony Eyetoy. However, these games are fairly limited in the use of 3D scene information. Using multi-camera setups, this limitation can be surpassed, but typically many different image processing and computer vision techniques are involved. Our framework divides multi-camera based games into basic algorithms that are easily combinable into several sequentially executed stages. Therefore the amount of effort to develop new games can significantly be reduced. The capabilities of our framework are demonstrated with a number of conceptual games, proving that multi-camera controlled video games can be created with off-the-shelf hardware.

Gesture gaming is a method by which users having a laptop/pc/x-box play games using natural or bodily gestures. This paper presents a way of playing free flash games on the internet using an ordinary webcam with the help of open source technologies. Emphasis in human activity recognition is given on the pose estimation and the consistency in the pose of the player. These are estimated with the help of an ordinary web camera having different resolutions from VGA to 20mps. Our work involved giving a 10 second documentary to the user on how to play a particular game using gestures and what are the various kinds of gestures that can be performed in front of the system. The initial inputs of the RGB values for the gesture component is obtained by instructing the user to place his component in a red box in about 10 seconds after the short documentary before the game is finished. Later the system opens the concerned game on the internet on popular flash game sites like miniclip, games arcade, GameStop etc and loads the game clicking at various places and brings the state to a place where the user is to perform only gestures to start playing the game. At any point of time the user can call off the game by hitting the esc key and the program will release all of the controls and return to the desktop. It was noted that the results obtained using an ordinary webcam matched that of the Kinect and the users could relive the gaming experience of the free flash games on the net. Therefore effective in game advertising could also be achieved thus resulting in a disruptive growth to the advertising firms.

Journal 4 Research - J4R Journal, 2018

This paper describes a command interface for games based on hand gestures and voice command defined by postures, movement and location. The system uses computer vision requiring no sensors or markers by the user. In voice command the speech recognizer, recognize the input from the user. It stores and passes command to the game, action takes place. We propose a simple architecture for performing real time colour detection and motion tracking using a webcam. The next step is to track the motion of the specified colours and the resulting actions are given as input commands to the system. We specify blue colour for motion tracking and green colour for mouse pointer. The speech recognition is the process of automatically recognizing a certain word spoken by a particular speaker based on individual information included in speech waves. This application will help in reduction in hardware requirement and can be implemented in other electronic devices also.

Journal of Visual Communication and Image Representation, 2012

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues.

We propose to develop a novel motion-driven interface, without having to hold additional devices such as Wii Remote. The body gesture can be first processed as silhouette, and a finite state machine follows to recognize the continuous motion as skeletons. Users can have more fun playing with avatars more naturally.

Wireless Personal Communications, 2016

The use of motion controllers and cameras such as Microsoft Kinect as a gaming peripheral have increased the immersion of computer games today. Moving to using gestures instead of conventional handheld game controllers has increased the interest of players. Xbox 360, Xbox-One and the games that have been developed to be used with Kinect enable users to play using gestures, but this limits us to play our old favorite games and the new non-Kinect based games with same conventional peripherals. To overcome this limitation, we have developed a user friendly interface in MATLAB that replaces keyboard and mouse controls with our own designed gestures and enables us to play all non-Kinect based games using Kinect. For testing and demonstration purpose, we have applied the developed interface to three games which are Counter-Strike Global Offense, Candy Crush and Asphalt 8 Airborne.