Digital Media Engineering

The study is to find relative EEG power spectrum and pattern of coherence discriminating attentive and inattentive hand movements. Eight undergraduate students aged from 20 to 27 who had not hand disability participated in this study. Participants were asked to perform visuo-motor task. EEG was measured at C3 in 10~20 international system and four areas orthogonally directed 2.5cm away from C3. Significant result discriminating movement and rest was found through coherence analysis between movement areas or movement area and non-movement area, but was individually different. Because it was anticipated that major factors caused by the differences among individuals were attributed to the attention of the subjects, relative power of alpha and beta bands was identified. As a result, significant relative powers of alpha and beta bands were found in a group of high coherence level, but were not found in a group of low level. Next, participants were divided into two groups according to relative powers of alpha and beta bands. The comparison between two groups was performed. As a result, the coherence of the alpha band in the attentive group was greater than that of the inattentive group. It was found that the coherence of the beta band in the inattentive group was happening. Therefore, individual differences of coherence were influenced by attention. The significant coherence patterns that could discriminate attentive movement and inattentive movement were found.

This study is to propose new method, called by TDP (time dependenet parameter) anlaysis, of physiological signal processing for emoiton recognition. TDP consised of delay, activation, half recovery and full recovery. TDP was determined from running average and normalization of physiological signals for finding tonic and phasic reponse according to emotion at entire time range from stimulating emotion to recovery. As the results of this study, TDP analysis and adaptive TDP analysis enhanced accuracy of emotion recognition in the comparison with tonic analysis. Speciallly, TDP analysis enhanced the accuracy while adaptive TDP analysis reduced the individual difference of the accuracy.

This study is to propose a realistic game system using a multi-modal interface, including gaze tracking, hand gesture recognition and bio-signal analysis. Our research is novel in the following four ways, compared to previous game systems. First, a highly immersive and realistic game is implemented on a head mounted display (HMD), with a gaze tracker, a gesture recognizer and a bio-signal analyzer. Second, since the camera module for eye tracking is attached below the HMD, a user's gaze position onto the HMD display can be calculated without wearing any additional eye tracking devices. Third, an aiming cursor in the game system is controlled by the gaze tracking. The grabbing and throwing behaviors toward a target are performed by the user's hand gestures using a data glove. Finally, the level of difficulty in the game system is adaptively controlled according to the measurement and analysis of a user's bio-signals.

With the recent increase in the number of three-dimensional (3D) applications, the need for interfaces to these applications has increased. Although the eye tracking method has been widely used as an interaction interface for hand-disabled persons, this approach cannot be used for depth directional navigation. To solve this problem, we propose a new brain computer interface (BCI) method in which the BCI and eye tracking are combined to analyze depth navigation, including selection and two-dimensional (2D) gaze direction, respectively.

This paper proposes an emotionally interactive lighting system to support affective experiences while enjoying video content. The emotional lighting system is controlled by changes in an individual's emotional state. The newly designed system consists of an emotion recognition system that uses three different physiological signals (photoplethysmography, skin temperature, and galvanic skin response), an emotion lighting control system, and an emotion ambient lighting system. The physiological responses of nineteen subjects are evaluated in terms of emotional arousal and relaxation states with two different colors of light. The physiological signals are analyzed in both the time and frequency domain, and test results show that the peak-to-peak interval value and ratio of high to very-low frequencies of the photoplethysmography signals decreases significantly following a red light stimulus when arousing video content is played, whereas the same factors exhibit a significant increase after a blue light stimulus while relaxing video content is watched. These results show that red and blue light is an effective intensifier of emotional arousal and relaxation experiences during displays of arousing and relaxing video content, respectively. 1

The primary aims of this research were to examine (1) mu and beta event-related desynchronization/ synchronization (ERD/ERS) during motor imagery tasks with varying movement duration and (2) the potential impacts of movement duration on ERD/ERS patterns. Motor imagery tasks included brief and continuous imagined hand movements. During an imagery task, participants imagined an indicated movement for 1 s (i.e., brief movement imagery) or 5 s (i.e., continuous movement imagery). The results of the study support (1) that mu and beta ERD/ERS patterns are elicited during imagined hand movements and (2) that movement duration affects ERS and does not affect ERD patterns, during motor movement imagery. Additionally, brief movement imagery had a greater impact on mu and beta ERD; continuous movement imagery had a greater impact on mu and beta ERS. This research will be useful for designing future brainecomputer interfaces as it provides valuable insight into the dynamics of electroencephalographic (EEG) oscillatory changes during motor imagery tasks with varying movement duration. Relevance to industry: : Brainecomputer interfaces (BCIs) have gained considerable interests by both research and industry communities who want to improve the quality of life for those who suffer from severe motor disabilities, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, and cerebral palsy (CP). The results of this study should be applied to EEG-based BCI system design in order to enhance accuracy and classification performance for BCI system control.

Speller UI systems tend to be less accurate because of individual variation and the noise of EEG signals. Therefore, we propose a new method to combine the EEG signals and gaze-tracking. This research is novel in the following four aspects. First, two wearable devices are combined to simultaneously measure both the EEG signal and the gaze position. Second, the speller UI system usually has a 6 × 6 matrix of alphanumeric characters, which has disadvantage in that the number of characters is limited to 36. Thus, a 12 × 12 matrix that includes 144 characters is used. Third, in order to reduce the highlighting time of each of the 12 × 12 rows and columns, only the three rows and three columns (which are determined on the basis of the 3 × 3 area centered on the user's gaze position) are highlighted. Fourth, by analyzing the P300 EEG signal that is obtained only when each of the 3 × 3 rows and columns is highlighted, the accuracy of selecting the correct character is enhanced. The experimental results showed that the accuracy of proposed method was higher than the other methods.

Previous research has indicated that viewing 3D displays may induce greater visual fatigue than viewing 2D displays. Whether viewing 3D displays can evoke measureable emotional responses, however, is uncertain. In the present study, we examined autonomic nervous system responses in subjects viewing 2D or 3D displays. Autonomic responses were quantified in each subject by heart rate, galvanic skin response, and skin temperature. Viewers of both 2D and 3D displays showed strong positive correlations with heart rate, which indicated little differences between groups. In contrast, galvanic skin response and skin temperature showed weak positive correlations with average difference between viewing 2D and 3D. We suggest that galvanic skin response and skin temperature can be used to measure and compare autonomic nervous responses in subjects viewing 2D and 3D displays.

Knowledge about people's emotions can serve as an important context for automatic service delivery in context-aware systems. Hence, human facial expression recognition (FER) has emerged as an important research area over the last two decades. To accurately recognize expressions, FER systems require automatic face detection followed by the extraction of robust features from important facial parts. Furthermore, the process should be less susceptible to the presence of noise, such as different lighting conditions and variations in facial characteristics of subjects. Accordingly, this work implements a robust FER system, capable of providing high recognition accuracy even in the presence of aforementioned variations. The system uses an unsupervised technique based on active contour model for automatic face detection and extraction. In this model, a combination of two energy functions: Chan-Vese energy and Bhattacharyya distance functions are employed to minimize the dissimilarities within a face and maximize the distance between the face and the background. Next, noise reduction is achieved by means of wavelet decomposition, followed by the extraction of facial movement features using optical flow. These features reflect facial muscle movements which signify static, dynamic, geometric, and appearance characteristics of facial expressions. Post-feature extraction, feature selection, is performed using Stepwise Linear Discriminant Analysis, which is more robust in contrast to previously employed feature selection methods for FER. Finally, expressions are recognized using trained HMM(s). To show the robustness of the proposed system, unlike most of the previous works, which were evaluated using a single dataset, performance of the proposed system is assessed in a large-scale experimentation using five publicly available different datasets. The weighted average recognition rate across these datasets indicates the success of employing the proposed system for FER.

Brain Computer Interface (BCI) studies have been done to help people manipulate electronic devices in a 2D space but less has been done for a vigorous 3D environment. The purpose of this study was to investigate the possibility of applying Steady State Visual Evoked Potentials (SSVEPs) to a 3D LCD display. Eight subjects (4 females) ranging in age between 20 to 26 years old participated in the experiment. They performed simple navigation tasks on a simple 2D space and virtual environment with/without 3D flickers generated by a Flim-Type Patterned Retarder (FPR). The experiments were conducted in a counterbalanced order. The results showed that 3D stimuli enhanced BCI performance, but no significant effects were found due to the small number of subjects. Visual fatigue that might be evoked by 3D stimuli was negligible in this study. The proposed SSVEP BCI combined with 3D flickers can allow people to control home appliances and other equipment such as wheelchairs, prosthetics, and orthotics without encountering dangerous situations that may happen when using BCIs in real world. 3D stimuli-based SSVEP BCI would motivate people to use 3D displays and vitalize the 3D related industry due to its entertainment value and high performance.

This study is aimed to determine significant physiological parameters of brain and heart under meditative state, both in each activities and their dynamic correlations. Electrophysiological changes in response to meditation were explored in 12 healthy volunteers who completed 8 weeks of a basic training course in autogenic meditation. Heart coherence, representing the degree of ordering in oscillation of heart rhythm intervals, increased significantly during meditation. Relative EEG alpha power and alpha lagged coherence also increased. A significant slowing of parietal peak alpha frequency was observed. Parietal peak alpha power increased with increasing heart coherence during meditation, but no such relationship was observed during baseline. Average alpha lagged coherence also increased with increasing heart coherence during meditation, but weak opposite relationship was observed at baseline. Relative alpha power increased with increasing heart coherence during both meditation and...

Out of the body" tactile illusion refers to the phenomenon in which one can perceive tactility as if emanating from a location external to the body without any stimulator present there. Taking advantage of such a tactile illusion is one way to provide and realize richer interaction feedback without employing and placing actuators directly at all stimulation target points. However, to further explore its potential, it is important to better understand the underlying physiological and neural mechanism. As such, we measured the brain wave patterns during such tactile illusion and mapped out the corresponding brain activation areas. Participants were given stimulations at different levels with the intention to create veridical (i.e., non-illusory) and phantom sensations at different locations along an external hand-held virtual ruler. The experimental data and analysis indicate that both veridical and illusory sensations involve, among others, the parietal lobe, one of the most important components in the tactile information pathway. In addition, we found that as for the illusory sensation,

Objective: The aim of this study was to investigate the correlation between 3D visual fatigue and physiological measures by canonical correlation analysis enabling to categorical correlation. Background: Few studies have been conducted to investigate the physiological mechanism underlying the visual fatigue caused by processing 3D information which may make the cognitive mechanism overloaded. However, even the previous studies lack validation in terms of the correlation between physiological variables and the visual fatigue. Method: 9 Female and 6 male subjects with a mean age of voluntarily participated in this experiment. All participants were asked to report how they felt about their health sate at after viewing 3D. In addition, Low & Hybrid measurement test(Event Related Potential, Steady-state Visual Evoked Potential) and for evaluating cognitive fatigue before and after viewing 3D were performed. The physiological signal were measured with subjective fatigue evaluation before ...

In the VnR (Virtual and Real Worlds) of recent virtual reality convergence, the modelling of realistic human face is focused on the facial appearance such as the shape of facial parts and muscle movement. However, the facial changing parameters caused by environmental factors beyond the facial appearance factors can be regarded as important ones in terms of effectively representing virtual avatar. Therefore, this study evaluates user's visual feeling response according to the opacity variation of eye reflection of virtual avatar which is considered as a new parameter for reprenting realistic avatar. Experimental result showed that more clear eye reflection induced more realistic visual feeling of subjects. This result can be regarded as a basis for designing realistic virtual avatar by supporting a new visual realistic representing factor (eye reflection) and its degree of representation (reflectance ratio).

This research is to evaluate the arousal level by using cardiovascular response. PPG was used in this study as one of the method of measuring it rather than ECG (Electrocardiography) for the purpose of solving ergonomic problem of sensing. The participants were in the age group of 20 (mean=24, standard deviation=1.25): five men and five women. Each experiment composed with four identical sets. First, a black screen was displayed for 30 second rest. Then, the prepared 6 pair images were randomly presented for 10 second stimulation and for 30 second non-stimulation. PPG was measured on the earlobes of experimenters at 200Hz sampling frequency. PPG amplitude, PPI(Pulse to Pulse Interval), and PRV(Pulse Rate Variability) were analyzed according to arousal level. T-test was performed to compare between the PPG variables of rest and relaxation, rest and arousal, and relaxation and arousal. Relative to the rest state, PPG amplitude decreased in relaxed state and increased in aroused state. Relative to the rest state, PPI decreased in both emotional states. However, more significant decline was observed in aroused state. PRV's LF and HF were used in the form of LF/HF to compare between the relaxed and the aroused state. Therefore, PPG signal showed significant differences between relaxed and aroused state. In conclusion, evaluation of human arousal level used in the PPG analysis demonstrated that PPG has better usability and comforter measurement than ECG and is clearly an alternative method of measuring arousal level.