G. Skoumas

Intuitive Recipes for Uncertainty Decoding with SNR Features for Noise Robust ASR

In this work, uncertainty decoding in automatic speech recognition is investigated in the context... more In this work, uncertainty decoding in automatic speech recognition is investigated in the context of robustness to additive and convolutional noise. In Garner (2009) and in Garner (2011), explicit calculation of a Signal to Noise Ratio (SNR) cepstrum by means of a noise estimate is shown to have theoritical and practiacal adavantages over the usual energy based spectrum, especially when they are combined with traditional feature enhancement techniques, such as Cepstral Mean Normalization (CMN) and Cestral Variance Normalization (CVN). This is the reason we try to investigate how these features behave when we are trying to handle the uncertainty which is introduced by environmental or convolutional noise. We compute uncertainty, which can be translated as a variance estimate, in the normal SNR front end and pass this to the decoder. Unfortunatelly, this approximation suffers from theoritical and practical problems, epsecially in low SNR conditions. The issue described and the performance of the uncertainty decoding schemes we used, are examined with the aurora 2 digit recognition task.

Download

Towards knowledge-enriched path computation

Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2014

Directions and paths, as commonly provided by navigation systems, are usually derived considering... more Directions and paths, as commonly provided by navigation systems, are usually derived considering absolute metrics, e.g., finding the shortest path within an underlying road network. With the aid of crowdsourced geospatial data we aim at obtaining paths that do not only minimize distance but also lead through more popular areas using knowledge generated by users. We extract spatial relations such as "nearby" or "next to" from travel blogs, that define closeness between pairs of points of interest (PoIs) and quantify each of these relations using a probabilistic model. Subsequently, we create a relationship graph where each node corresponds to a PoI and each edge describes the spatial connection between the respective PoIs. Using Bayesian inference we obtain a probabilistic measure of spatial closeness according to the crowd. Applying this measure to the corresponding road network, we obtain an altered cost function which does not exclusively rely on distance, and enriches an actual road networks taking crowdsourced spatial relations into account. Finally, we propose two routing algorithms on the enriched road networks. To evaluate our approach, we use Flickr photo data as a ground truth for popularity. Our experimental results-based on real world datasets-show that the paths computed w.r.t. our alternative cost function yield competitive solutions in terms of path length while also providing more "popular" paths, making routing easier and more informative for the user.

Download

Multimodal Saliency and Fusion for Movie Summarization Based on Aural, Visual, and Textual Attention

IEEE Transactions on Multimedia, 2013

Multimodal streams of sensory information are naturally parsed and integrated by humans using sig... more Multimodal streams of sensory information are naturally parsed and integrated by humans using signal-level feature extraction and higher-level cognitive processes. Detection of attention-invoking audiovisual segments is formulated in this work on the basis of saliency models for the audio, visual and textual information conveyed in a video stream. Aural or auditory saliency is assessed by cues that quantify multifrequency waveform modulations, extracted through nonlinear operators and energy tracking. Visual saliency is measured through a spatiotemporal attention model driven by intensity, color and orientation. Textual or linguistic saliency is extracted from partof-speech tagging on the subtitles information available with most movie distributions. The individual saliency streams, obtained from modality-depended cues, are integrated in a multimodal saliency curve, modeling the time-varying perceptual importance of the composite video stream and signifying prevailing sensory events. The multimodal saliency representation forms the basis of a generic, bottom-up video summarization algorithm. Different fusion schemes are evaluated on a movie database of multimodal saliency annotations with comparative results provided across modalities. The produced summaries, based on low-level features and content-independent fusion and selection, are of subjectively high aesthetic and informative quality.

Download

On quantifying qualitative geospatial data

Proceedings of the Second ACM SIGSPATIAL International Workshop on Crowdsourced and Volunteered Geographic Information - GEOCROWD '13, 2013

Living in the era of data deluge, we have witnessed a web content explosion, largely due to the m... more Living in the era of data deluge, we have witnessed a web content explosion, largely due to the massive availability of User-Generated Content (UGC). In this work, we specifically consider the problem of geospatial information extraction and representation, where one can exploit diverse sources of information (such as image and audio data, text data, etc), going beyond traditional volunteered geographic information. Our ambition is to include available narrative information in an effort to better explain geospatial relationships: with spatial reasoning being a basic form of human cognition, narratives expressing such experiences typically contain qualitative spatial data, i.e., spatial objects and spatial relationships. To this end, we formulate a quantitative approach for the representation of qualitative spatial relations extracted from UGC in the form of texts. The proposed method quantifies such relations based on multiple text observations. Such observations provide distance and orientation features which are utilized by a greedy Expectation Maximization-based (EM) algorithm to infer a probability distribution over predefined spatial relationships; the latter represent the quantified relationships under user-defined probabilistic assumptions. We evaluate the applicability and quality of the proposed approach using real UGC data originating from an actual travel blog text corpus. To verify the result quality, we generate grid-based "maps" visualizing the spatial extent of the various relations.

Download

Continuous monitoring of nearest trajectories

Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems - SIGSPATIAL '14, 2014

Analyzing tracking data of various types of moving objects is an interesting research problem wit... more Analyzing tracking data of various types of moving objects is an interesting research problem with numerous real-world applications. Several works have focused on continuously monitoring the nearest neighbors of a moving object, while others have proposed similarity measures for finding similar trajectories in databases containing historical tracking data. In this work, we introduce the problem of continuously monitoring nearest trajectories. In contrast to other similar approaches, we are interested in monitoring moving objects taking into account at each timestamp not only their current positions but their recent trajectory in a defined time window. We first describe a generic baseline algorithm for this problem, which applies for any aggregate function used to compute trajectory distances between objects, and without any restrictions on the movement of the objects. Using this as a framework, we continue to derive an optimized algorithm for the cases where the distance between two moving objects in a time window is determined by their maximum or minimum distance in all contained timestamps. Furthermore, we propose additional optimizations for the case that an upper bound on the velocities of the objects exists. Finally, we evaluate the efficiency of our proposed algorithms by conducting experiments on three real-world datasets.

Download

Video Event Detection and Summarization Using Audio, Visual and Text Saliency

Detection of perceptually important video events is formulated here on the basis of saliency mode... more Detection of perceptually important video events is formulated here on the basis of saliency models for the audio, visual and textual information conveyed in a video stream. Audio saliency is assessed by cues that quantify multifrequency waveform modulations, extracted through nonlinear operators and energy tracking. Visual saliency is measured through a spatiotemporal attention model driven by intensity, color and motion. Text saliency is extracted from part-of-speech tagging on the subtitles information available with most movie distributions. The various modality curves are integrated in a single attention curve, where the presence of an event may be signified in one or multiple domains. This multimodal saliency curve is the basis of a bottom-up video summarization algorithm, that refines results from unimodal or audiovisual-based skimming. The algorithm performs favorably for video summarization in terms of informativeness and enjoyability. Index Terms — multimodal saliency, aud...

Download

A framework for computation of popular paths from crowdsourced data

2015 IEEE 31st International Conference on Data Engineering, 2015

Efficient identification and approximation of k-nearest moving neighbors

Proceedings of the 21st ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems - SIGSPATIAL'13, 2013

ABSTRACT Nowadays, massive amounts of tracking data for various types of moving objects, includin... more ABSTRACT Nowadays, massive amounts of tracking data for various types of moving objects, including vehicles, humans and animals, are becoming available. Analyzing this type of spatio-temporal data is crucial for discovering movement patterns, understanding and forecasting behaviors, and developing novel applications and services. One problem of particular interest is finding objects that move close together with a certain object during some periods of time. In this paper, we focus on finding the k-nearest moving neighbors for a given query object and time interval. We formulate the problem, using a similarity function that takes into consideration both the proximity and the direction of the trajectories, and we firstly present an exact algorithm. Then, we focus on approximate algorithms in order to reduce the execution time, investigating two directions. The first employs line simplification to approximate the compared trajectories, thus reducing the calculations needed to identify the nearest neighbors. The second relies on estimates of prior probabilities derived from trajectory distributions and attempts to achieve a faster approximation of the k-nearest neighbors. A detailed experimental evaluation of the aforementioned algorithms on three real-world datasets is finally presented in order to verify their efficiency and accuracy.

User-contributed Data Collection D3

Location estimation using crowdsourced geospatial narratives

The "crowd" has become a very important geospatial data provider. Subsumed under the term Volunte... more The "crowd" has become a very important geospatial data provider. Subsumed under the term Volunteered Geographic Information (VGI), non-expert users have been providing a wealth of quantitative geospatial data online. With spatial reasoning being a basic form of human cognition, narratives expressing geospatial experiences, e.g., travel blogs, would provide an even bigger source of geospatial data. Textual narratives typically contain qualitative data in the form of objects and spatial relationships. The scope of this work is (i) to extract these relationships from user-generated texts, (ii) to quantify them and (iii) to reason about object locations based only on this qualitative data. We use information extraction methods to identify toponyms and spatial relationships and to formulate a quantitative approach based on distance and orientation features to represent the latter. Positional probability distributions for spatial relationships are determined by means of a greedy Expectation Maximization-based (EM) algorithm. These estimates are then used to "triangulate" the positions of unknown object locations. Experiments using a text corpus harvested from travel blog sites establish the considerable location estimation accuracy of the proposed approach.

Download

Efficient identification and approximation of k-nearest moving neighbors

ABSTRACT Nowadays, massive amounts of tracking data for various types of moving objects, includin... more ABSTRACT Nowadays, massive amounts of tracking data for various types of moving objects, including vehicles, humans and animals, are becoming available. Analyzing this type of spatio-temporal data is crucial for discovering movement patterns, understanding and forecasting behaviors, and developing novel applications and services. One problem of particular interest is finding objects that move close together with a certain object during some periods of time. In this paper, we focus on finding the k-nearest moving neighbors for a given query object and time interval. We formulate the problem, using a similarity function that takes into consideration both the proximity and the direction of the trajectories, and we firstly present an exact algorithm. Then, we focus on approximate algorithms in order to reduce the execution time, investigating two directions. The first employs line simplification to approximate the compared trajectories, thus reducing the calculations needed to identify the nearest neighbors. The second relies on estimates of prior probabilities derived from trajectory distributions and attempts to achieve a faster approximation of the k-nearest neighbors. A detailed experimental evaluation of the aforementioned algorithms on three real-world datasets is finally presented in order to verify their efficiency and accuracy.

Uploads

Papers by G. Skoumas

Log In