Papers by Sakire Arslan Ay
Proceedings of the AAAI Conference on Artificial Intelligence
Capstone senior design projects provide students with a collaborative software design and develop... more Capstone senior design projects provide students with a collaborative software design and development experience to reinforce learned material while allowing students latitude in developing real-world applications. Our two-semester capstone classes are required for all computer science majors. Students must have completed a software engineering course — capstone classes are typically taken during their last two semesters. Project proposals come from a variety of sources, including industry, WSU faculty (from our own and other departments), local agencies, and entrepreneurs. We have recently targeted projects in AI — although students typically have little background, they find the ideas and methods compelling. This paper outlines our instructional approach and reports our experiences with three projects.
Leveraging georeferenced meta-data for the management of large video collections
Unrestricted The rapid adoption and deployment of ubiquitous video sensors has led to the collect... more Unrestricted The rapid adoption and deployment of ubiquitous video sensors has led to the collection of voluminous amounts of data and hence there is an increasing need for techniques that manage these collections in a variety of applications, including surveillance and monitoring systems, web-based video search engines, among others. However, the indexing and search of large video databases remains a very challenging task. Current techniques that extract features purely based on the visual signals of a video are struggling to achieve good results, particularly on the large scale. By considering video related meta-data information more relevant and precisely delimited search results can be obtained.; Latest technological trends have enabled the cost- and energy-efficient deployment of video cameras together with other sensors (e.g., GPS and compass units). The sensor data acquired automatically in conjunction of videos provide important cues about the video content. In this disserta...
Features of WORRDS include: Realtimeand historical data visualization via charts, tables, and oth... more Features of WORRDS include: Realtimeand historical data visualization via charts, tables, and other widgets; 2D map creation using the Google Tango WORRDS app; Custom user and security management; View and edit SHiB layouts with a builtin map editor; Reliable database daemon to sync WORRDSdb with CASASdb.
Title : Self-assisted 12-lead ECG Sensor Placement using RGB-D Sensors in Microsoft Kinect
Introduction: Cardiovascular diseases account for more than 50% of deaths in the United States [1... more Introduction: Cardiovascular diseases account for more than 50% of deaths in the United States [1]. The most standard method of measuring heart activity is through a 12-lead Electrocardiogram (ECG), using wet-gel electrodes [2]. Application of these wet electrodes requires set-up assistance which occupies the precious time of the doctor (> 10 min) not including the time to transport/register the patient. Prolonged ECG usage could further lead to skin irritation and rashes. In addition, it has been reported that 0.4-4% of the all the 12-lead ECGs acquired in various clinical settings have been recorded with incorrect connections of the electrode cables with the highest figure reported in intensive care units[3], [4]. In this context, this work aims to provide a self-operable solution for placing wearable sensors [5] with no or minimal medical assistance. An interactive software is developed to designate where non-contact sensors are to be placed on the body to get the best results...
Towards a Semi-Autonomous Wheelchair for Users with ALS
This paper discusses a prototype system built over two years by teams of undergraduate students w... more This paper discusses a prototype system built over two years by teams of undergraduate students with the goal of assisting users with Amyotrophic Lateral Sclerosis (ALS). The current prototype powered wheelchair uses both onboard and offboard sensors to navigate within and between rooms, avoiding obstacles. The wheelchair can be directly controlled via multiple input devices, including gaze tracking — in this case, the wheelchair can augment the user’s control to avoid obstacles. In its fully autonomous mode, the user can select a position on a pre-built map and the wheelchair will navigate to the desired location. This paper introduces the design and implementation of our system, as well as detailing three sets of experiments to characterize its performance. The long-term goal of this work is to improve the lives of users with mobility impairments, with a particular focus on those with limited motor abilities.
Precise Placement of Precordial Electrodes with +/-0.5 cm Accuracy for Recording ECG in Self-operable Diagnostic Devices
Acute and chronic syndromes account for ~50% of the estimated 500,000 cardiovascular deaths annua... more Acute and chronic syndromes account for ~50% of the estimated 500,000 cardiovascular deaths annually in the United States and similarly in densely populated countries as India. More than 100 years after its invention, surface electrocardiogram introduced by William Einthoven is still the most common fundamental technique and the gold standard for diagnosis, prognosis, screening and evaluating heart disease for many clinical conditions. Progress in wearable diagnostic devices have made possible low-cost, convenient and 24 × 7 point-of-care healthcare; however, their use is still limited to clinical diagnostic measurements. The main gap is the electrode placement at the correct intended location to realize high sensitivity with accurate waveform reconstruction. To achieve a precision of less than 1cm placement accuracy in precordial electrodes placement and to obtain consistent intra- and inter-observer results for avoiding wrong diagnosis, an interactive image-processing software has...
Lecture Notes in Computer Science, 2012
Currently a large number of user-generated videos are produced on a daily basis. It is further in... more Currently a large number of user-generated videos are produced on a daily basis. It is further increasingly common to combine videos with a variety of meta-data that increase their usefulness. In our prior work we have created a framework for integrated, sensor-rich video acquisition (with one instantiation implemented in the form of smartphone applications) which associates a continuous stream of location and direction information with the acquired videos, hence allowing them to be expressed and manipulated as spatio-temporal objects. In this study we propose a novel multi-level grid-index and a number of related query types that facilitate application access to such augmented, large-scale video repositories. Specifically our grid-index is designed to allow fast access based on a bounded radius and viewing direction-two criteria that are important in many applications that use videos. We present performance results with a comparison to a multi-dimensional R-tree implementation and show that our approach can provide significant speed improvements of at least 30%, considering a mix of queries.
Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems - GIS '10, 2010
Recently various sensors, such as GPS and compass devices, can be cost-effectively manufactured a... more Recently various sensors, such as GPS and compass devices, can be cost-effectively manufactured and this allows their deployment in conjunction with mobile video cameras. Hence, recorded clips can automatically be annotated with geospatial information and the resulting georeferenced videos may be used in various Geographic Information System (GIS) applications. However, the research community is lacking large-scale and realistic test datasets of such sensor-fused information to evaluate their techniques since collecting realworld test data requires considerable time and effort. To fill this void, we propose an approach for generating synthetic video meta-data with realistic geospatial properties for mobile video management research. We highlight the essential aspects of the georeferenced video meta-data and present an approach to simulate the behavioral patterns of mobile cameras in the synthetic data. The data generation process can be customized through user parameters for a variety of GIS applications that use mobile videos. We demonstrate the feasibility and applicability of the proposed approach by providing comparisons with real-world data.
Proceedings of the first annual ACM SIGMM conference on Multimedia systems - MMSys '10, 2010
Increasingly geographic properties are being associated with videos, especially those captured fr... more Increasingly geographic properties are being associated with videos, especially those captured from mobile cameras. The meta data from camera-attached sensors can be used to model the coverage area of the scene as a spatial object such that videos can be organized, indexed and searched based on their field of views (FOV). The most accurate representation of an FOV is through the geometric shape of a circular sector. However, spatial search and indexing methods are traditionally optimized for rectilinear shapes because of their simplicity. Established methods often use an approximation shape, such as a minimum bounding rectangle (MBR), to efficiently filter a large archive for possibly matching candidates. A second, refinement step is then applied to perform the time-consuming, precise matching function. MBR estimation has been successful for general spatial overlap queries, however it provides limited flexibility for georeferenced video search. In this study we propose a novel vector-based model for FOV estimation which provides a more versatile basis for georeferenced video search while providing competitive performance for the filter step. We demonstrate how the vector model can provide a unified method to perform traditional overlap queries while also enabling searches that, for example, concentrate on the vicinity of the camera's position or harness its view direction. To the best of our knowledge no comparable technique exists today.
Proceedings of the second annual ACM conference on Multimedia systems - MMSys '11, 2011
Mobile devices are increasingly popular for the versatile capture and delivery of video content. ... more Mobile devices are increasingly popular for the versatile capture and delivery of video content. However, the acquisition and transmission of large amounts of video data on mobile devices face fundamental challenges such as power and wireless bandwidth constraints. To support diverse mobile video applications, it is critical to overcome these challenges. We present a design framework that brings together several key ideas to enable energy-efficient mobile video management applications. First, we leverage off-the-shelf smartphones as mobile video sensors. Second, concurrently with video recordings we acquire geospatial sensor meta-data to describe the videos. Third, we immediately upload the metadata to a server to enable low latency video search. This last step allows for very energy-efficient transmissions, as the sensor data sets are small and the bulky video data can be uploaded on demand, if and when needed. We present the design, a simulation study, and a preliminary prototype of the proposed system. Experimental results show that our approach substantially prolongs the battery life of mobile devices while only slightly increasing the search latency.
Proceedings of the 19th ACM international conference on Multimedia - MM '11, 2011
Background and Motivation Experimental Results and Conclusion ICampus IWatch CT
Proceedings of the seventeen ACM international conference on Multimedia - MM '09, 2009
An increasing number of recorded videos are being tagged with geographic properties of the camera... more An increasing number of recorded videos are being tagged with geographic properties of the camera scenes. This metadata is of significant use for storing, indexing and searching large collections of videos. By considering video related meta-information, more relevant and precisely delimited search results can be returned. Our system implementation demonstrates a prototype of a georeferenced video search engine (GRVS) that utilizes an estimation model of a camera's viewable scene for efficient video search. For video acquisition, our system provides an automated annotation software that captures videos and their respective field of views (F OV). The acquisition software allows communitydriven data contributions to the search engine.
Multimedia Systems, 2010
Conventional video search systems, to find relevant videos, rely on textual data such as video ti... more Conventional video search systems, to find relevant videos, rely on textual data such as video titles, annotations, and text around the video. Nowadays, video recording devices such as cameras, smartphones and car blackboxes are equipped with GPS sensors and able to capture videos with spatiotemporal information such as time, location and camera direction. We call such videos georeferenced videos. This paper presents a georeferenced video retrieval system, GeoSearch, which efficiently retrieves videos containing a certain point or range in the map. To enable a fast search of georeferenced videos, GeoSearch adopts a novel data structure MBTR (Minimum Bounding Tilted Rectangle) in the leaf nodes of R-Tree. New algorithms are developed to build MBTRs from georeferenced videos and to efficiently process point and range queries on MBTRs. We demonstrate our system on real georeferenced videos, and show that, compared to previous methods, GeoSearch substantially reduces the index size and also improves the search speed for georeferenced video data. Our online demo is available at "".
Tagging facilitates video search in many social media and web applications. While manual tagging ... more Tagging facilitates video search in many social media and web applications. While manual tagging is time consuming, subjective and sometimes inaccurate, auto-tagging facilitated by content-based techniques is compute-intensive and challenging to apply across domains. We have developed a complementary system, named SRV-TAGS, to automatically generate tags for outdoor videos based on their geographic properties, to index the videos based on their generated tags and to provide textual search services. The system works with our geo-referenced video management web portal, enabling users to manage, search and watch videos.
GeoInformatica, 2013
With the wide spread of smartphones, a large number of user-generated videos are produced everyda... more With the wide spread of smartphones, a large number of user-generated videos are produced everyday. The embedded sensors, e.g., GPS and the digital compass, make it possible that videos are accessed based on their geo-properties. In our previous work, we have created a framework for integrated, sensor-rich video acquisition (with one instantiation implemented in the form of smartphone applications) which associates a continuous stream of location and viewing direction information with the collected videos, hence allowing them to be expressed and manipulated as spatio-temporal objects. These sensor meta-data are considerably smaller in size compared to the visual content and are helpful in effectively and efficiently searching for geo-tagged videos in large-scale repositories. In this study, we propose a novel three-level grid-based index structure and introduce a number of related query types, including typical spatial queries and ones based on bounded radius and viewing direction restriction. These two criteria are important in many video applications and we demonstrate the importance with a real-world dataset. Moreover, experimental results on a large-scale synthetic dataset show that our approach can provide a
ACM Transactions on Multimedia Computing, Communications, and Applications, 2008
An increasing number of novel applications produce a rich set of different data types that need t... more An increasing number of novel applications produce a rich set of different data types that need to be managed efficiently and coherently. In this article we present our experience with designing and implementing a data management infrastructure for a distributed immersive performance (DIP) application. The DIP project investigates a versatile framework for the capture, recording, and replay of video, audio, and MIDI (Musical Instrument Digital Interface) streams in an interactive environment for collaborative music performance. We are focusing on two classes of data streams that are generated within this environment. The first category consists of high-resolution isochronous media streams, namely audio and video. The second class comprises MIDI data produced by electronic instruments. MIDI event sequences are alphanumeric in nature and fall into the category of the data streams that have been of interest to data management researchers in recent years. We present our data management ...
Proceeding of the 16th ACM international conference on Multimedia - MM '08, 2008
Video sensors are becoming ubiquitous and the volume of captured video material is very large. Th... more Video sensors are becoming ubiquitous and the volume of captured video material is very large. Therefore, tools for searching video databases are indispensable. Current techniques that extract features purely based on the visual signals of a video are struggling to achieve good results. By considering video related meta-information, more relevant and precisely delimited search results can be obtained. In this study we propose a novel approach for querying videos based on the notion that the geographical location of the captured scene in addition to the location of a camera can provide valuable information and may be used as a search criterion in many applications. This study provides an estimation model of the viewable area of a scene for indexing and searching and reports on a prototype implementation. Among our objectives is to stimulate a discussion of these topics in the research community as information fusion of different georeferenced data sources is becoming increasingly important. Initial results illustrate the feasibility of the proposed approach.
Journal of Visual Communication and Image Representation, 2010
User generated video content is experiencing significant growth which is expected to continue and... more User generated video content is experiencing significant growth which is expected to continue and further accelerate. As an example, users are currently uploading 20h of video per minute to YouTube. Making such video archives effectively searchable is one of the most critical challenges of multimedia management. Current search techniques that utilize signal-level content extraction from video struggle to scale.Here we
ACM Transactions on Multimedia Computing, Communications, and Applications, 2008
An increasing number of novel applications produce a rich set of different data types that need t... more An increasing number of novel applications produce a rich set of different data types that need to be managed efficiently and coherently. In this article we present our experience with designing and implementing a data management infrastructure for a distributed immersive performance (DIP) application. The DIP project investigates a versatile framework for the capture, recording, and replay of video, audio,
Uploads
Papers by Sakire Arslan Ay