Processing flows of information

The Journal of Supercomputing, 2014

In this paper, we develop the optimal minimum-energy scheduler for the adaptive joint allocation of the task sizes, computing rates, communication rates and communication powers in virtualized networked data centers (VNetDCs) that operate under hard per-job delay-constraints. The considered VNetDC platform works at the Middleware layer of the underlying protocol stack. It aims at supporting real-time stream service (such as, for example, the emerging big data stream computing (BDSC) services) by adopting the software-as-a-service (SaaS) computing model. Our objective is the minimization of the overall computing-plus-communication energy consumption. The main new contributions of the paper are the following ones: (i) the computing-plus-communication resources are jointly allotted in an adaptive fashion by accounting in real-time for both the (possibly, unpredictable) time fluctuations of the offered workload and the reconfiguration costs of the considered VNetDC platform; (ii) hard per-job delay-constraints on the overall allowed computing-pluscommunication latencies are enforced; and, (iii) to deal with the inherently nonconvex nature of the resulting resource optimization problem, a novel solving approach is developed, that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The sensitivity of the energy consumption of Energy-efficient adaptive networked datacenters A typical VNetDC architecture for real-time stream processing . It operates at the middleware layer and atop the virtualization layer. Black boxes indicate virtual network interface cards (VNICs), which terminate end-to-end TCP-based connections Due to the aforementioned features, at the best of the authors' knowledge, the afforded problem covers a still unexplored research topic and this is confirmed by some recent position papers .

Communications in Computer and Information Science, 2014

In the paper complex event processing systems are considered. The survey of problems and approaches to their solutions associated with this technology is contained in the paper. Based on this survey a mathematical model for complex event processing systems is proposed. Samples of such a model are called a CEP-machines. Authors have demonstrated that this model is closely related with the notion of an pre-automaton introduced in their earlier papers. The proposed model provides rigorous formulations of problems associated with complex event processing. Authors have proved that processing ability of such systems is determined by some compromise between the class of processed event stream and a number of complex events. Further in the paper the problem of CEP-machine synthesis basing on the formal specification has been solved. Authors have established the conditions ensuring possibility of algorithmic realisation for CEP-machines.

International Journal of Production Research, 2016

Future manufacturing systems need to be more flexible, to embrace tougher and constantly changing market demands. They also need to make better use of plant data, ideally utilizing all data from the entire plant. Low-level data should be refined to real-time information for decision making, to facilitate competitiveness through informed and timely decisions. The Line Information System Architecture, LISA, is designed to enable flexible factory integration and data utilization. In LISA, international standards and established off-the-shelf technologies have been combined with the main objective to be industrially applicable. LISA is an event-driven architecture with a prototype-oriented information model and formalized transformation services. It features loose coupling, flexibility, and ease of retrofitting legacy devices. The architecture has been evaluated on both real industrial data and industrial demonstrators and is also being installed at a large automotive company.

Big Data in Bioeconomy

To remain competitive, organizations are increasingly taking advantage of the high volumes of data produced in real time for actionable insights and operational decision-making. In this chapter, we present basic concepts in real-time analytics, their importance in today’s organizations, and their applicability to the bioeconomy domains investigated in the DataBio project. We begin by introducing key terminology for event processing, and motivation for the growing use of event processing systems, followed by a market analysis synopsis. Thereafter, we provide a high-level overview of event processing system architectures, with its main characteristics and components, followed by a survey of some of the most prominent commercial and open source tools. We then describe how we applied this technology in two of the DataBio project domains: agriculture and fishery. The devised generic pipeline for IoT data real-time processing and decision-making was successfully applied to three pilots in...

Lecture Notes in Computer Science, 2015

Thanks to the recent introduction of peer-to-peer communication between browsers with WebRTC, real time processing of streams can now be deployed on browsers in addition to traditional server-side execution environments. In this paper we present the Web Liquid Streams framework for building and executing stream processing topologies capable of gathering data from Web-enabled sensors and process it through JavaScript operators scattered across a peer-to-peer Cloud of computing peers. WLS offers i) support for arbitrary topologies and data streams, ii) deployment on heterogeneous Web-enabled devices, iii) transparent stream delivery across the WebRTC, WebSockets and ZeroMQ protocols, iv) stateful and stateless operators. WLS takes care of the deployment of the topology on the available resources, while users are only required to implement the operators and describe the topology graph using JSON. The structure of the topology can be dynamically adapted without stopping the stream flowing through it. We present the platform and its programming interface, showing a first evaluation of the system.

Knowledge and Information Systems, 2016

In crisis management systems, situational awareness is usually at the basis of guiding the intervention process, and it is required to rapidly process data acquired from information sources on the field such as sensors or even humans. Given the variety and heterogeneity of sources and the amount of information that can be collected, together with the urgency of taking decisions, such information needs to be rapidly collected, filtered and aggregated in a form that can be used in subsequent machine-assisted decision support processes. At the same time, uncertainties in the input data or approximations in the processing phase may lead to an incorrect interpretation of the real situation in progress, which may generate mismanagements and severe consequences. This paper presents an event processor for crisis management systems that combines heterogeneous input sources to detect a critical situation. Complex event processing technology is applied for correlating data and creating events that describe the critical situation. Anomaly detection techniques are then used to analyze such events and detect possible anomalies, i.e. events not pertaining to the identified critical situation. The devised event processor creates trusted events that describe a critical situation merging inputs from heterogeneous and potentially untrusted sources. A prototype of the solution has been implemented and exercised within the crisis management system developed during the Secure! project. The experimental validation activities performed make use of different input sources, such as Twitter and sensors deployed on field (a doppler radar for people detection and accelerometers for vibrations detection). The objective of the experimental campaign is to show i) the adequacy of the solution to rapidly process the information and describe the critical situation, and ii) its capability in detecting anomalous events that could impair the accuracy of the description of the critical situation.

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

There is a large variety of binary sensors in use today, and useful context-aware services can be defined using such binary sensors. However, the currently available approaches for programming context-aware services do not conveniently support binary sensors. Indeed, no existing approach simultaneously supports a notion of state, central to binary sensors, offers a complete set of operators to compose states, allows to define reusable abstractions by means of such compositions, and implements efficient online processing of these operators. This paper proposes a new language for event processing specifically targeted to binary sensors. The central contributions of this language are a native notion of state and semi-causal operators for temporal state composition including: Allen's interval relations generalized for handling multiple intervals, and temporal filters for handling delays. Compared to other approaches such as CEP (complex event processing), our language provides less ...

ACM SIGPLAN Notices

In spite of the fact that many sensors in use today are binary (i.e. produce only values of 0 and 1), and that useful context-aware applications are built exclusively on top of them, there is currently no development approach specifically targeted to binary sensors. Dealing with notions of state and state combinators, central to binary sensors, is tedious and error-prone in current approaches. For instance, developing such applications in a general programming language requires writing code to process events, maintain state and perform state transitions on events, manage timers and/or event histories. In another paper, we introduced a domain specific language (DSL) called Allen, specifically targeted to binary sensors. Allen natively expresses states and state combinations, and detects contexts on line, on incoming streams of binary events. Expressing state combinations in Allen is natural and intuitive due to a key ingredient: semi-causal operators. That paper focused on the concept of the language and its main operators, but did not address its implementation challenges. Indeed, online evaluation of expressions containing semi-causal operators is difficult, because semicausal sub-expressions may block waiting for future events, thus generating unknown values, besides 0 and 1. These unknown values may or may not propagate to the containing expressions, depending on the current value of the other arguments. This paper presents a compiler and runtime for the Allen language, and shows how they implement its state combining operators, based on reducing complex expressions to a core subset of operators, which are implemented natively. We define several assisted living applications both in Allen and in a general scripting language. We show that the former are much more concise in Allen, achieve more effective code reuse, and ease the checking of some domain properties. Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Lecture Notes in Computer Science, 2018

Many ICT applications need to make sense of large volumes of streaming data to detect situations of interest and enable timely reactions. The Stream Reasoning (SR) domain aims to combine the performance of stream/event processing and the reasoning expressiveness of knowledge representation systems by adopting Semantic Web standards to represent streaming elements. In this paper, we argue that the mainstream SR model is not flexible enough to properly express the temporal relations common in many applications. We show that the model can miss relevant information and lead to inconsistent derivations. Moving from these premises, we introduce a novel SR model that provides expressive ontological and temporal reasoning by neatly decoupling their scope to avoid information losses and inconsistencies. We implement the model in the DOTR system that defines ontological reasoning using Datalog rules and temporal reasoning using the TESLA Complex Event Processing language, which builds on metric temporal logic. We demonstrate the expressiveness of our model through various examples and benchmarks, and we show that DOTR outperforms state-of-the-art SR tools, processing data with millisecond latency.

Business & Information Systems Engineering, 2020

Business Process Management is a boundaryspanning discipline that aligns operational capabilities and technology to design and manage business processes. The Digital Transformation has enabled human actors, information systems, and smart products to interact with each other via multiple digital channels. The emergence of this hyper-connected world greatly leverages the prospects of business processes-but also boosts their complexity to a new level. We need to discuss how the BPM discipline can find new ways for identifying, analyzing, designing, implementing, executing, and monitoring business processes. In this research note, selected transformative trends are explored and their impact on current theories and IT artifacts in the BPM discipline is discussed to stimulate transformative thinking and prospective research in this field.

Journal of Big Data, 2017

Several modern data-intensive applications need to process large volumes of data on the fly as they are produced. Examples range from credit card fraud detection systems, which analyze massive streams of credit card transactions to identify suspicious patterns, to environmental monitoring applications that continuously analyze sensor data, to click stream analysis of Web sites that identify frequent patterns of interactions. More in general, stream processing is a central requirement in today's information systems. This state of facts pushed the development of several stream processing engines (SPEs) that continuously analyze streams of data to produce new results as new elements enter the streams. Unfortunately, existing SPEs adopt different processing models and standardized execution semantics have not yet emerged. This severely hampers the usability

Web Engineering, 2018

A critical issue in the Web of Things (WoT) is the need to process and analyze the interactions of Web-interconnected real-world objects. Complex Event Processing (CEP) is a powerful technology for analyzing streams of information about real-time distributed events, coming from different sources, and for extracting conclusions from them. However, in many situations these events are not free from uncertainty, due to either unreliable data sources and networks, measurement uncertainty, or to the inability to determine whether an event has actually happened or not. This short research paper discusses how CEP systems can incorporate different kinds of uncertainty, both in the events and in the rules. A case study is used to validate the proposal, and we discuss the benefits and limitations of this CEP extension.

Lecture Notes on Data Engineering and Communications Technologies, 2017

One important class of applications for the Internet of Things is related to the need to gain timely and continuous situational awareness, like smart cities, automated traffic control, or emergency and rescue operations. Events happening in the real-world need to be detected in real-time based on sensor data and other data sources. Complex Event Processing (CEP) is a technology to detect complex (or composite) events in data streams and has been successfully applied in high volume and high velocity applications like stock market analysis. However, these application domains faced only the challenge of high performance, while the Internet of Things and Mobile Big Data introduce a new set of challenges caused by mobility. This chapter aims to explain these challenges and to give an overview on how they are solved respectively how far state-of-the-art research has advanced to be useful to solve Mobile Big Data problems. At the infrastructure level the main challenge is to trade performance against resource consumption and energy efficiency and operator placement is the most dominant mechanism to address these problems. At the application and consumer level, mobile queries pose a new set of challenges for CEP related to continuously changing positions of consumers and data sources, and the need to adapt the query processing to these changes. Finally, proper methods and tools for systematical testing and reproducible performance evaluation for mobile distributed CEP are needed but not yet available.

Semantic Web

The field of Complex Event Processing (CEP) relates to the techniques and tools developed to efficiently process pattern-based queries over data streams. The Semantic Web, through its standards and technologies, is in constant pursue to provide solutions for such paradigm while employing the RDF data model. The integration of Semantic Web technologies in this context can handle the heterogeneity, integration and interpretation of data streams at semantic level. In this paper, we propose and implement a new query language, called SPA , that extends SPARQL with new Semantic Complex Event Processing (SCEP) operators that can be evaluated over RDF graph-based events. The novelties of SPA includes (i) the separation of general graph pattern matching constructs and temporal operators; (ii) the support for RDF graph-based events and multiple RDF graph streams; and (iii) the expressibility of temporal operators such as Kleene+, conjunction, disjunction and event selection strategies; and (iv) the operators to integrate background information and streaming RDF graph streams. Hence, SPA enjoys good expressiveness compared with the existing solutions. Furthermore, we provide an efficient implementation of SPA using a non-deterministic automata (NFA) model for an efficient evaluation of the SPA queries. We provide the syntax and semantics of SPA and based on this, we show how it can be implemented in an efficient manner. Moreover, we also present an experimental evaluation of its performance, showing that it improves over state-of-the-art approaches.

Future Internet

The amalgamation of Vehicular Ad hoc Network (VANET) with the Internet of Things (IoT) leads to the concept of the Internet of Vehicles (IoV). IoV forms a solid backbone for Intelligent Transportation Systems (ITS), which paves the way for technologies that better explain about traffic efficiency and their management applications. IoV architecture is seen as a big player in different areas such as the automobile industry, research organizations, smart cities and intelligent transportation for various commercial and scientific applications. However, as VANET is vulnerable to various types of security attacks, the IoV structure should ensure security and efficient performance for vehicular communications. To address these issues, in this article, an authentication-based protocol (A-MAC) for smart vehicular communication is proposed along with a novel framework towards an IoV architecture model. The scheme requires hash operations and uses cryptographic concepts to transfer messages be...

The Journal of Supercomputing, 2016

Time-to-solution is an important metric when parallelizing existing code. The REPARA approach provides a systematic way to instantiate stream and data parallel patterns by annotating the sequential source code with C++11 attributes. Annotations are automatically transformed in a target parallel code that uses existing libraries for parallel programming (e.g., FastFlow). In this paper we apply this approach for the parallelization of a data stream processing application. The description shows the effectiveness of the approach in easily and quickly prototyping several parallel variants of the sequential code by obtaining good overall performance in terms of both throughput and latency. Keywords Code Annotations • Parallel Patterns • Data Stream Processing. 1 Introduction Data Stream Processing [2] (briefly, DaSP) is a paradigm that enables the computation on unbounded data streams coming from various sources (e.g., sensors, social media and network devices). Streaming data are not stored persistently in memory and then computed; rather they are processed "onthe-fly" by continuous queries whose operators run constantly over time in order to produce outputs in a continuous way. The goal is to extract greater knowledge from the data by maintaining a feasible history of the stream, and to detect recurrent patterns by generating complex events in real-time. The parallelization of stateful operators, in which the stream history is maintained in succinct data structures (e.g., synopses, sketches and wavlets) or, more often, in windows containing the most recent data [9], has received

Journal of Intelligent Information Systems, 2017

In-depth analysis of user interactions with applications in large systems is widely adopted as a means to understand user's behavior for strategic purposes such as fraud detection, system security, weblog analysis, social networking, and customer relationship management. Overall, the user behavior presents characteristics, relationships, structures, and effects of a sequence of actions in a specific application domain. The interaction of users with applications at the business-level generates events that make the elements of the user behavior. Formal modelling and representation of complex patterns of user actions using expressive languages are critical aspects of behavior analysis. We present a model to describe the behavior elements and their relationships. The model also provides a systematic mechanism for describing and presenting events, sequence of events, and complex behavior patterns. A behavior pattern can be defined as a sequence of typed events that occur during specific time intervals. An event consists of a tuple of attributes whose values represent an observation of the behavior. In this paper, first we define a semantic model of the user behavior to address the issues around the user behavior representation, and then we present syntax and semantics of a generic Behavior Pattern Language (BPL), which enables the analysts to define a variety of complex behavior patterns in a declarative manner. We present the feasibility of the approach through several examples of complex behavior patterns expressed using the proposed language.

Proceedings of the 2020 International Conference on Multimedia Retrieval

The Internet of Multimedia Things (IoMT) is an emerging concept due to the large amount of multimedia data produced by sensing devices. Existing event-based systems mainly focus on scalar data, and multimedia event-based solutions are domain-specific. Multiple applications may require handling of numerous known/unknown concepts which may belong to the same/different domains with an unbounded vocabulary. Although deep neural network-based techniques are effective for image recognition, the limitation of having to train classifiers for unseen concepts will lead to an increase in the overall response-time for users. Since it is not practical to have all trained classifiers available, it is necessary to address the problem of training of classifiers on demand for unbounded vocabulary. By exploiting transfer learning based techniques, evaluations showed that the proposed framework can answer within ∼0.01 min to ∼30 min of response-time with accuracy ranges from 95.14% to 98.53%, even when all subscriptions are new/unknown. CCS CONCEPTS • Information systems → Multimedia streaming; • Computing methodologies → Neural networks; • Software and its engineering → Publish-subscribe / event-based architectures.

Multimedia Tools and Applications

The enormous growth of multimedia content in the field of the Internet of Things (IoT) leads to the challenge of processing multimedia streams in real-time. Event-based systems are constructed to process event streams. They cannot natively consume multimedia event types produced by the Internet of Multimedia Things (IoMT) generated data to answer multimedia-based user subscriptions. Machine learning-based techniques have enabled rapid progress in solving real-world problems and need to be optimised for the low response time of the multimedia event processing paradigm. In this paper, we describe a classifier construction approach for the training of online classifiers, that can handle dynamic subscriptions with low response time and provide reasonable accuracy for the multimedia event processing. We find that the current object detection methods can be configured dynamically for the construction of classifiers in real-time, by tuning hyperparameters even when training from scratch. O...

Proceedings of the 2020 International Conference on Multimedia Retrieval

Event recognition is among one of the popular areas of smart cities that has attracted great attention for researchers. Since Internet of Things (IoT) is mainly focused on scalar data events, research is shifting towards the Internet of Multimedia Things (IoMT) and is still in infancy. Presently multimedia event-based solutions provide low response-time, but they are domain-specific and can handle only familiar classes (bounded vocabulary). However multiple applications within smart cities may require processing of numerous familiar as well as unseen concepts (unbounded vocabulary) in the form of subscriptions. Deep neural network-based techniques are popular for image recognition, but have the limitation of training of classifiers for unseen concepts as well as the requirement of annotated bounding boxes with images. In this work, we explore the problem of training of classifiers for unseen/unknown classes while reducing response-time of multimedia event processing (specifically object detection). We proposed two domain adaptation based models while leveraging Transfer Learning (TL) and Large Scale Detection through Adaptation (LSDA). The preliminary results show that proposed framework can achieve 0.5 mAP (mean Average Precision) within 30 min of response-time for unseen concepts. We expect to improve it further using modified LSDA while applying fastest classification (MobileNet) and detection (YOLOv3) network, along with elimination of requirement of annotated bounding boxes. CCS CONCEPTS • Information systems → Multimedia streaming; • Computing methodologies → Neural networks; • Software and its engineering → Publish-subscribe / event-based architectures.

International Journal of Distributed Sensor Networks, 2015

Real time environmental monitoring (hereafter, RTEM) is crucial for observing and studying dynamic, rare, or abrupt phenomena in the environment. Conventional RTEM systems are developed in a domain-specific way and not based on a well-defined distributed system model. Therefore, it is challenging to develop such RTEM systems, to extend them to support new requirements, and to integrate them with other application systems. In this paper, we raise challenging issues in RTEM, propose a generic distributed system model for RTEM to address those issues, and present the Galilee middleware system for RTEM based on the model. The distributed system model is based on the concept of data streams. It is intended to simplify and to conceptualize the design of RTEM systems. Therefore, the simple data stream-based distributed system model facilitates the development of general RTEM middleware system. The Galilee middleware system for RTEM is based on the distributed system model and designed to s...

Sensors (Basel, Switzerland), 2017

Context aware systems are able to adapt their behavior according to the environment in which the user is. They can be integrated into an Internet of Things (IoT) infrastructure, allowing a better perception of the user's physical environment by collecting context data from sensors embedded in devices known as smart objects. An IoT extension called the Internet of Mobile Things (IoMT) suggests new scenarios in which smart objects and IoT gateways can move autonomously or be moved easily. In a comprehensive view, Quality of Context (QoC) is a term that can express quality requirements of context aware applications. These requirements can be those related to the quality of information provided by the sensors (e.g., accuracy, resolution, age, validity time) or those referring to the quality of the data distribution service (e.g, reliability, delay, delivery time). Some functionalities of context aware applications and/or decision-making processes of these applications and their user...

International Journal of System Assurance Engineering and Management, 2019

We are living in a world which is connected to the internet, and as a result, a significant amount of data is being generated which can be processed using efficient methods for the technology development for the mankind. This paper focuses on the implementation of the HUT architecture for analyzing the IoT data, for the pollution control of a smart city. We propose our solution in a realworld smart city use case by obtaining the correlation between different types of gases responsible for pollution and further predicting the solutions for the prevention of pollution in real time.

Lecture Notes in Computer Science, 2015

Detection rules represent one of the components of the rule models in event processing systems. These rules can be discovered from data using data mining techniques or domain experts' knowledge. We demonstrate a system that provides its users the means for creating and validating such rules. The system is applied on real-life environmental scenarios, where the main source of data comes from sensors. Based on historical data about events of interest, the scope is to formulate rules that could have caused these events. Using a scalable infrastructure the rules can be tested on massive amount of data in order to observe how past events would fit to these rules. In addition, we create semantic annotations of the dataset and use them in the system outputs in order to support interoperability with other systems.

Proceedings of the 33rd Annual ACM Symposium on Applied Computing, 2018

Scientific Programming, 2019

Complex event processing (CEP) is a computational intelligence technology capable of analyzing big data streams for event pattern recognition in real time. In particular, this technology is vastly useful for analyzing multicriteria conditions in a pattern, which will trigger alerts (complex events) upon their fulfillment. However, one of the main challenges to be faced by CEP is how to define the quantitative analysis to be performed in response to the produced complex events. In this paper, we propose the use of the MEdit4CEP-CPN model-driven tool as a solution for conducting such quantitative analysis of events of interest for an application domain, without requiring knowledge of any scientific programming language for implementing the pattern conditions. Precisely, MEdit4CEP-CPN facilitates domain experts to graphically model event patterns, transform them into a Prioritized Colored Petri Net (PCPN) model, modify its initial marking depending on the application scenario, and make...

International Journal of Adaptive, Resilient and Autonomic Systems, 2014

Several new applications of mobile computing environments, such as Intelligent Transportation Systems, Fleet Management and Logistics, and integrated Industrial Process Automation share the requirement of remote monitoring and high performance processing of huge data streams produced by large sets of mobile nodes. Two key requirements for the deployment and operation of such mobile infrastructures are the handling of large and variable numbers of wireless connections to the monitored mobile nodes regardless of their current use or locations, and to automatically adapt to variations in the volume of the mobile data streams. This article describes the design, implementation, and evaluation of an autonomic mechanism for load balancing of mobile data streams. The autonomic capability has been incorporated into a scalable middleware system based on a Data Centric Publish Subscribe approach using the OMG Data Distribution Service (DDS) standard and aimed at real-time and adaptive handling...

Granular Computing, 2020

The execution of processes in companies generates traces of event data, stored in the underlying information system(s), capturing the actual execution of the process. Analyzing event data, i.e., the focus of process mining, yields a detailed understanding of the process, e.g., we are able to discover the control flow of the process and detect compliance and performance issues. Most process mining techniques assume that the event data are of the same and/or appropriate level of granularity. However, in practice, the data are extracted from different systems, e.g., systems for customer relationship management, Enterprise Resource Planning, etc., record the events at different granularity levels. Hence, pre-processing techniques that allow us to abstract event data into the right level of granularity are vital for the successful application of process mining. In this paper, we present a literature study, in which we assess the state-of-the-art in the application of such event abstracti...

Neural Computing and Applications, 2018

Air pollution generated by road traffic in large cities is a great concern in today's society since pollution has an important impact on human health, even causing premature deaths. To address the problem, this paper presents an Intelligent Transportation System model based on Complex Event Processing technology and Colored Petri Nets (CPNs). It takes into consideration the levels of environmental pollution and road traffic, according to the air quality levels accepted by the international recommendations as well as the handbook emission factors for road transport methodology. This proposal, therefore, tackles a common problem in today's large cities, where traffic restrictions must be applied due to environmental pollution. CPNs are used in this work as a tool to make decisions about traffic regulations, so as to reduce pollution levels.

Proceedings of the 5th ACM/SPEC international conference on Performance engineering, 2014

Collections of network traces have long been used in network traffic analysis. Flow analysis can be used in network anomaly discovery, intrusion detection and more generally, discovery of actionable events on the network. The data collected during processing may be also used for prediction and avoidance of traffic congestion, network capacity planning, and the development of software-defined networking rules. As network flow rates increase and new network technologies are introduced on existing hardware platforms, many organizations find themselves either technically or financially unable to generate, collect, and/or analyze network flow data. The continued rapid growth of network trace data, requires new methods of scalable data collection and analysis. We report on our deployment of a system designed and implemented at the University of Kentucky that supports analysis of network traffic across the enterprise. Our system addresses problems of scale in existing systems, by using distributed computing methodologies, and is based on a combination of stream and batch processing techniques. In addition to collection, stream processing using Storm is utilized to enrich the data stream with ephemeral environment data. Enriched stream-data is then used for event detection and near real-time flow analysis by an in-line complex event processor. Batch processing is performed by the Hadoop MapReduce framework, from data stored in HBase BigTable storage. In benchmarks on our 10 node cluster, using actual network data, we were able to stream process over 315k flows/sec. In batch analysis were we able to process over 2.6M flows/sec with a storage compression ratio of 6.7:1.

Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems

Evaluation of Distributed Complex Event Processing (CEP) systems is a rather challenging task. To simplify this task, we developed the open simulation framework for Distributed CEP, called DCEP-Sim. The goal of this tutorial is to facilitate the process of using DCEP-Sim. Since DCEP-Sim is designed and implemented in the popular network simulator ns-3 we introduce the most important concepts of ns-3. Simulations in ns-3 are congured and executed though a main program called an ns-3 script. We use a simple example script to explain how simulations with DCEP-Sim are set up and executed. To give an idea how DCEP-Sim can be adjusted to particular needs, we explain how DCEP-Sim can be adapted (e.g., through changing the workload and the network topology) and how new Distributed CEP solutions can be added by explaining how to add a new operator to DCEP-Sim. CCS CONCEPTS • Computing methodologies → Simulation tools;

Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems

Distributed Complex Event Processing (CEP) is gaining increasing interest for two reasons: (1) to scale system performance to handle higher workloads in real-time, and (2) to perform in-network processing, e.g., in mobile networks to reduce the amount of data that has to be transferred through the network. System scalability and the complexity of mobile systems are some of the major challenges when evaluating the performance of new Distributed CEP solutions. We propose an open framework for distributed CEP (DCEP-Sim) built on a well-established network simulator, i.e, ns-3. e design of DCEP-Sim is based on the engineering principles of separation of concerns and the separation of mechanisms and policies. By leveraging the ns-3 feature of object aggregation it is very easy to add new policies, e.g., placement or selection policies, and evaluate them without changing anything else in the DCEP-Sim. e fact that ns-3 includes many accurate network models implies that Distributed CEP simulation with DCEP-Sim will also be much more accurate than ad-hoc handcra ed simulations. We demonstrate in a use case how easy it is to con gure performance evaluation experiments and we perform experiments to con rm that the integration of the Distributed CEP in ns-3 is good foundation for large-scale experiments. e evaluation results demonstrate that DCEP-Sim substantially reduces the e ort and costs of Distributed CEP evaluation.

Journal of Ambient Intelligence and Humanized Computing, 2016

Securing our built and natural environment against natural disasters is one of the central elements of the functioning of any society. All sectors are to some extent impacted by disasters and concerned with related resilience and security issues. The general objective is to reduce the loss of human life, environmental, economic and material damage from natural and man-made disasters, including extreme weather events, crime and terrorism threats: the strategies rely on assessing risks, forecasting extreme events, communication and alerting and deploying resources (human, technological and financials) in different domains in optimal ways. Communication and computing networks are not only critical infrastructures on their own, but underpin many other critical networks (e.g. energy, transport, health,…). This is particularly true during nature or man-made emergencies, where the malfunctioning or disruption of the communication channel or of an IT system can have a cascading effect on several other infrastructures or services. The vulnerability of critical infrastructures, including the communication networks, stems, to a large extent, from the fact that ICT systems are deployed in an environment or for an application that it was not designed with resilience as primary concern. The deployment of ICT in new critical systems is exacerbating the problem by introducing new risks and vulnerabilities, in particular for a multisensory interconnected system. Security monitoring of urban areas is one of the main requests from citizens around the world. This was evidenced by the past series of terrorist attacks against New York's World Trade Center in 2001, Madrid' s train system in 2004, London Underground in 2007, Moscow Metro in 2010 and Paris in 2015. These incidents have exposed vulnerabilities of urban environments against terrorist actions, which mainly stem from their diversity, heterogeneity and complexity. The challenges of the urban environment are unique with particular and very specific issues. In order to cope with the complexity of operations in the urban environments, security and defense agencies have been increasingly turning in the last years to pervasive multi-sensory technologies for enhancing their ability to acquire, analyze and visualize events and situations. However, numerous challenges are associated with such technologies. The large-scale nature of both the geographically dispersed environment and the volume of the data, the multiple distributed heterogeneous components that need to be assembled spanning sensors, sensor processing, signal processing components often from multiple vendors, are some of the issues that need to be addressed. Moreover, we need to take in consideration that some activities need to be automated (i.e. video analysis since manual observation of multiple camera is not possible, anomaly detection ande high-level intelligent reasoning for event inference), as these are features without which the benefit of the system is limited.

Concurrency and Computation: Practice and Experience, 2016

Ambient Assisted Living (AAL) main goal is the development of health monitoring systems for patients with chronic diseases and elderly people through the use of body, home and environmental sensors that increase their degree of independence and mobility. A comprehensive software infrastructure for AAL systems should be able to cover scenarios involving several patient mobility levels, locations, and physical and cognitive abilities. Cloud computing can provide to AAL systems the ability to extend the limited processing power of mobile devices, but its main role is to integrate all stakeholders through the storage and processing of health data and the orchestration of healthcare business logic. On the other hand, the Internet of Things (IoT) provides the ability to connect sensors and actuators, integrating and making them available through the Internet. This paper presents the Mobile-Hub/SDDL, a middleware for AAL based on cloud computing and IoT. We discuss how this middleware can handle the requirements of the main health monitoring scenarios and present results that demonstrate the ability to opportunistically discover and connect with sensors in a timely manner and the scalability necessary for handling the connection and data processing of many connected patients.

Future Generation Computer Systems, 2020

In the current scenario, IoT is an ideal and novel technology, which fulfills the needs of most of the commercial, non-commercial, government, and private organizations by its real-time supportive nature and characteristics. However, real-time processing itself a very critical research topic. But, most of the IoT applications are empowered by real-time data processing. Thus, it become a vital part of IoT. In this work, we proposed a four-layer infrastructure for IoT along with stream processing. Further, we use stream processing techniques along with IoT infrastructure for applications and analyze the performance of stream processing techniques for IoT applications. Also, we compare and find the five most suitable distributed stream processing systems for IoT, based on its performance and characteristics. We use two benchmark applications to evaluate the performance of distributed stream processing systems against response time, throughput, jitter, and scalability. Based on that, we suggest the adapted solution for IoT applications. We evaluate the performance with peak stream rates from 100k to 1M along with the various frequencies of benchmark applications. Further, on the basis of results, we conclude that Apache NiFi is the most suitable solution for IoT applications.

Human-Centered Software Engineering, 2018

In this paper, we present a data-driven approach to enable the creation of evidence-based usability test scenarios. By utilising product usage data to create usability test scenarios, we aim to improve the reliability of the test results and to provide better insights into product usability. The approach consists of four elements: the collection of product usage data, the transformation of these data into logs of user activities, the creation of models of user behaviour, and the guided creation of usability test scenarios based on the models. We discuss the challenges that can be encountered when applying this approach based on our experiences with two case studies in product development. We have created a prototype scenario planning tool and performed a preliminary evaluation of the tool with usability engineers working at Philips Healthcare. The evaluation shows that tool-supported evidence-based usability test creation would be valuable in their daily work.

Proceedings of the Second International Conference on Internet-of-Things Design and Implementation

New generations of cloud applications are increasingly complex and pose lower latency requirements. e la er is forcing the industry to reduce network latency by adding computation nodes near the edge of the network, also known as Fog Computing. To utilize the Fog nodes e ciently, the dynamic placement and migration of application components must be supported. To this end, a Fogaware application programming and deployment framework, called Foglets, has been proposed. is demonstration shows how the Foglets framework can be easily used to deploy applications in the Fog Computing infrastructure: A video streaming application allows a moving user to continue watching a video on the closest available screen. CCS CONCEPTS •Computer systems organization →Distributed architectures; Sensor networks;

Sensors, 2019

Smart cities are urban environments where Internet of Things (IoT) devices provide a continuous source of data about urban phenomena such as traffic and air pollution. The exploitation of the spatial properties of data enables situation and context awareness. However, the integration and analysis of data from IoT sensing devices remain a crucial challenge for the development of IoT applications in smart cities. Existing approaches provide no or limited ability to perform spatial data analysis, even when spatial information plays a significant role in decision making across many disciplines. This work proposes a generic approach to enabling spatiotemporal capabilities in information services for smart cities. We adopted a multidisciplinary approach to achieving data integration and real-time processing, and developed a reference architecture for the development of event-driven applications. This type of applications seamlessly integrates IoT sensing devices, complex event processing,...

Proceedings of the 9th ACM SIGPLAN International Workshop on Reactive and Event-Based Languages and Systems

Functional reactive programming (FRP) languages allow programmers to write declarative expressions to denote eventstreams. They feature operators (e.g., map, filter, merge, ...) to build compositions of events. Among these operators we would expect to find the traditional Boolean operators to express complex events. To some extent this is true. However, a remarkable absent in existing FRP languages is the logical conjunction, i.e., the and operator. In this paper, we describe the difficulty of introducing logical conjunction in FRP. We then present a new experimental language called FLOREnce that reconciles functional reactive programming with 'logic reactive programming' abstractions. The latter emerge naturally when adding a reactive conjunction operator to an event-based FRP language. We present a prototypical implementation of FLOREnce and we study its expressiveness by conducting an experiment. CCS Concepts: • Software and its engineering → Integrated and visual development environments; • Humancentered computing → Visualization systems and tools.

Annals of Mathematics and Artificial Intelligence, 2016

In this paper we study the coordination of Emergency Medical Service (EMS) for patients with acute myocardial infarction with ST-segment elevation (STEMI). This is a health problem with high associated mortality. A "golden standard" treatment for STEMI is angioplasty, which requires a catheterization lab and a highly qualified cardiology team. It should be performed as soon as possible since the delay to treatment worsens the patient's prognosis. The decrease of the delay is achieved by coordination of EMS, which is especially important in the case of multiple simultaneous patients. Nowadays, this process is based on the First-Come-First-Served (FCFS) principle and it heavily depends on human control and phone communication with high proneness to human error and delays. The objective is, therefore, to automate the EMS coordination while minimizing the time from symptom onset to reperfusion and thus to lower the mortality and morbidity resulting from this disease. In this paper, we present a multi-agent decision-support system for the distributed coordination of EMS focusing on urgent out-of-hospital STEMI patients awaiting angioplasty. The system is also applicable to emergency patients of any pathology needing pre-hospital acute medical care and urgent hospital treatment. The assignment of patients to ambulances and angioplasty-enabled hospitals with cardiology teams is performed via a three-level optimization model. At each level, we find a globally efficient solution by a modification of the distributed relaxation method for the assignment problem called the auction algorithm. The efficiency of the proposed model is demonstrated by simulation experiments. Keywords EMS Coordination • Ambulance Coordination • Angioplasty • PCI • Distributed Optimization • Auction Algorithm • Emergency Medical Assistance 1 Introduction Based on the World Health Organization data, ischemic heart disease (IHD) is the single most frequent cause of death killing 7.4 million people in 2012, which is 13.2% of all the deaths worldwide [43]. It is a disease characterized by ischaemia (reduced blood supply) of the heart muscle, usually due to coronary artery disease. At any stage of coronary artery disease, the acute rupture of an atheromatous plaque may lead to an acute myocardial infarction (AMI), also called a heart attack [45].

Sensors, 2021

The Internet of Things (IoT) has emerged from the proliferation of mobile devices and objects connected, resulting in the acquisition of periodic event flows from different devices and sensors. However, such sensors and devices can be faulty or affected by failures, have poor calibration, and produce inaccurate data and uncertain event flows in IoT applications. A prominent technique for analyzing event flows is Complex Event Processing (CEP). Uncertainty in CEP is usually observed in primitive events (i.e., sensor readings) and rules that derive complex events (i.e., high-level situations). In this paper, we investigate the identification and treatment of uncertainty in CEP-based IoT applications. We propose the DST-CEP, an approach that uses the Dempster–Shafer Theory to treat uncertainties. By using this theory, our solution can combine unreliable sensor data in conflicting situations and detect correct results. DST-CEP has an architectural model for treating uncertainty in event...

Future Generation Computer Systems, 2020

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACM Transactions on Autonomous and Adaptive Systems, 2016

The use of Complex Event Processing (CEP) and Stream Processing (SP) systems to process high-volume, high-velocity Big Data has renewed interest in procedures for managing these systems. In particular, self-management and adaptation of runtime platforms have been common research themes, as most of these systems run under dynamic conditions. Nevertheless, the research landscape in this area is still young and fragmented. Most research is performed in the context of specific systems, and it is difficult to generalize the results obtained to other contexts. To enable generic and reusable CEP/SP system management procedures and self-management policies, this research introduces the Attributed Graph Rewriting for Complex Event Processing Management ( AGeCEP ) formalism. AGeCEP represents queries in a language- and technology-agnostic fashion using attributed graphs. Query reconfiguration capabilities are expressed through standardized attributes, which are defined based on a novel classi...

Future Internet

This paper focuses on Internet of Things (IoT) architectures and knowledge generation out of streams of events as the primary elements concerning the creation of user-centric IoT services. We provide a general, symmetrical IoT architecture, which enables two-way bidirectional communication between things and users within an application domain. We focus on two main components of the architecture (i.e., Event Engine and Process Engine) that handle event transformation by implementing parametric Complex Event Processing (CEP). More specifically, we describe and implement the transformation cycle of events starting from raw IoT data to their processing and transformation of events for calculating information that we need in an IoT-enabled application context. The implementation includes a library of composite transformations grouping the gradual and sequential steps for transforming basic IoT events into business events, which include ingestion, event splitting, and calculation of measu...