Non-intrusive transaction monitoring using system logs

Electronic commerce (EC) has remarkably reshaped today's business practices. The logic embedded in business transactions are becoming more exible and sophisticated; the number of components within these business transactions are fastly increasing; and business transactions are getting more distributed and interactive as well. These, on the other hand, make today's business applications more vulnerable to resource availability constraints and consistency problems due to failures. For example, network congestion, server overload, user absence, and other exceptions all might cause temporary or permanent bottlenecks to ongoing business work ows. Developing system support to make work ow activities more reliable, e cient, and adaptive to changes or exceptions therefore poses important yet challenging questions that call for practical solutions. The TAM project at OGI aims at developing a exible framework and algorithms that address these challenges. Comparing with other research e orts, our approach is unique in that we combine advanced transaction processing (TP) technologies with adaptive methods to guide our system design, and build the mechanisms by extending the functions of current TP systems. On the theoretical side, we have developed a Transactional Activity Model (TAM) for the speci cation and management of activities with transactional properties. TAM is a careful combination of a compositional activity model 5] with well-de ned extended transaction models (ETMs) 3] such as split/join transactions 9]. It provides simple and e ective speci cation facilities that allow business process designers to specify the behavioral composition and re nement of complex activities and a wide variety of activity interaction dependencies in a high-level and declarative way. The embedded transactional semantics give business work ows automatic shield from execution anomalies caused by concurrency or failures. TAM also provides a set of dynamic activity restructuring operations with well-de ned semantics 7, 11] that allow ongoing transactional activities to bypass execution bottlenecks or deal with failures. On the practical side, we have built a system as a prototype implementation of TAM, for e ectively monitoring and managing transactional applications. An important novel feature of our system is that it is based on the standard OLTP reference architecture 4, 2], and is built on top of a commercial OLTP product Transarc-Encina. Another feature that distinguishes our system is its support for dynamic activity restructuring as an e ective facility for activity adaptation. To our knowledge, it is the rst implementation of activity restructuring with consideration of transactional properties on OLTP systems. Our implementation continues to follow a special method called "Design for Implementation", which we carefully developed in our previous work on distributed extended transaction management 8]. The method is largely based on component technology, open implementation, program specialization, and plug-in adaptors for practical implementations. Our implementation also utilizes and leverages on the

International Journal of Enterprise Information Systems, 2015

This work focuses on issues typically encountered in organizations whose core business largely depends on ICT: continuous monitoring, continuous auditing, controlling and assessment of transactions risk. Organizations have been making efforts to implement methods and systems which enable them to increase reliability of their business and, simultaneously, to be in accordance with their organizational objectives and compliant with external regulations. Thus, this work presents and validates an innovative solution to implement Continuous Assurance services in information systems applicable to any organizational transaction, regardless of its type, dimension, business area or even its information system support technology. This last objective is pursued having as support an ontological model at an abstraction level that guarantees that independence. This research culminated with the development of a prototype and consequent results analysis, using data collected from the near-real imple...

The application of process mining techniques to real-life information systems is often challenging. Considering a Purchase to Pay (P2P) process, several case notions such as order and item are involved, interacting with each other. Therefore, creating an event log where events need to relate to a single case (i.e., process instance) leads to convergence (i.e., the duplication of an event related to different cases) and divergence (i.e., the inability to separate events within the same case) problems. To avoid such problems, object-centric event logs have been proposed, where each event can be related to different objects. These can be exploited by a new set of process mining techniques. This paper describes OCEL (Object-Centric Event Log), a generic and scalable format for the storage of object-centric event logs. The implementation of the format can use either JSON or XML, and tool support is provided.

Proceedings of the 1st International Conference on Very Large Data Bases - VLDB '75, 1975

This paper is NOT THE PUBLISHED VERSION; but the author's final, peer-reviewed manuscript. The published version may be accessed by following the link in th citation below.

2001

With the emergence of GRID environments and distributed systems with dynamic resources and varying user profiles, there is an increasing need to develop reliable tools that can effectively coordinate the requirements of an application with available computing resources. The ability to predict the behaviour of complex aggregated systems under dynamically changing workloads is particularly desirable, leading to effective resource usage and optimisation of networked systems. In an effort to explore these issues, the ...

Learning Organization, The, 2013

Purpose-The paper aims to present a solution which makes it possible to control and audit organizational transactions in real time, helping to determine the degree of reliability with which they are carried out, mitigating the organizational risk. This auditing is made at a very low level of organizational transactions executed and supported exclusively in a digital format, contrary to what happens in most monitoring of transactions, which occurs at a high level. Moreover, it describes the conceptual architecture of the solution, its components and functionalities as well as the development and technical issues which should be taken into consideration on the deployment and evaluation of the solution. Design/methodology/approach-The work follows the design science methodology. It presents the problem and motivation of the investigation, the solution design and how it is being deployed. Furthermore, it presents the expected results based on the proposed architecture and on the results which are currently being achieved with the prototype implementation. Findings-The prototype is being put into practice, thus the gathering of results and their evaluation is not yet complete. However, preliminary results are really satisfactory and very close to those expected and enumerated. Originality/value-The research contributes to a new vision of organizational auditing focused on assurance services in transactions executed and supported in a digital format in compliance with the formalisms of a business ontological model of organizational transactions.

Proceedings of the First International Conference on E-Business and Telecommunication Networks, 2004

This paper proposes an approach to monitor response time of transaction-based Internet applications and protocols that uses a passive network traffic monitoring technique and stores the resulting statistics in a management information base compatible with the SNMP architecture. The work is within the scope of the Trace platform, which provides support for high-layer protocols, services and networked applications management. The implementation of the proposed approach comprises the extension of the monitoring agent, a key component of the platform, so that it stores information related to response time and generates performance-related reports.

Proceedings of the 2000 ACM symposium on Applied computing - SAC '00, 2000

Tasks like supply chain management, design of interorganizational workflow and design of virtual organizations or consortia, require mechanisms to analyze interaction requirements spanning across autonomous organizations. While existing means of analysis would help in identifying pertinent actors and interactions among them, properties which could manifest by virtue of the interactions themselves may go undetected. In this paper, we present an approach that looks for properties of a given problem domain which manifest due to the interactions that take place in the domain. As a specific application, we use data mining to look for knowledge about emergent properties from transaction data among a set of autonomous organizations, so as to design inter-organizational workflow and logistics among the organizations.

2020

We present a common data model for representing causal events across a wide range of platforms and granularities. The model was developed for attack provenance analysis under the DARPA Transparent Computing program. The unified model successfully expresses data provenance across a range of granularities (e.g., object or byte level) and platforms (e.g., Linux and Android, BSD, and Windows). This paper describes our experience developing the common data model, the lessons learned, and performance results in controlled lab experiments.

2008 The Eighth International Conference on Quality Software, 2008

Monitoring the execution of large enterprise systems is needed to ensure that such complex systems are performing as expected. However, common techniques for monitoring, such as code instrumentation and profiling have significant performance overhead, and require access to the source code and to system experts. In this paper, we propose using execution logs to monitor the execution of applications. Unfortunately, execution logs are not designed for monitoring purposes. Each occurrence of an execution event results in a different log line, since a log line contains dynamic information which varies for each occurrence of the event. We propose an approach which abstracts log lines to a set of execution events. Our approach can handle log lines without having strict requirements on the format of a log line. Through a case study on a large enterprise application, we demonstrate that our approach performs well when abstracting execution logs for large enterprise applications. We compare our approach against the SLCT tool which is commonly used to find line patterns in logs.