Online Analytical Processing

Parallel and distributed data warehouse architectures have been evolved to support online queries on massive data in a short time. Unfortunately, the emergence of e-application has been creating extremely high volume of data that reaches to terabyte threshold. The conventional data warehouse management system is costlier in terms of storage space and processing speed and sometimes it is unable to handle such huge amount of data. As a result, there is a crucial need for the new algorithms and techniques to store and manipulate these data. In this paper, we have presented a compression-based distributed data warehouse architecture -'DMBVA' for storage of warehouse data, and support online queries efficiently. We have achieved a factor of 25-30 compression compared to SQL server data warehouse. The main computational component of data warehouse is the generation and querying on the data cube. Our algorithm -'PCVDC' generates data cube directly from the compressed form of data in parallel. The reduction in the size of data cube is a factor of 30-45 compared to existing methods. The response time has also been significantly improved. These improvements are achieved by eliminating the suffix and prefix redundancy, virtual nature of the data cube, direct addressability of compressed form of data and parallel computation. Experimental evaluation shows the improved performance over the existing systems.

Досліджено, що в діяльності компанії важлива роль приділяється інформаційному забезпеченню, оскільки підприємству важливо інформувати та забезпечувати інформаційним менеджментом підприємство. Досліджено процес прийняття управлінських рішень. Процеси та програми, які використовують закордоном та в Україні допомагають підприємству краще розвиватися в інформаційному забезпеченні. Досліджено різні міжнародні системи, що використовують у процесі прийняття рішення в інвестиційній діяльності: системи підтримки прийняття рішень DSS (Decision Support System), цільові комплексні програми, "аналіз вигід і витрат", системи оперативної аналітичної обробки OLAP (OnLine Analysis Processing). Наведено основні види інформаційних ресурсів. Запропоновано план прийняття рішення, який ґрунтується на стадіях прийняття рішення.

Real data mining analysis applications call for a framework which adequately supports knowledge discovery as a multi-step process, where the input of one mining operation can be the output of another. Previous studies, primarily focusing on fast computation of one speci c mining task at a time, ignore this vital issue. Motivated by this observation, we develop a unied model supporting all major mining and analysis tasks. Our model consists of three distinct worlds, corresponding to intensional and extensional dimensions, and to data sets. The notion of dimension is a centerpiece of the model. Equipped with hierarchies, dimensions integrate the output of seemingly dissimilar mining and analysis operations in a clean manner. We propose an algebra, called the dimension algebra, for manipulating intensional dimensions, as well as operators that serve as bridges" between the worlds. We demonstrate by examples that several real data mining processes can be captured using our model and algebra. We demonstrate the naturality of the algebra by establishing several identities. Finally, w e discuss e cient implementation of the proposed framework.

Conversational interfaces to Business Intelligence (BI) applications enable data analysis using a natural language dialog in small incremental steps. To truly unleash the power of conversational BI to democratize access to data, a system needs to provide effective and continuous support for data analysis. In this paper, we propose BI-REC, a conversational recommendation system for BI applications to help users accomplish their data analysis tasks. We define the space of data analysis in terms of BI patterns, augmented with rich semantic information extracted from the OLAP cube definition, and use graph embeddings learned using Graph-SAGE to create a compact representation of the analysis state. We propose a two-step approach to explore the search space for useful BI pattern recommendations. In the first step, we train a multi-class classifier using prior query logs to predict the next high-level actions in terms of a BI operation (e.g., Drill-Down or Roll-up) and a measure that the user is interested in. In the second step, the high-level actions are further refined into actual BI pattern recommendations using collaborative filtering. This two-step approach allows us to not only divide and conquer the huge search space, but also requires less training data. Our experimental evaluation shows that BI-REC achieves an accuracy of 83% for BI pattern recommendations and up to 2× speedup in latency of prediction compared to a state-of-the-art baseline. Our user study further shows that BI-REC provides recommendations with a precision@3 of 91.90% across several different analysis tasks.

Visual big data analytics aims at supporting big data analytics via visual metaphors, with a plethora of applications in modern settings and scenarios. In all these domains, visual big data analytics paradigms offer several advantages, among which some noticeable ones are: (i) fast knowledge understanding from big data sets; (ii) pattern and trend discovery from big data sets; (iii) entity and model discovery from big data sets; (iv) sharing insights among organizations. Among several proposals, OLAPbased visual big data analytics methodologies and tools represents a successful case of visual big data analytics frameworks, which is entirely based on OLAP analysis. In this context, an OLAP cube is typically explored with multiple aggregations selecting different subsets of cube dimensions to analyze trends or to discover unexpected results. Unfortunately, such analytic process is generally manual and fails to statistically explain results. On the basis of these considerations, in this paper we propose an innovative OLAP-shaped visual big data analytics framework that incorporates a state-of-the-art statistical technique for supporting exploration and visualization of OLAP data cubes. An experimental evaluation with a medical data set presents statistically significant results and interactive visualizations, which link risk factors and degree of disease.

Data Warehouses and OLAP (On Line Analytical Processing) technologies are dedicated to analyzing structured data issued from organizations' OLTP (On Line Transaction Processing) systems. Furthermore, in order to enhance their decision support systems, these organizations need to explore XML (eXtensible Markup Language) documents as an additional and important source of unstructured data. In this context, this paper addresses the warehousing of document-centric XML documents. More specifically, we propose a two-method approach to build Document Warehouse conceptual schemas. The first method is for the unification of XML document structures; it aims to elaborate a global and generic view for a set of XML documents belonging to the same domain. The second method is for designing multidimensional galaxy schemas for Document Warehouses. Les entrepôts de données et les technologies d'analyses en ligne OLAP («On Line Analytical Processing») sont dédiés à l'analyse des données structurées issues des systèmes OLTP («On Line Transaction Processing») des organisations. De plus, ces organisations ont besoin d'explorer des documents XML, comme une importante source additionnelle de données non structurées, à des fins de prise de décisions. Dans ce contexte, cet article s'intéresse à l'entreposage de documents XML orienté-document. Plus particulièrement, nous proposons une approche composées de deux méthodes pour la construction d'un schéma conceptuel d'un Entrepôt de Documents. La première méthode est pour l'unification des structures de documents XML ; elle vise à élaborer une vue globale et générique pour un ensemble de documents XML appartenant à un même domaine. La seconde méthode est pour la modélisation multidimensionnelle en galaxie de schémas d'Entrepôts de Documents.

Les entrepôts de données sont actuellement reconnus comme étant un composant essentiel des systèmes d'aide à la décision dans la mesure où ils offrent la meilleure réponse aux problèmes de prise de décision des différents domaines fonctionnels des organisations. Cependant, la conception et la construction des entrepôts de données demeurent une tâche très complexe, difficile à accomplir. Cette complexité est essentiellement la conséquence de l'absence de techniques et de méthodes reconnues dans le domaine. Dans ce contexte, cet article identifie différentes règles pour la conception des entrepôts de données à partir de données relationnelles et introduit une nouvelle méthode pour l'automatisation de ce processus en se fondant sur les technologies MDA et XML.

Faced with the continuous increasing amount of data and the complexity of multidimensional queries, data warehouses must be split to be stored in distributed architectures and to enhance the performance of the queries. On the other hand, XML documents promise a considerable flexibility towards storing and manipulating different types of data warehouse hierarchies. In this article, we propose a method for the fragmentation of XML data warehouses by splitting the dimension documents via an attribute-based method, and splitting the fact document based on the fragments of the dimensions.

During the constantly increasing competition, higher educational institutions try to develop strategies and apply new instruments to ensure learning and teaching quality. The education sector's decision-making process faces strict requirements to quickly process a great volume of information and data. It is important to make right and immediate decisions on the management of the education process and introduce quick decision-making support systems, which will increase the demand for academic attendance analysis, which is impossible without regular monitoring of the data. All the above-mentioned requires a comprehensive approach to the process management and appliance of modern instruments and informational technologies. The work discusses the decision-making support systems' architecture, which includes three main components: data storage server, data collection, analysis instruments, and labor market research methodologies. Based on the analysis of the received data from the monitoring of teaching and study, the data warehouse and Online Analytical Processing (OLAP) technology are used for it. OLAP cube enables us to visualize, make statistical analysis, and study data. In the decision-making process solution, the problem of achieving the qualitative compliance of the professional potential and vacancies available on the labor market has an important role. The most important approach is organizing a market-oriented, 􀅮lexible, and rearranging education system out of suggested solutions. The approach provided in work will support to education quality assurance service to identify trends, problems, achievement promptly to organize decision-making to improve the education process. The analysis based on the data received from the study and teaching monitoring is presented for which the data storage and OLAP (Online Analytical Processing) technology are applied. OLAP cube enables to visualize data, make statistical analysis and study data. The paper presents the problems related to the correspondence of professional potential and the vacancies available on the labour market during the decision-making process and solution ways. The most important thing is to organize a market-oriented, 􀅮lexible, easy re-constructible education system.

During the constantly increasing competition, higher educational institutions try to develop strategies and apply new instruments to ensure learning and teaching quality. The education sector's decision-making process faces strict requirements to quickly process a great volume of information and data. It is important to make right and immediate decisions on the management of the education process and introduce quick decision-making support systems, which will increase the demand for academic attendance analysis, which is impossible without regular monitoring of the data. All the above-mentioned requires a comprehensive approach to the process management and appliance of modern instruments and informational technologies. The work discusses the decision-making support systems' architecture, which includes three main components: data storage server, data collection, analysis instruments, and labor market research methodologies. Based on the analysis of the received data from the monitoring of teaching and study, the data warehouse and Online Analytical Processing (OLAP) technology are used for it. OLAP cube enables us to visualize, make statistical analysis, and study data. In the decision-making process solution, the problem of achieving the qualitative compliance of the professional potential and vacancies available on the labor market has an important role. The most important approach is organizing a market-oriented, 􀅮lexible, and re-arranging education system out of suggested solutions. The approach provided in work will support to education quality assurance service to identify trends, problems, achievement promptly to organize decision-making to improve the education process. The analysis based on the data received from the study and teaching monitoring is presented for which the data storage and OLAP (Online Analytical Processing) technology are applied. OLAP cube enables to visualize data, make statistical analysis and study data. The paper presents the problems related to the correspondence of professional potential and the vacancies available on the labour market during the decision-making process and solution ways. The most important thing is to organize a market-oriented, 􀅮lexible, easy re-constructible education system.

An OLAP cube is typically explored with multiple aggregations selecting different subsets of cube dimensions to analyze trends or to discover unexpected results. Unfortunately, such analytic process is generally manual and fails to statistically explain results. In this work, we propose to combine dimension lattice traversal and parametric statistical tests to identify significant metric differences between cube cells. We present a 2D interactive visualization of the OLAP cube based on a checkerboard that enables isolating and interpreting significant measure differences between two similar cuboids, which differ in one dimension and have the same values on the remaining dimensions. Cube exploration and visualization is performed by automatically generated SQL queries. An experimental evaluation with a medical data set presents statistically significant results and interactive visualizations, which link risk factors and degree of disease.

We propose a framework for efficient OLAP on information networks with a focus on the most interesting kind, the topological OLAP (called "T-OLAP"), which incurs topological changes in the underlying networks. T-OLAP operations generate new networks from the original ones by rolling up a subset of nodes chosen by certain constraint criteria. The key challenge is to efficiently compute measures for the newly generated networks and handle user queries with varied constraints. Two effective computational techniques, T-Distributiveness and T-Monotonicity are proposed to achieve efficient query processing and cube materialization. We also provide a T-OLAP query processing framework into which these techniques are weaved. To the best of our knowledge, this is the first work to give a framework study for topological OLAP on information networks. Experimental results demonstrate both the effectiveness and efficiency of our proposed framework.

Nowadays automated data collection tools and mature database technology lead to tremendous amounts of data stored in databases, data warehouses and other information repositories in business organizations. In this paper, a novel approach of developing a decision support system using on-line analytical processing (OLAP) is presented. The OLAP application is optimal for data queries that do not change data. The system database is designed to promote: heavy indexing to improve query performance, denormalisation of the database to satisfy common query requirements and improve query response times, and use of a star-snowflake schema to organize the data within the database. The existing partitioning strategy is used to partition the fact table by time into equal segments, differentsize segments and on different dimension of data warehouse, which provide the good query performance, optimize hardware performance, and simplify the management of the data warehouse by reducing the volume of data to satisfy a query. Entity-relation modeling is used to create a single complex model, which proven effective in creating efficient online transaction processing systems. In this system, cubes are used to organize and summarize data for efficient analytical querying. In addition, an enterprise information system model has also been presented in this paper in order to optimize the utilization of operational data for use strategically. The proposed system is evaluated on the cube data for the Rajshahi and Khulna regions for ten years from 1994 to 2003. Experimental results indicate the effectiveness of the proposed OLAP based Decision Support System.

A tool which algorithmically traces the effectiveness of the text files would be helpful in determining whether the text file have all the characteristic of important concepts. Every text source is build up on key phrases, and these paramount phrases follow a certain grammatical linguistic pattern widely used. An enormous amount of information can be derived from these key concepts for the further analysis such as their dispersion, relationship among the concepts etc. The relationship among the key concepts can be used to draw a concept graphs. So, this chapter presents a detailed methodologies and technologies which evaluate the effectiveness of the extracted information from text files.

FunctionalLinkNeuralNetwork(FLNN)hasemergedasanimportanttoolforsolvingnon-linear classificationproblemandhasbeensuccessfullyappliedinmanyengineeringandscientificproblems. TheFLNNstructureismuchmoremodestthanordinaryfeedforwardnetworkliketheMultilayer Perceptron (MLP) due to its flat network architecture which employs less tuneable weights for training.However,thestandardBackpropagation(BP)learningusesforFLNNtrainingpronetoget trapinlocalminimawhichaffecttheFLNNclassificationperformance.TorecovertheBP-learning drawback,thispaperproposesanArtificialBeeColony(ABC)optimizationwithmodificationon beeforagingbehaviour(mABC)asanalternativelearningschemeforFLNN.Thisismotivatedby goodexplorationandexploitationcapabilitiesofsearchingoptimalweightparametersexhibitbyABC algorithm.TheresultoftheclassificationaccuracymadebyFLNNwithmABC(FLNN-mABC) iscomparedwiththeoriginalFLNNarchitecturewithstandardBackpropagation(BP)(FLNN-BP) andstandardABCalgorithm(FLNN-ABC).TheFLNN-mABCalgorithmprovidesbetterlearning schemefortheFLNNnetworkwithaverageoverallimprovementof4.29%ascomparedtoFLNN-BPandFLNN-ABC.

In the Big Data community, MapReduce has been seen as one of the key enabling approaches for meeting continuously increasing demands on computing resources imposed by massive data sets. The reason for this is the high scalability of the MapReduce paradigm which allows for massively parallel and distributed execution over a large number of computing nodes. This paper identifies MapReduce issues and challenges in handling Big Data with the objective of providing an overview of the field, facilitating better planning and management of Big Data projects, and identifying opportunities for future research in this field. The identified challenges are grouped into four main categories corresponding to Big Data tasks types: data storage (relational databases and NoSQL stores), Big Data analytics (machine learning and interactive analytics), online processing, and security and privacy. Moreover, current efforts aimed at improving and extending MapReduce to address identified challenges are presented. Consequently, by identifying issues and challenges MapReduce faces when handling Big Data, this study encourages future Big Data research.

Providing a customized support for the OLAP brings tremendous challenges to the OLAP technology. Standing at the crossroads of the preferences and the data warehouse, two emerging trends are pointed out; namely: (i) the personalization and (ii) the recommendation. Although the panoply of the proposed approaches, the user-centric data warehouse community issues have not been addressed yet. In this paper we draw an overview of several user centric data warehouse proposals. We also discuss the two promising concepts in this issue, namely, the personalization and the recommendation of the data warehouses. We compare the current approaches among each others with respect to some criteria.

Bitmap indices have become popular access methods for data warehouse applications and decision support systems with large amounts of read-mostly data. This paper could arrive a number of results such as ; Bitmap Index highly improves the performance of Query Answering in Data Warehouses, It highly increases the efficiency of Complex Query processing through using bitwise operations (AND, OR). A prototype of Data Warehouse “STUDENTS DW” has been built according to the conditions of W. Inomn of Data Warehouses. This prototype is built for student's information.

The key objective of the chapter would be to study the classification accuracy, using feature selection with machine learning algorithms. The dimensionality of the data is reduced by implementing Feature selection and accuracy of the learning algorithm improved. We test how an integrated feature selection could affect the accuracy of three classifiers by performing feature selection methods. The filter effects show that Information Gain (IG), Gain Ratio (GR) and Relief-f, and wrapper effect show that Bagging and Naive Bayes (NB), enabled the classifiers to give the highest escalation in classification accuracy about the average while reducing the volume of unnecessary attributes. The achieved conclusions can advise the machine learning users, which classifier and feature selection methods to use to optimize the classification accuracy, and this can be important, especially at risk-sensitive applying Machine Learning whereas in the one of the aim to reduce costs of collecting, processing and storage of unnecessary data.

The goal of this article is to depict Object Oriented Conceptual Model Data Cube using it as an example. In short, the paper has described: 1) Structural level in object oriented class definition of data cube; 2) The basic methods for computing data cube from relational database by select operator; 3) To set tasks for future work to develop appropriate methods of class to construct data cube in different way from relational operator.

Data warehousing has become an essential tool in modern organizations driven by increasing business complexity and technological advancements. Organizations collect vast amounts of data from multiple sources that require efficient storage and analysis solutions. This research paper examines the role of data warehousing in decision making, its integration with emerging technologies, and its growing impact on various industries. This research is significant as it highlights the transformative role of data warehousing in decision-making across industries. By improving data quality, accessibility, and integration, data warehouses enhance business intelligence and operational efficiency. The study provides valuable insights into leveraging data warehousing technologies, addressing challenges, and fostering innovation in data-driven environments.

Big Data is an important topic for discussion and research. It has gained this importance due to the meaningful value that could be extracted from these data. The application of Big Data in the modern business allows enterprises to take faster and smarter decisions, achieving a real competitive advantage. However, a lot of Big Data projects provide disappointing results that don't address the decision-makers' needs due to many reasons. The main reason for this failure can be summarized in neglecting the study of the decision-making aspect of these projects. In light of this challenge, this study proposes the integration of decision aspect into Big Data as a solution. Therefore, this article presents three main contributions: 1) Clarify the definition of Big Data; 2) Presents BD-Da model, a conceptual model describes the levels that should be considered to develop a Big Data project aiming to solve a problem that calls a decision; 3) Describes a particular, logical, requireme...

Modeling plays an important role for the solution of the complex research problems. When the database became large and complex then it is necessary to create a unified model for getting the desired information in the minimum time and to implement the model in a better way. The present paper deals with the modeling for searching of the desired information from a large database by storing the data inside the three dimensional data cubes. A sample case study is considered as a real data related to the ground water and municipal water supply, which contains the data from the various localities of a city. For the demonstration purpose, a sample size is taken as nine but when it becomes very large for number of localities of different cities then it is necessary to store the data inside data cubes. A well known object-oriented Unified Modeling Language (UML) is used to create Unified class and state models. For verification purpose, sample queries are also performed and corresponding results are depicted.

The process of selecting suppliers is a formidable challenge in managing industrial construction projects. Construction companies generate a huge amount of data that is spread across multiple databases, but this data is not used to support decisions for future projects. To address this issue, a data warehouse was developed specifically for construction companies which are specializing in industrial projects. The proposed data warehouse utilizes operational databases from past projects, including planning and execution details, to organize and analyze data for supplier selection decisions in ongoing and future projects. The warehouse's dimensional model was built to meet the requirements of construction enterprises and the available data. The methodology employs Online Analytical Processing (OLAP), which enables direct queries and generates relevant reports to support construction management decisions and evaluate suppliers for different types of work. By adopting this approach, construction companies can make more informed decisions about which suppliers to choose for their projects, ultimately improving their economic performance. By leveraging the data available to them, these companies can enhance the quality of their decisions and improve outcomes for their industrial construction projects.

Transformation design is a key step in model-driven engineering, and it is a very challenging task, particularly in context of the model-driven data warehouse. Currently, this process is ensured by human experts. The authors propose a new methodology using machine learning techniques to automatically derive these transformation rules. The main goal is to automatically derive the transformation rules to be applied in the model-driven data warehouse process. The proposed solution allows for a simple design of the decision support systems and the reduction of time and costs of development. The authors use the inductive logic programming framework to learn these transformation rules from examples of previous projects. Then, they find that in model-driven data warehouse application, dependencies exist between transformations. Therefore, the authors investigate a new machine learning methodology, learning dependent-concepts, that is suitable to solve this kind of problem. The experimental evaluation shows that the dependent-concept learning approach gives significantly better results.

This paper studies a new machine learning application with possibly a challenging benchmark for relational learning systems. We are interesting in the automation of Model-Driven Data Warehouse [6, 13, 2] using machine learning techniques. Intend is to automatically derive the transformations rules to be applied in the model-driven process. This aims to reduce the contribution of transformations designer and therefore time and cost of development. We propose to express the model transformation problem as an Inductive Logic Programming one: existing project traces (or projects experiences) are used to define the background knowledge and examples. The aleph ILP engine is used to learn best transformation rules. In our application, we need to deal with several dependent-concepts. Taking into account work in Predicate Invention, Layered Learning, Cascade Learning and Context Learning, we propose a new methodology that automatically updates the background knowledge of the concepts to be learned. Experimental results support the conclusion that this approach is suitable to solve this kind of problem.

Nowadays, there are an emergence of spatial or geographic data stored in several and heterogeneous databases, mostly in Geographic Information Systems (GIS). The diversity of GIS and the increasing accumulation of non-spatial (simple attributes) and spatial (geometric shapes) data make it difficult to apply conventional OLAP and data mining tools. Thus, the need to build a spatial data warehouse over heterogeneous GIS is becoming necessary in many fields. Our architecture is a central type architecture based on GML (for spatial data representation) and more generally on XML (for all data). In this paper we will focus on the study of data integration into a data warehouse, and data representation. We will show how the specificities of our architecture contribute to manage spatial and non-spatial data.

DBPubs is a system for effectively analyzing and exploring the content of database publications by combining keyword search with OLAP-style aggregations, navigation, and reporting. DBPubs starts with keyword search over the content of publications. The publications' metadata such as title, authors, venues, year, and so on, provide traditional OLAP static dimensions, which are combined with dynamic dimensions discovered from the content of the publications in the search result, such as frequent phrases, relevant phrases, and topics. We compute publication ranks based on the link structure between documents, i.e., citations, and aggregate them to find seminal papers, discover trends, and rank authors. We deploy an OLAP tool for multidimensional content exploration through traditional OLAP rollup-drilldown operations on the static and dynamic dimensions, solutions for multi-cube analysis, dynamic navigation of the content, and highlighting of interesting dices of the multidimension...

Content Management Systems (CMS) store enterprise data such as insurance claims, insurance policies, legal documents, patent applications, or archival data like in the case of digital libraries. Search over content allows for information retrieval, but does not provide users with great insight into the data. A more analytical view is needed through analysis, aggregations, groupings, trends, pivot tables or charts, and so on. Multidimensional Content eXploration (MCX) is about effectively analyzing and exploring large amounts of content by combining keyword search with OLAP-style aggregation, navigation, and reporting. We focus on unstructured data or generally speaking documents or content with limited metadata, as it is typically encountered in CMS. We formally present how CMS content and metadata should be organized in a well-defined multidimensional structure, so that sophisticated queries can be expressed and evaluated. The CMS metadata provide traditional OLAP static dimensions...

Visualization of multivariate data is a big challenge to problems of visual analytics. A system of data visualization is composed of visual mapping stage and visual display stage. The stage of visual mapping converts data to graph and the stage of visual display shows the graph on screen in accordance with human's retinal perception which is specified by visual features and Gestalt's laws. Based on data characteristics, multidimensional cubes representing multivariate data are classified as non-spatial multidimensional cube for non-spatial data, spatial multidimensional cube for spatio-temporal data, spatiotemporal multidimensional cube for movement data, and 3D-spatio-temporal multidimensional cube for flight data. For a visualization system responding human's retinal perception, multidimensional cubes have to enable analysts to answer elementary questions concerning individual values, variation questions concerning part of data or overall data, and relation questions resulting in the correlation among attributes.

In this paper the so-called Event Cube is introduced, a multidimensional data structure that can hold information about all business dimensions. Like the data cubes of online analytic processing (OLAP) systems, the Event Cube can be used to improve the business analysis quality by providing immediate results under different levels of abstraction. An exploratory analysis of the application of process mining on multidimensional process data is the focus of this paper. The feasibility and potential of this approach is demonstrated through some practical examples.

Universities collect and generate a considerable amount of data on students throughout their academic career. Currently in South Kivu, most universities have an information system in the form of a database made up of several disparate files. This makes it difficult to use this data efficiently and profitably. The aim of this study is to develop this transactional database-based information system into a data warehouse-oriented system. This tool will be able to collect, organize and archive data on the student's career path, year after year, and transform it for analysis purposes. In the age of Big Data, a number of artificial intelligence techniques have been developed, making it possible to extract useful information from large databases. This extracted information is of paramount importance in decision-making. By way of example, the information extracted by these techniques can be used to predict which stream a student should choose when applying to university. In order to develop our contribution, we analyzed the IT information systems used in the various universities and applied the bottom-up method to design our data warehouse model. We used the relational model to design the data warehouse.

Due to the increasingly di culty of discovering patterns in real-world databases using only conventional OLAP tools, an automated p r ocess such as data mining is currently essential. As data mining over large data sets can take a prohibitive amount of time related t o the computational complexity of the algorithms, parallel processing has often been used as a solution. However, when data does not t in memory, some solutions do not apply and a database system may be r equired r ather than at les. Most implementations use the database system loosely-coupled with the data mining algorithms. In this work we address the data consuming activities through parallel processing and data fragmentation on the database server, providing a tight integration with data mining techniques. Experimental results showing the potential bene ts of this integration were obtained, despite the di culties to process a complex application.

Recent years have shown the need of an automated p r ocess to discover interesting and hidden patterns in real-world databases, handling large volumes of data. This sort of process implies a lot of computational power, memory and disk I O, which can only be p r ovided b y p arallel computers. Our work contributes with a solution that integrates a machine learning algorithm, parallelism and a tightly-coupled use of a DBMS system, addressing performance problems with parallel processing and data fragmentation.

In data warehousing applications, numerous OLAP queries involve the processing of holistic aggregators such as computing the "top n," median, quantiles, etc. In this paper, we present a novel approach called dynamic bucketing to efficiently evaluate these aggregators. We partition data into equiwidth buckets and further partition dense buckets into sub-buckets as needed by allocating and reclaiming memory space. The bucketing process dynamically adapts to the input order and distribution of input datasets. The histograms of the buckets and subbuckets are stored in our new data structure called structure trees. A recent selection algorithm based on regular sampling is generalized and its analysis extended. We have also compared our new algorithms with this generalized algorithm and several other recent algorithms. Experimental results show that our new algorithms significantly outperform prior ones not only in the runtime but also in accuracy.

Agradeço aos amigos do doutorado, Ruben, Ellison e Cristiano, pela amizade. Aos amigos Daniel Diaz, Mateus, Marcelo e Werther pela amizade e disposição em ajudar-me. A Alberto Sulaiman e Rosalvo Streit pelo apoio e incentivo dado ao meu trabalho no Banco Central. À Professora Valéria, pela dedicação e seriedade com o desenvolvimento desta tese. A Ana Paula, companheira de todas as horas, pelo incentivo nos momentos mais difíceis e pela paciência e compreensão. A Giovanna e Guilherme por suportarem minha ausência e alegrarem meus dias. A meus pais, pelo exemplo e dedicação dados a mim e meus irmãos. A meus irmãos Flávio, Tuca e Jú, pelo incentivo, apoio e amizade. O uso de ferramentas de processamento analítico de dados (OLAP) para realização de análises estratégicas de uma organização possibilita que usuários responsáveis pela tomada de decisões possam identificar tendências e padrões, de forma a conduzir melhor o negócio da empresa em que atuam. Entretanto, o desenvolvimento de sistemas de processamento analítico em dados XML no meio acadêmico e comercial não possui todas as funcionalidades das ferramentas OLAP para dados tradicionais e também não contempla documentos XML interligados. Portanto, a necessidade de desenvolver sistemas OLAP para auxiliar nas análises estratégicas dos dados de uma organização, representados no formato XML e interligados por um conjunto de referências, constitui a principal motivação para o desenvolvimento deste trabalho. Atualmente, pesquisas vêm sendo desenvolvidas no contexto acadêmico com o objetivo de realizar processamento analítico em dados representados em XML. No entanto, em razão destas tecnologias terem sido originalmente concebidas para propósitos distintos, esta não é uma tarefa trivial. Para ajudar no desenvolvimento desses sistemas OLAP, neste trabalho é proposto um sistema computacional constituído de uma linguagem de consulta analítica, LMDQL, a qual permite a realização de consultas a dados multidimensionais baseados em documentos XML interligados entre si. Como as linguagens padrões do W3C de consulta a documentos XML (e.g. XPath e XQuery) não têm suporte a links, foi especificada, e incorporada ao processador da LMDQL, XLPath, uma linguagem de navegação para documentos XML com links. Além disso, para modelar cubos de dados em documentos XML interligados e para tratar as heterogeneidades sintática, semântica e estrutural encontradas em dados representados por XML, é apresentado um metamodelo multidimensional de dados, chamado XLDM. A partir destas especificações, foi desenvolvido um protótipo do sistema que possibilita a análise de documentos XML interligados. Este protótipo inclui a LMDQL, XLDM, XLPath e um driver, sql2xquery, que realiza o mapeamento de consultas SQL em XQuery, viabilizando a incorporação deste protótipo em um servidor OLAP para dados no formato relacional. Para avaliar as propostas apresentadas nesta tese, são realizados estudos de casos em três domínios distintos, nas áreas médica, financeira e de vendas, de forma a mostrar a expressividade e utilização do sistema proposto. São discutidos resultados de testes de desempenho para avaliar o tempo de execução do processamento analítico em documentos XML interligados. Nestes testes, foi avaliada a eficiência do driver em três SGDB para dados XML e foi investigado o impacto no tempo de execução do processamento analítico em documentos XML que usam links para expressar informação adicional aos dados XML. O custo de processamento de consultas em LMDQL foi comparado com o custo destas mesmas consultas em MDX (um padrão de mercado), mostrando que o benefício alcançado com o uso de LMDQL não impacta no tempo de execução das consultas.

Universities collect and generate a considerable amount of data on students throughout their academic career. Currently in South Kivu, most universities have an information system in the form of a database made up of several disparate files. This makes it difficult to use this data efficiently and profitably. The aim of this study is to develop this transactional database-based information system into a data warehouse-oriented system. This tool will be able to collect, organize and archive data on the student's career path, year after year, and transform it for analysis purposes. In the age of Big Data, a number of artificial intelligence techniques have been developed, making it possible to extract useful information from large databases. This extracted information is of paramount importance in decision-making. By way of example, the information extracted by these techniques can be used to predict which stream a student should choose when applying to university. In order to develop our contribution, we analyzed the IT information systems used in the various universities and applied the bottom-up method to design our data warehouse model. We used the relational model to design the data warehouse.

In many application contexts, like statistical databases, scientific databases, query optimizers, OLAP, and so on, data are often summarized into synopses of aggregate values. Summarization has the great advantage of saving space, but querying aggregate data rather than the original ones introduces estimation errors which cannot be in general avoided, as summarization is a lossy compression. A central problem in designing summarization techniques is to retain a certain degree of accuracy in reconstructing query answers. In this paper we restrict our attention to two-dimensional data, which are relevant for a number of applications, and propose a hierarchical summarization technique which is combined with the use of indices, i.e. compact structures providing an approximate description of portions of the original data. Experimental results show that the technique gives approximation errors much smaller than other "general purpose" techniques, such as wavelets and various types of multi-dimensional histogram.

The continuous growth of OLAP users and data impose additional stress on data management and hardware infrastructure. The distribution of multidimensional data through a number of servers allows the increasing of storage and processing power without an exponential increase of financial costs. But this solution adds another dimension to the problem: space. Even in centralized OLAP, cube selection efficiency is complex, but now, we must also know where to materialize subcubes. This paper proposes algorithms that solve the distributed OLAP selection problem under space constraints, considering a query profile, using discrete particle swarm optimization in its normal, cooperative, multi-phase and hybrid genetic versions.

In OLAP, the materialization of multidimensional structures is a sine qua non condition of performance. Problems that come along with this need have triggered a huge variety of proposals: the picking of the optimal set of aggregation combinations, to materialize into centralized OLAP repositories, emerges among them. This selection is based on general purpose combinatorial optimization algorithms, such as greedy, evolutionary, swarm and randomizing approaches. Only recently, the distributed approach has come to stage, introducing another source of complexity: space. Now, it's not enough to select the appropriate data structures, but also to know where to locate them. To solve this extended problem, optimizing heuristics are faced with extra complexity, hardening its search for solutions. This paper presents a polymorphic algorithm, coined as metamorphosis algorithm that combines genetic, particle swarm and hill climbing metaheuristics. It is used to solve the extended cube selection and allocation problem generated in M-OLAP architectures.

Answering performance to business queries, mainly of aggregated nature, known as On-Line Analytical Processing queries, depends heavily on the proper selection of multidimensional structures, known as materialized sub- cubes or views. As user's queries profiles change, these structures have to be recalibrated, once elected the new appropriated selection through a cube view selection algorithm. In the very core of all these algorithms is the estimation of the cost of answering queries and maintenance, given M materialized OLAP se- lected structures that may be of a distributed nature. This paper introduces a new extended cost model that supports both centralized or multi-node OLAP architecture, query and maintenance cost of non-linear nature and incremental or integral (from scratch) maintenance.

The problem of understanding the behavior of business processes and of services is rapidly becoming a priority in medium and large companies. To this end, recently, analysis tools as well as variations of data mining techniques have been applied to process and service execution logs to perform OLAP-style analysis and to discover behavioral (process and protocol) models out of execution data. All these approaches are based on one key assumption: events describing executions and stored in process and service logs include identifiers that allow associating each event to the process or service execution they belong to (e.g., can correlate all events related to the processing of a certain purchase order or to the hiring of a given employee). In reality, however, such information rarely exists. In this paper, we present a framework for discovering correlations among messages in service logs. We characterize the problem of message correlation and propose novel algorithms and techniques based on heuristics on the characteristics of conversations and of message attributes that can act as identifier for such conversations. As we will show, there is no right or wrong way to correlate messages, and such correlation is necessarily subjective. To account for this subjectiveness, we propose an approach where algorithms suggest candidate correlators, provide measures that help users understand the implications of choosing a given correlators, and organize candidate correlators in such a way to facilitate visual exploration. The approach has been implemented and experimental results show its viability and scalability on large synthetic and real-world datasets. We believe that message correlation is a very important and challenging area of research that will witness many contributions in the near future due to the pressing industry needs for process and service execution analysis.

Materialized views in data warehouses are typically complicated, making the maintenance of such views difficult. However, they are also very important for improving the speed of access to the information in the data warehouse. So, the selection of materialized views is crucial to the operation of the data warehouse both with respect to maintenance and speed of access. Most research to date has treated the selection of materialized views as an optimization problem with respect to the cost of view maintenance and/or with respect to the cost of queries. In this paper, we consider practical aspects of data warehousing. We identify problems with the star and snowflake schema and suggest solutions. We also identify practical problems that may arise during view selection and suggest heuristics based on data dependencies and access patterns that can be used to measure if one set of views is better than another set of views, or used to improve a set of views.

In the context of Volunteered Geographic Information (VGI), volunteers are not involved in the decisional processes. Moreover, VGI systems do not offer advanced historical analysis tools. Therefore, in this work, we propose to use Data Warehouse (DW) and OLAP systems to analyze VGI data, and we define a new DW design methodology that allows involving volunteers in the definition of analysis needs over VGI data. We validate it using a real biodiversity case study.

Universities collect and generate a considerable amount of data on students throughout their academic career. Currently in South Kivu, most universities have an information system in the form of a database made up of several disparate files. This makes it difficult to use this data efficiently and profitably. The aim of this study is to develop this transactional database-based information system into a data warehouse-oriented system. This tool will be able to collect, organize and archive data on the student's career path, year after year, and transform it for analysis purposes. In the age of Big Data, a number of artificial intelligence techniques have been developed, making it possible to extract useful information from large databases. This extracted information is of paramount importance in decision-making. By way of example, the information extracted by these techniques can be used to predict which stream a student should choose when applying to university. In order to develop our contribution, we analyzed the IT information systems used in the various universities and applied the bottom-up method to design our data warehouse model. We used the relational model to design the data warehouse.

One of the critical deficiencies of SQL is lack of support for ndimensional array-based computations which are frequent in OLAP environments. Relational OLAP (ROLAP) applications have to emulate them using joins, recently introduced SQL Window Functions and complex and inefficient CASE expressions. The designated place in SQL for specifying calculations is the SELECT clause, which is extremely limiting and forces the user to generate queries using nested views, subqueries and complex joins. Furthermore, SQL-query optimizer is pre-occupied with determining efficient join orders and choosing optimal access methods and largely disregards optimization of complex numerical formulas. Execution methods concentrated on efficient computation of a cube [11], [16] rather than on random access structures for inter-row calculations. This has created a gap that has been filled by spreadsheets and specialized MOLAP engines, which are good at formulas for mathematical modeling but lack the formalism of the relational model, are difficult to manage, and exhibit scalability problems. This paper presents SQL extensions involving array based calculations for complex modeling. In addition, we present optimizations, access structures and execution models for processing them efficiently.

One of the critical deficiencies of SQL is lack of support for ndimensional array-based computations which are frequent in OLAP environments. Relational OLAP (ROLAP) applications have to emulate them using joins, recently introduced SQL Window Functions and complex and inefficient CASE expressions. The designated place in SQL for specifying calculations is the SELECT clause, which is extremely limiting and forces the user to generate queries using nested views, subqueries and complex joins. Furthermore, SQL-query optimizer is pre-occupied with determining efficient join orders and choosing optimal access methods and largely disregards optimization of complex numerical formulas. Execution methods concentrated on efficient computation of a cube [11], [16] rather than on random access structures for inter-row calculations. This has created a gap that has been filled by spreadsheets and specialized MOLAP engines, which are good at formulas for mathematical modeling but lack the formalism of the relational model, are difficult to manage, and exhibit scalability problems. This paper presents SQL extensions involving array based calculations for complex modeling. In addition, we present optimizations, access structures and execution models for processing them efficiently.