Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Introduction General Context
Data Reconciliation
Data Consolidation
Data Disambiguation
Managing Imprecise and Uncertain Data
Probabilistic Data Models
Possibilistic Data Models
Handling Missing and Incomplete Data
Completeness of Relational Data Sources
Completion and Statistical Editing
Improving Data Freshness
Conclusions
Methodological Approaches and Benchmarks
Modeling Quality Metadata
CWM Extension for Qod Metadata Management
Computing Quality Metadata
Data Points Characteristics
Designing Analytic Workflows for Qod Evaluation
Indexing Quality Metadata
Manipulation of Data and Quality Metadata
Query# Query Results
Qod Metadata and Analytic Workflow Modeling
Design of Analytic Workflows
Multidimensional Indexing of Qod Metadata
Data Quality Awareness for KDD Process
KDD Downstream: Quality Metadata Exploitation
Summary
Introduction
Quality-Aware Integration of Biomedical Data
Related Work
Monitoring the Quality of Stream Data
Conclusion
Conclusions and Research Perspectives Summary
Statistical Metadata Computation
References
All Topics
Computer Science
Information Systems

Quality awareness for managing and mining data

Profile image of Laure Berti-EquilleLaure Berti-Equille

Abstract

Autonomy Heterogeneity no yes totally semi DIS DW & MIS VMS CIS RS P2P no

Figures (68)
Figure 2: Applications and Data Types Coverage Beyond the defense of the “Habilitation a Diriger des Recherches (HDR)”, a wish would be that this dissertation may be useful for one- or two-semester courses on data quality in advanced undergraduate or graduate courses on database manage- ment and data warehouse design that are usually required in Information System, Business IT, or Computer Science curriculum.
Figure 2: Applications and Data Types Coverage Beyond the defense of the “Habilitation a Diriger des Recherches (HDR)”, a wish would be that this dissertation may be useful for one- or two-semester courses on data quality in advanced undergraduate or graduate courses on database manage- ment and data warehouse design that are usually required in Information System, Business IT, or Computer Science curriculum.
Contents
Contents
Figure 1.1: Taxonomy of Existing Techniques for Entity Resolution
Figure 1.1: Taxonomy of Existing Techniques for Entity Resolution
Table 1.3: Decision Models for Handling Duplicates (Batini & Scannapieco, 2006)
Table 1.3: Decision Models for Handling Duplicates (Batini & Scannapieco, 2006)
such that x, < aj; + %;, (triangular inequality) for all i, 7,4 vertices of the graph, «;; € {0,1}: if 7 and j are in the same cluster x;,; is 1; otherwise 2;; is 0. A method for online clustering in (Sarawagi & Kirpal, 2004) avoids the problem of data skew and redundant computation while creating clusters based on similar- ity along multiple strings rather than single strings, and it improves the efficiency when index searching with increasing threshold to return the most similar cluster. Fa eh le ee, ee a i: a eo cs ee i a ee, Ee et on i Se es: ae ee ee. ee
such that x, < aj; + %;, (triangular inequality) for all i, 7,4 vertices of the graph, «;; € {0,1}: if 7 and j are in the same cluster x;,; is 1; otherwise 2;; is 0. A method for online clustering in (Sarawagi & Kirpal, 2004) avoids the problem of data skew and redundant computation while creating clusters based on similar- ity along multiple strings rather than single strings, and it improves the efficiency when index searching with increasing threshold to return the most similar cluster. Fa eh le ee, ee a i: a eo cs ee i a ee, Ee et on i Se es: ae ee ee. ee
Table 1.4: Main Data Transformation Operators for ETL Table 1.4 presents the main ETL operators (e.g., format, add, merge, etc.) sup- ported by Potter’s Wheel and AJAX systems with, in the first column, the name of the operator and, in the second column, its definition.
Table 1.4: Main Data Transformation Operators for ETL Table 1.4 presents the main ETL operators (e.g., format, add, merge, etc.) sup- ported by Potter’s Wheel and AJAX systems with, in the first column, the name of the operator and, in the second column, its definition.
Based on the online study of Galhardas*, Table 1.5 presents the research proto- types and open source systems developed for data cleaning, data preparation and analysis.
Based on the online study of Galhardas*, Table 1.5 presents the research proto- types and open source systems developed for data cleaning, data preparation and analysis.
Contents
Contents
Table 2.2: Join over FK attributes considered as correct Join over FK attributes considered as correct. Ignoring these observations, we as- sume that the data are correct, fresh, consistent, complete and without duplicates, the join of tables PRODUCT and CUSTOMER over the CUST_ID attribute would result in a table consisting of 3 rows, as shown in Table 2.2. For the sake of simplic- ity, only PROD_ID, CUST_IDs, P_DESTINATION and CUST_CITY are projected in the join result of tables PRODUCT and CUSTOMER, respectively renamed as P and C.
Table 2.2: Join over FK attributes considered as correct Join over FK attributes considered as correct. Ignoring these observations, we as- sume that the data are correct, fresh, consistent, complete and without duplicates, the join of tables PRODUCT and CUSTOMER over the CUST_ID attribute would result in a table consisting of 3 rows, as shown in Table 2.2. For the sake of simplic- ity, only PROD_ID, CUST_IDs, P_DESTINATION and CUST_CITY are projected in the join result of tables PRODUCT and CUSTOMER, respectively renamed as P and C.
Join over incorrect FK attributes. If a value of the joining field is incorrect, then, assuming that referential integrity constraints have been enforced, the join would result in a row that, while incorrect, at least should be there. To see this, suppose that in the P3 row of PRODUCT table, the correct value C'2 is replaced by an incor- rect value, say C3. Then, the join operation would generate an incorrect row for a correct one, as shown in Table 2.3. Join over non FK attributes considered as correct. If the joining field is not a foreign key, then the result could contain for each incorrect joining value multiple rows that would not exist. For instance, in Table 2.1, consider joining the two tables over the P_DESTINATION field of PRODUCT table and the CUST_CITY field of CUSTOMER table. The result would yield five rows, as illustrated in Table 2.4.
Join over incorrect FK attributes. If a value of the joining field is incorrect, then, assuming that referential integrity constraints have been enforced, the join would result in a row that, while incorrect, at least should be there. To see this, suppose that in the P3 row of PRODUCT table, the correct value C'2 is replaced by an incor- rect value, say C3. Then, the join operation would generate an incorrect row for a correct one, as shown in Table 2.3. Join over non FK attributes considered as correct. If the joining field is not a foreign key, then the result could contain for each incorrect joining value multiple rows that would not exist. For instance, in Table 2.1, consider joining the two tables over the P_DESTINATION field of PRODUCT table and the CUST_CITY field of CUSTOMER table. The result would yield five rows, as illustrated in Table 2.4.
Join over incorrect non FK attributes. Now assume that the value ‘Reading, NJ, US’ in the P3 row of PRODUCT table is incorrectly replaced by the value ‘Reading UK’. The resulting join would now have six rows as shown in Table 2.5, two o which would be incorrect. Another type of error in that field could lead to multiple missing rows. To see this, consider the case where, in P3 row of the PRODUCT table, the value ‘Reading, NJ, US’ is incorrectly replaced by ‘London’. In this case, the resulting join would four rows instead of five.
Join over incorrect non FK attributes. Now assume that the value ‘Reading, NJ, US’ in the P3 row of PRODUCT table is incorrectly replaced by the value ‘Reading UK’. The resulting join would now have six rows as shown in Table 2.5, two o which would be incorrect. Another type of error in that field could lead to multiple missing rows. To see this, consider the case where, in P3 row of the PRODUCT table, the value ‘Reading, NJ, US’ is incorrectly replaced by ‘London’. In this case, the resulting join would four rows instead of five.
row of PRODUCT table. The correct answer is illustrated in Table 2.6 instead o! the one given in Table 2.2 for P1. Join over non FK attributes of duplicates. The result of joining PRODUCT and CUSTOMER over the non FK attributes P_DESTINATION and the CUST_CITY once the duplicates have been removed in CUSTOMER table would be composed of three rows (instead of five rows) as illustrated in Table 2.7.
row of PRODUCT table. The correct answer is illustrated in Table 2.6 instead o! the one given in Table 2.2 for P1. Join over non FK attributes of duplicates. The result of joining PRODUCT and CUSTOMER over the non FK attributes P_DESTINATION and the CUST_CITY once the duplicates have been removed in CUSTOMER table would be composed of three rows (instead of five rows) as illustrated in Table 2.7.
Figure 2.2: Relational Metamodel of CRM_DB
Figure 2.2: Relational Metamodel of CRM_DB
Modeling the relational resource instances, Figure 2.3 shows how a Rowset in- herits from Extent, from the Foundation package. It represents all the data com- prised in a ColumnSet. A RowSet can only be owned by a ColumnSet or any derived class. A RowSet contains Rows. Row inherits from Object. Its structure is defined by the corresponding ColumnSet and its Columns from the relational metamodel given in Annexes, Figure A.1 and A.2. (pages 152 et 153). Each Row is divided into ColumnValues, which match the value of a relational table, at the intersection of a row and a column. ColumnValue inherits from DataValue from ObjectModel.
Modeling the relational resource instances, Figure 2.3 shows how a Rowset in- herits from Extent, from the Foundation package. It represents all the data com- prised in a ColumnSet. A RowSet can only be owned by a ColumnSet or any derived class. A RowSet contains Rows. Row inherits from Object. Its structure is defined by the corresponding ColumnSet and its Columns from the relational metamodel given in Annexes, Figure A.1 and A.2. (pages 152 et 153). Each Row is divided into ColumnValues, which match the value of a relational table, at the intersection of a row and a column. ColumnValue inherits from DataValue from ObjectModel.
Figure 2.4: QoD Extension to CWM Data Instance Metamodel Quality dimensions are used as the vocabulary to define abstractly various as- pects of quality, as the user may perceive it. Of course, each user might have a different vocabulary, different preferences and goals underlying quality specifica- tions in the various dimensions of quality. In order to aggregate the QoD measures related to one quality dimension, we define the QoDDimension class composed of the associated QoDMeasures. QoD dimension has a name, a scoring method, and a acceptability value. The scoring method is used to compute the acceptability value with possibly weighting the degree of constraint satisfaction of the QoD measures associated to the QoD dimension and compared to expected values. Acceptability of a QoD dimension evaluated from a data object instance is the probability that each value of its associated QoD measures satisfies the specified constraint.
Figure 2.4: QoD Extension to CWM Data Instance Metamodel Quality dimensions are used as the vocabulary to define abstractly various as- pects of quality, as the user may perceive it. Of course, each user might have a different vocabulary, different preferences and goals underlying quality specifica- tions in the various dimensions of quality. In order to aggregate the QoD measures related to one quality dimension, we define the QoDDimension class composed of the associated QoDMeasures. QoD dimension has a name, a scoring method, and a acceptability value. The scoring method is used to compute the acceptability value with possibly weighting the degree of constraint satisfaction of the QoD measures associated to the QoD dimension and compared to expected values. Acceptability of a QoD dimension evaluated from a data object instance is the probability that each value of its associated QoD measures satisfies the specified constraint.
Figure 2.5: Example of QoD Metadata Associated to CRM_DB PRODUCT table Each QoD measure is associated to only one QoD dimension and one database object in- stance. For example, for PRICE column, nullValues% is a QoD measure associated to Completeness QoD dimension. This QoD measure is calculated at a given time (May 15th, 2006) by procedure plsql_id6 and returns the value 75% whereas the expected value for this measure equals 100%.
Figure 2.5: Example of QoD Metadata Associated to CRM_DB PRODUCT table Each QoD measure is associated to only one QoD dimension and one database object in- stance. For example, for PRICE column, nullValues% is a QoD measure associated to Completeness QoD dimension. This QoD measure is calculated at a given time (May 15th, 2006) by procedure plsql_id6 and returns the value 75% whereas the expected value for this measure equals 100%.
Generic examples of Level II Functions we use for characterizing aspects of consistency (CT) and accuracy (AC) dimensions are given in Table 2.9. Table 2.9: Examples of Level IT Functions
Generic examples of Level II Functions we use for characterizing aspects of consistency (CT) and accuracy (AC) dimensions are given in Table 2.9. Table 2.9: Examples of Level IT Functions
The larger the (”)-statistic, the farther away, in the generalized distance sense, the point (a1;, %2;,+++ , @mi) is from the centroid (£1;, £2;,:-- , Zmi) of the reference group. Thus, the tuple may be said to be more deviant from the average member of the group.
The larger the (”)-statistic, the farther away, in the generalized distance sense, the point (a1;, %2;,+++ , @mi) is from the centroid (£1;, £2;,:-- , Zmi) of the reference group. Thus, the tuple may be said to be more deviant from the average member of the group.
Table 2.12 presents the description of the main classification techniques (with the underlined ones we used). Table 2.12: Examples of Level IV Classification and Partitioning Methods
Table 2.12 presents the description of the main classification techniques (with the underlined ones we used). Table 2.12: Examples of Level IV Classification and Partitioning Methods
Figure 2.6: Example of Analytic Workflow for QoD evaluation of CRM_DB
Figure 2.6: Example of Analytic Workflow for QoD evaluation of CRM_DB
Table 2.14: Assigning Probabilities to QoD Dimension for a DB Object Instance
Table 2.14: Assigning Probabilities to QoD Dimension for a DB Object Instance
Table 2.15: Range-Encoded Bitmap Index with Binning for QoD Measures As illustrated in Table 2.15, the values of the QoD measure m,; are given in the second leftmost column. The identifier of the DB object instance which the m;; measure value is associated to is given in the fist column. The range of possible values of mj; is partitioned into five bins [0, 20), [20,40), etc. A “1-bit” indicates that the measure value falls into a specific bin. On the contrary, a “O-bit” indicates that the measure value does not fall into the specific bin.
Table 2.15: Range-Encoded Bitmap Index with Binning for QoD Measures As illustrated in Table 2.15, the values of the QoD measure m,; are given in the second leftmost column. The identifier of the DB object instance which the m;; measure value is associated to is given in the fist column. The range of possible values of mj; is partitioned into five bins [0, 20), [20,40), etc. A “1-bit” indicates that the measure value falls into a specific bin. On the contrary, a “O-bit” indicates that the measure value does not fall into the specific bin.
Figure 2.7: Syntax of Quality Contract Type Creation 2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
Figure 2.7: Syntax of Quality Contract Type Creation 2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
Figure 2.8: Syntax of Call Specification in Quality Contract Type Declaration 2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
Figure 2.8: Syntax of Call Specification in Quality Contract Type Declaration 2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
Table 2.16: Examples of Quality Contract Type Declaration
Table 2.16: Examples of Quality Contract Type Declaration
Checking Freshness on CELLS, ROWS and TABLE of PRODUCT At this stage, when the set of constraints C’ declared in the contracts are checked on the DB object instance 0, a temporary matrix temp is generated from the quality matrix QoD(o) associated to o. Each element of the temporary matrix indicates the degree of satisfaction of the related constraint: 6;; in [0,1].
Checking Freshness on CELLS, ROWS and TABLE of PRODUCT At this stage, when the set of constraints C’ declared in the contracts are checked on the DB object instance 0, a temporary matrix temp is generated from the quality matrix QoD(o) associated to o. Each element of the temporary matrix indicates the degree of satisfaction of the related constraint: 6;; in [0,1].
2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
2.7. EXTENDING THE SYNTAX OF A QUERY LANGUAGE
Table 2.19: Examples of Simple QWITH Queries with EXACT Constraint Checking Mode of the query. Of course, these queries lead to different results depending on the constraints imposed on the QoD dimensions evaluated from database object instances.
Table 2.19: Examples of Simple QWITH Queries with EXACT Constraint Checking Mode of the query. Of course, these queries lead to different results depending on the constraints imposed on the QoD dimensions evaluated from database object instances.
Figure 2.12: Quality-Extended Query Processing
Figure 2.12: Quality-Extended Query Processing
Figure 3.1: General Framework of Data Quality Awareness for the KDD Process metadata on local data sources may be used to drive the data loading process (for example, for loading only the most accurate and up-to-date data in the data ware- housing system). In the second case (post-loading), QoD measures are computed a posteriori from the data that have been already integrated and stored in the system.
Figure 3.1: General Framework of Data Quality Awareness for the KDD Process metadata on local data sources may be used to drive the data loading process (for example, for loading only the most accurate and up-to-date data in the data ware- housing system). In the second case (post-loading), QoD measures are computed a posteriori from the data that have been already integrated and stored in the system.
3.2. DATA QUALITY AWARENESS FOR KDD PROCESS
3.2. DATA QUALITY AWARENESS FOR KDD PROCESS
The average quality of the association rule R denoted ¢(R) can be computed by a weighted sum of the quality vector components of the rule as follows: Definition 3.3.1. The quality of the association rule R is defined by a fusion function denoted ®,; specific for each quality dimension j that merges the components of the quality vectors of the data sets constituting the extension of the right-hand and left-hand sides of the rule. The quality of the rule R is defined as a k-dimensional vector such as:
The average quality of the association rule R denoted ¢(R) can be computed by a weighted sum of the quality vector components of the rule as follows: Definition 3.3.1. The quality of the association rule R is defined by a fusion function denoted ®,; specific for each quality dimension j that merges the components of the quality vectors of the data sets constituting the extension of the right-hand and left-hand sides of the rule. The quality of the rule R is defined as a k-dimensional vector such as:
Based on the main data quality dimensions (e.g., freshness, accuracy, complete ness, and consistency), Table 3.2 presents several examples of fusion function al lowing the combination of quality measures per quality dimension for two dat sets noted x and y in the rule x = y.
Based on the main data quality dimensions (e.g., freshness, accuracy, complete ness, and consistency), Table 3.2 presents several examples of fusion function al lowing the combination of quality measures per quality dimension for two dat sets noted x and y in the rule x = y.
In the case of misclassification these inequalities give rise to three different threshold values 4, p and v (respectively for legitimately, potentially and not interest- ing rules) in the decision space as illustrated in Figure 3.4.
In the case of misclassification these inequalities give rise to three different threshold values 4, p and v (respectively for legitimately, potentially and not interest- ing rules) in the decision space as illustrated in Figure 3.4.
Figure 3.4: Decision Areas for Rule Post-Selection
Figure 3.4: Decision Areas for Rule Post-Selection
Table 3.4: Quality Measures and Estimated Probabilities of Selected Attributes of the KDD-Cup-98 Data Set fac in Table 3.4 (also noted f4c(q(Z)) in our formalism) is the probability den- sity that the data set Z has acceptable quality and correct data when the average quality score of the variable Z is G(Z). fun (also noted fy (qG(Z))) is the probability density that the data set Z has unacceptable quality due to erroneous data when the average quality score of T is q(Z). The top tena priori association rules discovered by (Wang et al., 2005) are given in Table 3.5 with the confidence, the support (in number of records), and the quality scores. Table 3.5 shows the score per quality dimension and the average quality score for each association rule. The scores are computed from the definitions of the quality dimensions given in Table 3.2 and the data quality scores previously given per attribute in Table 3.4.
Table 3.4: Quality Measures and Estimated Probabilities of Selected Attributes of the KDD-Cup-98 Data Set fac in Table 3.4 (also noted f4c(q(Z)) in our formalism) is the probability den- sity that the data set Z has acceptable quality and correct data when the average quality score of the variable Z is G(Z). fun (also noted fy (qG(Z))) is the probability density that the data set Z has unacceptable quality due to erroneous data when the average quality score of T is q(Z). The top tena priori association rules discovered by (Wang et al., 2005) are given in Table 3.5 with the confidence, the support (in number of records), and the quality scores. Table 3.5 shows the score per quality dimension and the average quality score for each association rule. The scores are computed from the definitions of the quality dimensions given in Table 3.2 and the data quality scores previously given per attribute in Table 3.4.
3.4. EXPERIMENTS ON QUALITY-AWARE RULE MINING
3.4. EXPERIMENTS ON QUALITY-AWARE RULE MINING
We observe that only 5 rules (i.e., R1, R5, R7, R9, R10) are potentially interesting among the top ten rules considering the quality of data they are computed from. With data quality-awareness, the other rules (R2, R3, R4, R6, R8) are not interesting despite a good rank in the top ten list. It’s also interesting to notice that the profit per rule predicted by (Wang et al., 2005) may be considerably counterbalanced by the cost of the rule computed from low-quality data (although it depends from initial costs defined in Table 3.3). The second best rule R2 whose predicted profit is $61.73 has a cost of $109.5 and thus is classified as not interesting due to the low quality of its data sets.
We observe that only 5 rules (i.e., R1, R5, R7, R9, R10) are potentially interesting among the top ten rules considering the quality of data they are computed from. With data quality-awareness, the other rules (R2, R3, R4, R6, R8) are not interesting despite a good rank in the top ten list. It’s also interesting to notice that the profit per rule predicted by (Wang et al., 2005) may be considerably counterbalanced by the cost of the rule computed from low-quality data (although it depends from initial costs defined in Table 3.3). The second best rule R2 whose predicted profit is $61.73 has a cost of $109.5 and thus is classified as not interesting due to the low quality of its data sets.
Figure 3.6: Decision Costs for Rule Selection with Different Data Quality Variations without Misclassification for the a Priori Probability 7° = 0.200
Figure 3.6: Decision Costs for Rule Selection with Different Data Quality Variations without Misclassification for the a Priori Probability 7° = 0.200
Figure 3.7: Amplitude of Cost Variations Depending on Data Quality Variations without Misclassification for the a priori Probability 7° = 0.200
Figure 3.7: Amplitude of Cost Variations Depending on Data Quality Variations without Misclassification for the a priori Probability 7° = 0.200
Another interesting result is shown in Figure 3.8 where the decisions for rule selection change simultaneously with the data quality variations. Among the top ten interesting rules discovered by (Wang et al., 2005) with the initial data quality (noted Init Qual), 5 rules (R1, R5, R7, R9 and R10) are potentially worth being selected based on their average data quality and 5 rules are not interesting (R2, R3, R4, R6 and R8). While increasing data quality up to +30%, 3 rules become legitimately interesting (R5, R7 and R9).
Another interesting result is shown in Figure 3.8 where the decisions for rule selection change simultaneously with the data quality variations. Among the top ten interesting rules discovered by (Wang et al., 2005) with the initial data quality (noted Init Qual), 5 rules (R1, R5, R7, R9 and R10) are potentially worth being selected based on their average data quality and 5 rules are not interesting (R2, R3, R4, R6 and R8). While increasing data quality up to +30%, 3 rules become legitimately interesting (R5, R7 and R9).
Figure 3.9: Decision Costs for Rule Selection with a Priori Probability in [0.1,0.5] with Misclassification 3.4. EXPERIMENTS ON QUALITY-AWARE RULE MINING While keeping the a priori probability 7° = 0.200 and using Eq. 3.13, 3.12 and 3.14, we compute the values of the three decision thresholds for rule selection with misclassification and we obtain: = 0.1053, p = 0.1667, and v = 4.6667. Based on the values for these thresholds, we can assign the rules to one of the three decision areas (see Table 3.6).
Figure 3.9: Decision Costs for Rule Selection with a Priori Probability in [0.1,0.5] with Misclassification 3.4. EXPERIMENTS ON QUALITY-AWARE RULE MINING While keeping the a priori probability 7° = 0.200 and using Eq. 3.13, 3.12 and 3.14, we compute the values of the three decision thresholds for rule selection with misclassification and we obtain: = 0.1053, p = 0.1667, and v = 4.6667. Based on the values for these thresholds, we can assign the rules to one of the three decision areas (see Table 3.6).
Figure 3.10: Decision Costs for Rule Selection with Different Data Quality Varia- tions with Misclassification for the a Priori Probability 7° = 0.200 Figure 3.10 shows that the costs are relatively stable with smaller amplitudes and more distinct and staggered cost ranges than in the case without misclassi- fication (see Figures 3.10 and 3.11 compared to Figures 3.6 and 3.7) except at the maxima of data quality variations (.e., 4 impact on decision costs. + 50%) when the misclassification has more
Figure 3.10: Decision Costs for Rule Selection with Different Data Quality Varia- tions with Misclassification for the a Priori Probability 7° = 0.200 Figure 3.10 shows that the costs are relatively stable with smaller amplitudes and more distinct and staggered cost ranges than in the case without misclassi- fication (see Figures 3.10 and 3.11 compared to Figures 3.6 and 3.7) except at the maxima of data quality variations (.e., 4 impact on decision costs. + 50%) when the misclassification has more
In Figure 3.12 only R7 is legitimately interesting among the top ten rules discov- ered by (Wang et al., 2005) with the initial data quality (noted Init Qual). Three rules are not interesting (R8, R3 and R4) and the other 6 rules are potentially inter- esting. Misclassification globally attenuates the “verdict” that classifies each rule correspondingly to one of the decision areas for the legitimately, potentially or not interesting rules. Some rules (e.g., R8) keep the same behavior with or without misclassification when data quality varies.
In Figure 3.12 only R7 is legitimately interesting among the top ten rules discov- ered by (Wang et al., 2005) with the initial data quality (noted Init Qual). Three rules are not interesting (R8, R3 and R4) and the other 6 rules are potentially inter- esting. Misclassification globally attenuates the “verdict” that classifies each rule correspondingly to one of the decision areas for the legitimately, potentially or not interesting rules. Some rules (e.g., R8) keep the same behavior with or without misclassification when data quality varies.
Figure 3.12: Decision Status on Rule Selection for Data Quality Variations with Misclassification for 7° = 0.200
Figure 3.12: Decision Status on Rule Selection for Data Quality Variations with Misclassification for 7° = 0.200
Contents
Contents
data domain, i.e., gene expression measurements through several physiopatholog- ical conditions, ti) genomic data domain, i.e., DNA gene, mRNA, protein sequences and their annotations, and iii) biomedical knowledge, i.e., biological and medical concepts that annotate the genes with concepts of ontologies like GO’ and UMLS”, using BioMeKE"', the application developed at INSERM U522 by Marquet et al. (2003). ; °Gene Ontology, GO: http://www.geneontology.org /
data domain, i.e., gene expression measurements through several physiopatholog- ical conditions, ti) genomic data domain, i.e., DNA gene, mRNA, protein sequences and their annotations, and iii) biomedical knowledge, i.e., biological and medical concepts that annotate the genes with concepts of ontologies like GO’ and UMLS”, using BioMeKE"', the application developed at INSERM U522 by Marquet et al. (2003). ; °Gene Ontology, GO: http://www.geneontology.org /
Figure 4.2: Mediation Architecture with XQuaL the router also broadcasts the QWITH statement with the regular part of the query to all available wrappers, it waits for their response. Finally, the one: of selected wrappers that maximize the satisfaction degrre of quality con straints specified in the contract will be chosen to execute the SQL statement Assume that the constraints are too strict and no wrapper can satisfy them in this case the relaxation module will renegotiate the contract terms witl selected wrappers; it will actually reduce the strictnes of the constraints de fined in the contract of the QWITH query and broadcasts the modified par of the contract to the wrappers able to answer. The wrappers that conforn the most to the modified contract will be selected to answer the query.
Figure 4.2: Mediation Architecture with XQuaL the router also broadcasts the QWITH statement with the regular part of the query to all available wrappers, it waits for their response. Finally, the one: of selected wrappers that maximize the satisfaction degrre of quality con straints specified in the contract will be chosen to execute the SQL statement Assume that the constraints are too strict and no wrapper can satisfy them in this case the relaxation module will renegotiate the contract terms witl selected wrappers; it will actually reduce the strictnes of the constraints de fined in the contract of the QWITH query and broadcasts the modified par of the contract to the wrappers able to answer. The wrappers that conforn the most to the modified contract will be selected to answer the query.
Table A.3: PRICE Outlier Detection in PRODUCT table (PL/SQL)
Table A.3: PRICE Outlier Detection in PRODUCT table (PL/SQL)

Related papers

A Classification of Data Quality Assessment Methods
Ajith Kumar Parlikad

Proceedings of the 16th International …, 2011

Data quality (DQ) assessment can be significantly enhanced with the use of the right DQ assessment methods, which provide automated solutions to assess DQ. The range of DQ assessment methods is very broad: from data profiling and semantic profiling to data matching and data validation. This paper gives an overview of current methods for DQ assessment and classifies the DQ assessment methods into an existing taxonomy of DQ problems. Specific examples of the placement of each DQ method in the taxonomy are provided and illustrate why the method is relevant to the particular taxonomy position. The gaps in the taxonomy, where no current DQ methods exist, show where new methods are required and can guide future research and DQ tool development.

View PDFchevron_right
A Taxonomy of Data Quality Problems
Fatima Rodrigues

Journal of Data and Information Quality

In today's society the exploration of one or more databases to extract information or knowledge to support management is a critical success factor for an organization. However, it is well known that several problems can affect data quality. These problems have a negative effect in the results extracted from data, influencing their correction and validity. In this context, it is quite important to understand theoretically and in practice these data problems. This paper presents a taxonomy of data quality problems, derived from real-world databases. The taxonomy organizes the problems at different levels of abstraction. Methods to detect data quality problems represented as binary trees are also proposed for each abstraction level. The paper also compares this taxonomy with others already proposed in the literature.

View PDFchevron_right
Data quality in information systems
Michael L Brodie

Information & Management, 1980

Until recently, data quality was poor'.y understood and seldom achieved, yet it is essential to tlihe effective use of information systems. This paper discusses/ the nature and importance of data quality. The role of dataquality is placed in the life cycle framework. Many new concepts, tools and i techniques from both programming lang,uages and database management systems are presented and rhiated to data quality. In particular, the coqcept of a databrlse constraint is considered in detail. Some current limitation/s and research directions are proposed.

View PDFchevron_right
A formal definition of data quality problems
Pedro Henrique

2005

The exploration of data to extract information or knowledge to support decision making is a critical success factor for an organization in today's society. However, several problems can affect data quality. These problems have a negative effect in the results extracted from data, affecting their usefulness and correctness. In this context, it is quite important to know and understand the data problems. This paper presents a taxonomy of data quality problems, organizing them by granularity levels of occurrence. A formal definition is presented for each problem included. The taxonomy provides rigorous definitions, which are information-richer than the textual definitions used in previous works. These definitions are useful to the development of a data quality tool that automatically detects the identified problems.

View PDFchevron_right
A Classification of Data Quality Assessment and Improvement Methods
Philip Woodall

Data quality (DQ) assessment and improvement in larger information systems would often not be feasible without using suitable " DQ methods " , which are algorithms that can be automatically executed by computer systems to detect and/or correct problems in datasets. Currently, these methods are already essential, and they will be of even greater importance as the quantity of data in organisational systems grows. This paper provides a review of existing methods for both DQ assessment and improvement and classifies them according to the DQ problem and problem context. Six gaps have been identified in the classification, where no current DQ methods exist, and these show where new methods are required as a guide for future research and DQ tool development.

View PDFchevron_right
Data quality assessment and improvement
Janne Harkonen

International Journal of Business Information Systems, 2016

Data quality has significance to companies, but is an issue that can be challenging to approach and operationalise. This study focuses on data quality from the perspective of operationalisation by analysing the practices of a company that is a world leader in its business. A model is proposed for managing data quality to enable evaluation and operationalisation. The results indicate that data quality is best ensured when organisation specific aspects are taken into account. The model acknowledges the needs of different data domains, particularly those that have master data characteristics. The proposed model can provide a starting point for operationalising data quality assessment and improvement. The consequent appreciation of data quality improves data maintenance processes, IT solutions, data quality and relevant expertise, all of which form the basis for handling the origins of products.

View PDFchevron_right
Data Quality In Information Systems* 1
Michael L Brodie

Information & Management, 1980

View PDFchevron_right
Data Quality:A Survey of Data Quality Dimensions
Payam Hassany Shariat Panahy, Aidah Mustapha

Nowadays, activities and decisions making in an organization is based on data and information obtained from data analysis, which provides various services for constructing reliable and accurate process. As data are significant resources in all organizations the quality of data is critical for managers and operating processes to identify related performance issues. Moreover, high quality data can increase opportunity for achieving top services in an organization. However, identifying various aspects of data quality from definition, dimensions, types, strategies, techniques are essential to equip methods and processes for improving data. This paper focuses on systematic review of data quality dimensions in order to use at proposed framework which combining data mining and statistical techniques to measure dependencies among dimensions and illustrate how extracting knowledge can increase process quality.

View PDFchevron_right
Data Quality Is Context Dependent
Leopoldo Bertossi

Lecture Notes in Business Information Processing, 2011

We motivate, formalize and investigate the notions of data quality assessment and data quality query answering as context dependent activities. Contexts for the assessment and usage of a data source at hand are modeled as collections of external databases, that can be materialized or virtual, and mappings within the collections and with the data source at hand. In this way, the context becomes "the complement" of the data source wrt a data integration system. The proposed model allows for natural extensions, like considering data quality predicates, and even more expressive ontologies for data quality assessment. Topics. Data quality and cleansing. ⋆ Research funded by the NSERC Strategic Network on BI (BIN, ADC05) ⋆⋆ Faculty Fellow of the IBM CAS. Also affiliated to University of Concepción (Chile). ⋆⋆⋆ Also affiliated to University of Ottawa.

View PDFchevron_right
Management of Data Quality Related Problems - Exploiting Operational Knowledge
Sunil Choenni

Proceedings of the 5th International Conference on Data Management Technologies and Applications, 2016

Dealing with data quality related problems is an important issue that all organizations face in realizing and sustaining data intensive advanced applications. Upon detecting these problems in datasets, data analysts often register them in issue tracking systems in order to address them later on categorically and collectively. As there is no standard format for registering these problems, data analysts often describe them in natural languages and subsequently rely on ad-hoc, non-systematic, and expensive solutions to categorize and resolve registered problems. In this contribution we present a formal description of an innovative data quality resolving architecture to semantically and dynamically map the descriptions of data quality related problems to data quality attributes. Through this mapping, we reduce complexity -as the dimensionality of data quality attributes is far smaller than that of the natural language space -and enable data analysts to directly use the methods and tools proposed in literature. Furthermore, through managing data quality related problems, our proposed architecture offers data quality management in a dynamic way based on user generated inputs. The paper reports on a proof of concept tool and its evaluation.

View PDFchevron_right

References (289)

  1. 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  2. 2 Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . . .
  3. 3 General Approach . . . . . . . . . . . . . . . . . . . . . . . . . . .
  4. Modeling Quality Metadata . . . . . . . . . . . . . . . . . . . . . 2.4.1 The CWM Metamodel . . . . . . . . . . . . . . . . . . . .
  5. 4.2 CWM Extension for QoD Metadata Management . . . . . 57
  6. 5 Computing Quality Metadata . . . . . . . . . . . . . . . . . . . . 2.5.1 Level I: QoD Profiling Functions . . . . . . . . . . . . . . .
  7. 5.2 Level II: QoD Constraint-Based Functions . . . . . . . . .
  8. 5.3 Level III: QoD Synopses Functions . . . . . . . . . . . . .
  9. 5.4 Level IV: QoD Mining Functions . . . . . . . . . . . . . . .
  10. 5.5 Designing Analytic Workflows for QoD Evaluation . . . .
  11. 5.6 Computing and Assigning Probabilities to QoD Dimensions 76
  12. 6 Indexing Quality Metadata . . . . . . . . . . . . . . . . . . . . . 2.6.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . .
  13. 6.2 Range-Encoded Bitmap Index for QoD measures . . . . .
  14. 7 Extending the Syntax of a Query Language . . . . . . . . . . . .
  15. 7.1 Declaration of Quality Requirements . . . . . . . . . . . .
  16. 7.2 Manipulation of Data and Quality Metadata . . . . . . . . 88 2.7.3 Quality-Extended Query Processing . . . . . . . . . . . .
  17. 8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  18. 8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  19. 8.2 Research Perspectives . . . . . . . . . . . . . . . . . . . . .
  20. 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  21. 2 Quality-Aware Integration of Biomedical Data . . . . . . . . . . 126 4.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . .
  22. 2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . .
  23. 2.3 Contributions and Perspectives . . . . . . . . . . . . . . .
  24. 3 Quality-Driven Query in Mediation Systems . . . . . . . . . . .
  25. 3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . .
  26. 3.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . .
  27. 3.3 Contributions and Perspectives . . . . . . . . . . . . . . .
  28. 4 Monitoring the Quality of Stream Data . . . . . . . . . . . . . .
  29. 4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . .
  30. 4.2 Prospective Work . . . . . . . . . . . . . . . . . . . . . . .
  31. 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  32. GIMS, http://www.cs.man.ac.uk/img/gims/
  33. DataFoundry, http://www.llnl.gov/CASC/datafoundry/ 3 TAMBIS, http://imgproj.cs.man.ac.uk/tambis/
  34. P/FDM, http://www.csd.abdn.ac.uk/ gjlk/mediator/
  35. DiscoveryLink, http://www.research.ibm.com/journal/sj/402/haas.html 6 EMBL, European Molecular Biology Laboratory: http://www.embl-heidelberg.de/ 7 Febrl, http://datamining.anu.edu.au/software/febrl/febrldoc/ 8 NCBI References Sequences http://www.ncbi.nlm.nih.gov/RefSeq/ Bibliography AGGARWAL, CHARU. 2007. Data Streams: Models and Algorithms. Springer. AGRAWAL, RAKESH, IMIELINSKI, TOMASZ, & SWAMI, ARUN N. 1993. Mining Association Rules Between Sets of Items in Large Databases. Pages 207-216 of: Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data. Washington, DC, USA.
  36. ANANTHAKRISHNA, ROHIT, CHAUDHURI, SURAJIT, & GANTI, VENKATESH. 2002. Eliminating Fuzzy Duplicates in Data Warehouses. Pages 586-597 of: Proceedings of 28th International Conference on Very Large Data Bases, VLDB 2002. Hong Kong, China.
  37. ARENAS, MARCELO, BERTOSSI, LEOPOLDO E., & CHOMICKI, JAN. 1999. Con- sistent Query Answers in Inconsistent Databases. Pages 68-79 of: Proceedings of the 18th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. Philadelphia, PA, USA.
  38. ARENAS, MARCELO, BERTOSSI, LEOPOLDO E., & CHOMICKI, JAN. 2000. Speci- fying and Querying Database Repairs Using Logic Programs with Exceptions. Pages 27-41 of: Proceedings of the 4th International Conference on Flexible Query Answering Systems, FQAS 2000. Warsaw, Poland.
  39. ARENAS, MARCELO, BERTOSSI, LEOPOLDO E., & CHOMICKI, JAN. 2003. Answer Sets for Consistent Query Answering in Inconsistent Databases. Theory and Prac- tice of Logic Programming (TPLP), 3(4-5), 393-424.
  40. BALLOU, DONALD P., & TAYI, GIRI KUMAR. 1989. Methodology for Allocating Resources for Data Quality Enhancement. Commun. ACM, 32(3), 320-329.
  41. BALLOU, DONALD P., & TAYI, GIRI KUMAR. 1999. Enhancing Data Quality in Data Warehouse Environments. Commun. ACM, 42(1), 73-78.
  42. BANSAL, NIKHIL, BLUM, AVRIM, & CHAWLA, SHUCHI. 2002. Correlation Clus- tering. Page 238 of: Proceedings of 43rd Symposium on Foundations of Computer Science, FOCS 2002. Vancouver, BC, Canada.
  43. BARBARÁ, DANIEL, GARCIA-MOLINA, HECTOR, & PORTER, DARYL. 1990. A Probalilistic Relational Data Model. Pages 60-74 of: Proceedings of the 2nd Inter- national Conference on Extending Database Technology, EDBT 1990. Lecture Notes in Computer Science, vol. 416. Venice, Italy.
  44. BARGA, ROGER S., & PU, CALTON. 1993. Accessing Imprecise Data: An Approach Based on Intervals. IEEE Data Eng. Bull., 16(2), 12-15.
  45. BASU, AYANENDRANATH, HARRIS, IAN R., & BASU, SRABASHI. 1997. Minimum Distance Estimation: The Approach Using Density-Based Distances. Handbook of Statistics, 15, 21-48.
  46. BATINI, CARLO, & SCANNAPIECO, MONICA. 2006. Data Quality: Concepts, Method- ologies and Techniques. Data-Centric Systems and Applications. Springer-Verlag.
  47. BATINI, CARLO, TIZIANA, CATARCI, & SCANNAPIECO, MONICA. 2004. A Survey of Data Quality Issues in Cooperative Systems. In: Tutorial of the 23rd Interna- tional Conference on Conceptual Modeling, ER 2004. Shanghai, China.
  48. BAXTER, ROHAN A., CHRISTEN, PETER, & CHURCHES, TIM. 2003. A Comparison of Fast Blocking Methods for Record Linkage. Pages 27-29 of: Proceedings of the KDD'03 Workshop on Data Cleaning, Record Linkage and Object Consolidation. Washington, DC, USA.
  49. BENJELLOUN, OMAR, SARMA, ANISH DAS, HALEVY, ALON, & WIDOM, JEN- NIFER. 2005 (June). The Symbiosis of Lineage and Uncertainty. Technical Report 2005-39. Stanford InfoLab, Stanford University, CA, USA.
  50. BERENGUER, GEMA, ROMERO, RAFAEL, TRUJILLO, JUAN, SERRANO, MANUEL, & PIATTINI, MARIO. 2005. A Set of Quality Indicators and Their Corresponding Metrics for Conceptual Models of Data Warehouses. Pages 95-104 of: Proceed- ings of the 7th International Conference on Data Warehousing and Knowledge Discov- ery, DaWaK 2005. Lecture Notes in Computer Science, vol. 3589. Copenhagen, Denmark.
  51. BERNSTEIN, PHILIP A., BERGSTRAESSER, THOMAS, CARLSON, JASON, PAL, SHANKAR, SANDERS, PAUL, & SHUTT, DAVID. 1999. Microsoft Repository Ver- sion 2 and the Open Information Model. Inf. Syst., 24(2), 71-98.
  52. BERTI-ÉQUILLE, LAURE. 1999b. Qualité des données multi-sources et recomman- dation multi-critère. Pages 185-204 of: Actes du congrès francophone INFormatique des ORganisations et Systèmes d'Information Décisionnels, INFORSID 1999. Toulon, France.
  53. BERTI-ÉQUILLE, LAURE. 1999c. Quality and Recommendation of Multi-Source Data for Assisting Technological Intelligence Applications. Pages 282-291 of: Proceedings of the International Conference on Database and Expert Systems Applica- tions, (DEXA'99). Lecture Notes in Computer Science, vol. 1677. Florence, Italy.
  54. BERTI-ÉQUILLE, LAURE. 2001. Integration of Biological Data and Quality-driven Source Negotiation. Pages 256-269 of: Proceedings of the 20th International Confer- ence on Conceptual Modeling, ER'2001. Lecture Notes in Computer Science, vol. 2224. Yokohama, Japan.
  55. BERTI-ÉQUILLE, LAURE. 2002. Annotation et recommandation collaboratives de documents selon leur qualité. Revue Ingénieire des Systèmes d'Information (ISI- NIS), Numéro Spécial "Recherche et Filtrage d'Information", 7(1-2/2002), 125-156.
  56. BERTI-ÉQUILLE, LAURE. 2003. Quality-Extended Query Processing for Distributed Sources. In: Proceedings of the 1rst International Workshop on Data Quality in Coop- erative Information Systems, DQCIS'2003. Siena, Italy.
  57. BERTI-ÉQUILLE, LAURE. 2003a. Quality-based Recommendation of XML Docu- ments. Journal of Digital Information Management, 1(3), 117-128.
  58. BERTI-ÉQUILLE, LAURE. 2004. Quality-Adaptive Query Processing over Dis- tributed Sources. Pages 285-296 of: Proceedings of the 9th International Conference on Information Quality, ICIQ 2004. Massachusetts Institute of Technology, Cam- bridge, MA, USA.
  59. BERTI-ÉQUILLE, LAURE, & MOUSSOUNI, FOUZIA. 2005. Quality-Aware Integra- tion and Warehousing of Genomic Data. Pages 442-454 of: Proceedings of the 10th International Conference on Information Quality, ICIQ 2005. Massachusetts Institute of Technology, Cambridge, MA, USA.
  60. BERTI-ÉQUILLE, LAURE, MOUSSOUNI, FOUZIA, & ARCADE, ANNE. 2001. Inte- gration of Biological Data on Transcriptome. Revue ISI-NIS, Numéro Spécial In- teropérabilité et Intégration des Systèmes d'Information, 6(3/2001), 61-86.
  61. BERTI-ÉQUILLE, LAURE. 2006a. Data Quality Awareness: a Case Study for Cost- Optimal Association Rule Mining. Knowl. Inf. Syst., 11(2), 191-215.
  62. BERTI-ÉQUILLE, LAURE. 2006b. Qualité des données. Techniques de l'Ingénieur, H3700, 1-19.
  63. BERTI-ÉQUILLE, LAURE. 2006c. Quality-Aware Association Rule Mining. Pages 440-449 of: Proceedings of the 10th Pacific-Asia Conference on Knowledge Discovery and Data Mining, (PAKDD 2006). Lecture Notes in Artificial Intelligence, vol. 3918. Springer.
  64. BERTOSSI, LEOPOLDO E., & BRAVO, LORETO. 2005. Consistent Query Answers in Virtual Data Integration Systems. Pages 42-83 of: Inconsistency Tolerance, Dagstuhl Seminar. Lecture Notes in Computer Science, vol. 3300. Schloss Dagstuhl, Germany.
  65. BERTOSSI, LEOPOLDO E., & CHOMICKI, JAN. 2003. Query Answering in Incon- sistent Databases. Pages 43-83 of: Logics for Emerging Applications of Databases, Dagstuhl Seminar. Schloss Dagstuhl, Germany.
  66. BERTOSSI, LEOPOLDO E., & SCHWIND, CAMILLA. 2004. Database Repairs and Analytic Tableaux. Ann. Math. Artif. Intell., 40(1-2), 5-35.
  67. BHATTACHARYA, INDRAJIT, & GETOOR, LISE. 2004. Iterative Record Linkage for Cleaning and Integration. Pages 11-18 of: Proceedings of the 9th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, DMKD 2004. Paris, France.
  68. BILENKO, MIKHAIL, & MOONEY, RAYMOND J. 2003. Adaptive Duplicate Detec- tion Using Learnable String Similarity Measures. Pages 39-48 of: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington, DC, USA.
  69. BILENKO, MIKHAIL, BASU, SUGATO, & SAHAMI, MEHRAN. 2005. Adaptive Prod- uct Normalization: Using Online Learning for Record Linkage in Comparison Shopping. Pages 58-65 of: Proceedings of the 5th IEEE International Conference on Data Mining, ICDM 2005. Houston, TX, USA.
  70. BILKE, ALEXANDER, BLEIHOLDER, JENS, BÖHM, CHRISTOPH, DRABA, KARSTEN, NAUMANN, FELIX, & WEIS, MELANIE. 2005. Automatic Data Fusion with Hum- Mer. Pages 1251-1254 of: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005. Trondheim, Norway.
  71. BOSC, PATRICK, LIETARD, NADIA, & PIVERT, OLIVIER. 2006. About Inclusion- Based Generalized Yes/No Queries in a Possibilistic Database Context. Pages 284-289 of: Proceedings of the 16th International Symposium on Foundations of Intel- ligent Systems, ISMIS 2006. Bari, Italy.
  72. BOUZEGHOUB, MOKRANE, & PERALTA, VERÓNIKA. 2004. A Framework for Anal- ysis of Data Freshness. Pages 59-67 of: Proceedings of the 1st International ACM SIGMOD 2004 Workshop on Information Quality in Information Systems, IQIS 2004. Paris, France.
  73. BRAUMANDL, REINHARD, KEIDL, MARKUS, KEMPER, ALFONS, KOSSMANN, DONALD, SELTZSAM, STEFAN, & STOCKER, KONRAD. 2001. ObjectGlobe: Open Distributed Query Processing Services on the Internet. IEEE Data Eng. Bull., 24(1), 64-70.
  74. BRAVO, LORETO, & BERTOSSI, LEOPOLDO E. 2003. Logic Programs for Consis- tently Querying Data Integration Systems. Pages 10-15 of: Proceedings of the 18th International Joint Conference on Artificial Intelligence, IJCAI-03. Acapulco, Mexico.
  75. BREUNIG, MARKUS M., KRIEGEL, HANS-PETER, NG, RAYMOND T., & SANDER, JÖRG. 2000. LOF: Identifying Density-Based Local Outliers. Pages 93-104 of: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. Dallas, TX, USA.
  76. BRIGHT, LAURA, & RASCHID, LOUIQA. 2002. Using Latency-Recency Profiles for Data Delivery on the Web. Pages 550-561 of: Proceedings of 28th International Conference on Very Large Data Bases, VLDB 2002. Hong Kong, China.
  77. BRY, FRANÇOIS. 1997. Query Answering in Information Systems with Integrity Constraints. Pages 113-130 of: Integrity and Internal Control in Information Sys- tems, IFIP TC11 Working Group 11.5, First Working Conference on Integrity and Inter- nal Control in Information Systems: Increasing the confidence in Information Systems, IICIS. Zurich, Switzerland.
  78. BUECHI, MARTIN, BORTHWICK, ANDREW, WINKEL, ADAM, & GOLDBERG, ARTHUR. 2003. ClueMaker: A Language for Approximate Record Matching. Pages 207-223 of: Proceedings of the 8th International Conference on Information Quality, ICIQ 2003. MIT, Cambridge, MA, USA.
  79. CALÌ, ANDREA, LEMBO, DOMENICO, & ROSATI, RICCARDO. 2003. On the Decid- ability and Complexity of Query Answering Over Inconsistent and Incomplete Databases. Pages 260-271 of: Proceedings of the 22nd ACM SIGACT-SIGMOD- SIGART Symposium on Principles of Database Systems, PODS. San Diego, CA, USA. CARREIRA, PAULO J. F., & GALHARDAS, HELENA. 2004. Execution of Data Map- pers. Pages 2-9 of: Proceedings of the 1st International ACM SIGMOD 2004 Work- shop on Information Quality in Information Systems, IQIS 2004. Paris, France.
  80. CARUSO, FRANCESCO, COCHINWALA, MUNIR, GANAPATHY, UMA, LALK, GAIL, & MISSIER, PAOLO. 2000. Telcordia's Database Reconciliation and Data Quality Analysis Tool. Pages 615-618 of: Proceedings of 26th International Conference on Very Large Data Bases, VLDB 2000. Cairo, Egypt.
  81. CAVALLO, ROGER, & PITTARELLI, MICHAEL. 1987. The Theory of Probabilistic Databases. Pages 71-81 of: Proceedings of 13th International Conference on Very Large Data Bases, VLDB 1987. Brighton, England.
  82. CERI, STEFANO, COCHRANE, ROBERTA, & WIDOM, JENNIFER. 2000. Practical Applications of Triggers and Constraints: Success and Lingering Issues. Pages 254-262 of: Proceedings of 26th International Conference on Very Large Data Bases, VLDB 2000. Cairo, Egypt.
  83. CHARIKAR, MOSES, GURUSWAMI, VENKATESAN, & WIRTH, ANTHONY. 2003. Clustering with Qualitative Information. Pages 524-533 of: Proceedings of 44th Symposium on Foundations of Computer Science, FOCS 2003. Cambridge, MA, USA. CHAUDHURI, SURAJIT, GANJAM, KRIS, GANTI, VENKATESH, & MOTWANI, RA- JEEV. 2003. Robust and Efficient Fuzzy Match for Online Data Cleaning. Pages 313-324 of: Proceedings of the 2003 ACM SIGMOD International Conference on Man- agement of Data. San Diego, CA, USA.
  84. CHAUDHURI, SURAJIT, GANTI, VENKATESH, & MOTWANI, RAJEEV. 2005. Robust Identification of Fuzzy Duplicates. Pages 865-876 of: Proceedings of the 21st In- ternational Conference on Data Engineering, ICDE 2005. Tokyo, Japan.
  85. CHAUDHURI, SURAJIT, GANTI, VENKATESH, & KAUSHIK, RAGHAV. 2006. A Prim- itive Operator for Similarity Joins in Data Cleaning. Page 5 of: Proceedings of the 22nd International Conference on Data Engineering, ICDE 2006. Atlanta, GA, USA.
  86. CHENG, REYNOLD, KALASHNIKOV, DMITRI V., & PRABHAKAR, SUNIL. 2003. Evaluating Probabilistic Queries over Imprecise Data. Pages 551-562 of: Pro- ceedings of the 2003 ACM SIGMOD International Conference on Management of Data. San Diego, CA, USA.
  87. CHO, JUNGHOO, & GARCIA-MOLINA, HECTOR. 2000. Synchronizing a Database to Improve Freshness. Pages 117-128 of: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. Dallas, TX, USA.
  88. CHOENNI, SUNIL, BLOK, HENK ERNST, & LEERTOUWER, ERIK. 2006. Handling Uncertainty and Ignorance in Databases: A Rule to Combine Dependent Data. Pages 295-309 of: Proceedings of 11th International Conference on Database Systems for Advanced Applications, DASFAA 2006. Lecture Notes in Computer Science, vol. 3882. Singapore.
  89. CHOMICKI, JAN. 2006. Consistent Query Answering: Opportunities and Limi- tations. Pages 527-531 of: Proceedings of 2nd International Workshop on Logical Aspects and Applications of Integrity Constraints, LAAIC 2006. Krakow, Poland.
  90. CHRISTEN, PETER, CHURCHES, TIM, & HEGLAND, MARKUS. 2004. Febrl -A Par- allel Open Source Data Linkage System. Pages 638-647 of: Proceedings of the 8th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2004. Lecture Notes in Computer Science, vol. 3056. Sydney, Australia.
  91. COHEN, WILLIAM W., RAVIKUMAR, PRADEEP, & FIENBERG, STEPHEN E. 2003. A Comparison of String Distance Metrics for Name-Matching Tasks. Pages 73-78 of: Proceedings of IJCAI-03 Workshop on Information Integration on the Web, IIWeb- 03. Acapulco, Mexico.
  92. COULON, CÉDRIC, PACITTI, ESTHER, & VALDURIEZ, PATRICK. 2005. Consistency Management for Partial Replication in a High Performance Database Cluster. Pages 809-815 of: Proceedings of 11th International Conference on Parallel and Dis- tributed Systems, ICPADS 2005, vol. 1. Fuduoka, Japan.
  93. CUI, YINGWEI, & WIDOM, JENNIFER. 2003. Lineage Tracing for General Data Warehouse Transformations. VLDB J., 12(1), 41-58.
  94. CULOTTA, ARON, & MCCALLUM, ANDREW. 2005. Joint Deduplication of Multiple Record Types in Relational Data. Pages 257-258 of: Proceedings of the 2005 ACM International Conference on Information and Knowledge Management, CIKM 2005. Bremen, Germany.
  95. DALVI, NILESH N., & SUCIU, DAN. 2004. Efficient Query Evaluation on Proba- bilistic Databases. Pages 864-875 of: Proceedings of the 30th International Confer- ence on Very Large Data Bases, VLDB 2004. Toronto, ON, Canada.
  96. DASU, TAMRAPARNI, & JOHNSON, THEODORE. 2003. Exploratory Data Mining and Data Cleaning. John Wiley.
  97. DASU, TAMRAPARNI, JOHNSON, THEODORE, MUTHUKRISHNAN, S., & SHKAPENYUK, VLADISLAV. 2002. Mining Database Structure or How To Build a Data Quality Browser. Pages 240-251 of: Proceedings of the 2002 ACM SIGMOD International Conference on Management of Data. Madison, WI, USA.
  98. DASU, TAMRAPARNI, VESONDER, GREGG T., & WRIGHT, JON R. 2003. Data Qual- ity Through Knowledge Engineering. Pages 705-710 of: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2003. Washington, DC, USA.
  99. DEMPSTER, ARTHUR PENTLAND, LAIRD, NAN M., & RUBIN, DONALD B. 1977. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, 39, 1-38.
  100. DOMINGOS, PEDRO, & HULTEN, GEOFF. 2001. Catching up with the Data: Re- search Issues in Mining Data Streams. In: Proceedings of 2001 ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, DMKD2001. Santa Barbara, CA, USA.
  101. DONG, XIN, HALEVY, ALON Y., & MADHAVAN, JAYANT. 2005. Reference Recon- ciliation in Complex Information Spaces. Pages 85-96 of: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data. Baltimore, MD, USA. DUMOUCHEL, WILLIAM, VOLINSKY, CHRIS, JOHNSON, THEODORE, CORTES, CORINNA, & PREGIBON, DARYL. 1999. Squashing Flat Files Flatter. Pages 6-15 of: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, KDD 1999. San Diego, CA, USA.
  102. ELFEKY, MOHAMED G., ELMAGARMID, AHMED K., & VERYKIOS, VASSILIOS S. 2002. TAILOR: A Record Linkage Tool Box. Pages 17-28 of: Proceedings of the 18th International Conference on Data Engineering, ICDE 2002. San Jose, CA, USA.
  103. ELMAGARMID, AHMED K., IPEIROTIS, PANAGIOTIS G., & VERYKIOS, VASSIL- IOS S. 2007. Duplicate Record Detection: A Survey. IEEE Trans. Knowl. Data Eng., 19(1), 1-16.
  104. EMBURY, SUZANNE M., BRANDT, SUE M., ROBINSON, JOHN S., SUTHERLAND, IAIN, BISBY, FRANK A., GRAY, W. ALEX, JONES, ANDREW C., & WHITE, RICHARD J. 2001. Adapting Integrity Enforcement Techniques for Data Rec- onciliation. Inf. Syst., 26(8), 657-689.
  105. ENGLISH, LARRY. 2002. Process Management and Information Quality: How Improving Information Production Processes Improves Information (Product) Quality. Pages 206-209 of: Proceedings of the Seventh International Conference on Information Quality, ICIQ 2002. MIT, Cambridge, MA, USA.
  106. ENGLISH, LARRY P. 1999. Improving Data Warehouse and Business Information Qual- ity. Wiley.
  107. FAGIN, RONALD, KOLAITIS, PHOKION G., MILLER, RENÉE J., & POPA, LUCIAN. 2003. Data Exchange: Semantics and Query Answering. Pages 207-224 of: Pro- ceedings of 9th International Conference on Database Theory, ICDT 2003. Lecture Notes in Computer Science, vol. 2572. Siena, Italy.
  108. FALOUTSOS, CHRISTOS. 2002. Sensor Data Mining: Similarity Search and Pattern Analysis. In: Tutorial of 28th International Conference on Very Large Data Bases, VLDB 2002. Hong Kong, China.
  109. FALOUTSOS, CHRISTOS, & LIN, KING-IP. 1995. FastMap: A Fast Algorithm for In- dexing, Data-Mining and Visualization of Traditional and Multimedia Datasets. Pages 163-174 of: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data. San Jose, CA, USA.
  110. FELLEGI, IVAN P., & SUNTER, A.B. 1969. A Theory for Record Linkage. Journal of the American Statistical Association, 64, 1183-1210.
  111. FLESCA, SERGIO, FURFARO, FILIPPO, & PARISI, FRANCESCO. 2005. Consistent Query Answers on Numerical Databases Under Aggregate Constraints. Pages 279-294 of: Proceedings of 10th International Symposium on Database Programming Languages, DBPL 2005. Trondheim, Norway.
  112. FOX, CHRISTOPHER J., LEVITIN, ANANY, & REDMAN, THOMAS. 1994. The Notion of Data and Its Quality Dimensions. Inf. Process. Manage., 30(1), 9-20.
  113. GALHARDAS, HELENA, FLORESCU, DANIELA, SHASHA, DENNIS, & SIMON, ERIC. 2000. AJAX: An Extensible Data Cleaning Tool. Page 590 of: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. Dallas, TX, USA. GALHARDAS, HELENA, FLORESCU, DANIELA, SHASHA, DENNIS, SIMON, ERIC, & SAITA, CRISTIAN-AUGUSTIN. 2001. Declarative Data Cleaning: Language, Model, and Algorithms. Pages 371-380 of: Proceedings of 27th International Con- ference on Very Large Data Bases, VLDB 2001. Roma, Italy.
  114. GELENBE, EROL, & HÉBRAIL, GEORGES. 1986. A Probability Model of Uncertainty in Data Bases. Pages 328-333 of: Proceedings of the Second International Conference on Data Engineering, ICDE 1986. Los Angeles, CA, USA.
  115. GRAHNE, GÖSTA. 2002. Information Integration and Incomplete Information. IEEE Data Eng. Bull., 25(3), 46-52.
  116. GRAVANO, LUIS, IPEIROTIS, PANAGIOTIS G., JAGADISH, H. V., KOUDAS, NICK, MUTHUKRISHNAN, S., PIETARINEN, LAURI, & SRIVASTAVA, DIVESH. 2001. Us- ing q-grams in a DBMS for Approximate String Processing. IEEE Data Eng. Bull., 24(4), 28-34.
  117. GRAVANO, LUIS, IPEIROTIS, PANAGIOTIS G., KOUDAS, NICK, & SRIVASTAVA, DI- VESH. 2003. Text Joins for Data Cleansing and Integration in an RDBMS. Pages 729-731 of: Proceedings of the 19th International Conference on Data Engineering, ICDE 2003. Bangalore, India.
  118. GUÉRIN, EMILIE, MOUSSOUNI, FOUZIA, & BERTI-ÉQUILLE, LAURE. 2001. Inté- gration des données sur le transcriptome. Pages 219-228 of: Actes de la journée de travail bi-thématique du GDR-PRC I3. Lyon, France.
  119. GUÉRIN, EMILIE, MARQUET, GWENAELLE, BURGUN, ANITA, LORÉAL, OLIVIER, BERTI-ÉQUILLE, LAURE, LESER, ULF, & MOUSSOUNI, FOUZIA. 2005. Integrat- ing and Warehousing Liver Gene Expression Data and Related Biomedical Re- sources in GEDAW. Pages 158-174 of: Proceedings of the 2nd International Work- shop on Data Integration in the Life Sciences, DILS 2005. San Diego, CA, USA.
  120. GUHA, SUDIPTO, RASTOGI, RAJEEV, & SHIM, KYUSEOK. 2001. Cure: An Efficient Clustering Algorithm for Large Databases. Inf. Syst., 26(1), 35-58.
  121. GUO, HONGFEI, LARSON, PER-ÅKE, RAMAKRISHNAN, RAGHU, & GOLDSTEIN, JONATHAN. 2004. Relaxed Currency and Consistency: How to Say "Good Enough" in SQL. Pages 815-826 of: Proceedings of the ACM SIGMOD Interna- tional Conference on Management of Data. Paris, France.
  122. GUO, HONGFEI, LARSON, PER-ÅKE, & RAMAKRISHNAN, RAGHU. 2005. Caching with 'Good Enough' Currency, Consistency, and Completeness. Pages 457-468 of: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005. Trondheim, Norway.
  123. HALEVY, ALON Y. 2001. Answering Queries Using Views: A Survey. VLDB J., 10(4), 270-294.
  124. HERNÁNDEZ, MAURICIO A., & STOLFO, SALVATORE J. 1998. Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem. Data Min. Knowl. Discov., 2(1), 9-37.
  125. HJALTASON, GÍSLI R., & SAMET, HANAN. 2003. Properties of Embedding Meth- ods for Similarity Searching in Metric Spaces. IEEE Trans. Pattern Anal. Mach. Intell., 25(5), 530-549.
  126. HOU, WEN-CHI, & ZHANG, ZHONGYANG. 1995. Enhancing Database Correct- ness: a Statistical Approach. Pages 223-232 of: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data. San Jose, CA, USA.
  127. HULL, RICHARD, & ZHOU, GANG. 1996. A Framework for Supporting Data In- tegration Using the Materialized and Virtual Approaches. Pages 481-492 of: Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data. Montreal, Quebec, Canada.
  128. HULTEN, GEOFF, SPENCER, LAURIE, & DOMINGOS, PEDRO. 2001. Mining time- changing data streams. Pages 97-106 of: Proceedings of the 7th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2001. San Francisco, CA, USA.
  129. HUNG, EDWARD, GETOOR, LISE, & SUBRAHMANIAN, V. S. 2003. PXML: A Prob- abilistic Semistructured Data Model and Algebra. Page 467 of: Proceedings of the 19th International Conference on Data Engineering, ICDE'03. Bangalore, India.
  130. IBRAHIM, HAMIDAH. 2002. A Strategy for Semantic Integrity Checking in Dis- tributed Databases. Pages 139-144 of: Proceedings of 9th International Conference on Parallel and Distributed Systems, ICPADS 2002. Taiwan, ROC.
  131. IMIELINSKI, TOMASZ, & LIPSKI, WITOLD JR. 1984. Incomplete Information in Relational Databases. J. ACM, 31(4), 761-791.
  132. JARKE, MATTHIAS, JEUSFELD, MANFRED A., QUIX, CHRISTOPH, & VASSILIADIS, PANOS. 1999. Architecture and Quality in Data Warehouses: An Extended Repository Approach. Inf. Syst., 24(3), 229-253.
  133. JARO, MATTHEW A. 1989. Advances in Record Linking Methodology as Applied to the 1985 Census of Tampa Florida. Journal of the American Statistical Society, 64, 1183-1210.
  134. JARO, MATTHEW A. 1995. Probabilistic Linkage of Large Public Health Data File. Statistics in Medicine, 14, 491-498.
  135. KAHN, BEVERLY K., STRONG, DIANE M., & WANG, RICHARD Y. 2002. Informa- tion Quality Benchmarks: Product and Service Performance. Commun. ACM, 45(4), 184-192.
  136. KALASHNIKOV, DMITRI V. & MEHROTRA, SHARAD. 2006. Domain-Independent Data Cleaning via Analysis of Entity-Relationship Graph. ACM Transactions on Database Systems, 31(2), 716-767.
  137. KARAKASIDIS, ALEXANDROS, VASSILIADIS, PANOS, & PITOURA, EVAGGELIA. 2005. ETL Queues for Active Data Warehousing. Pages 28-39 of: Proceedings of the 2nd International ACM SIGMOD 2005 Workshop on Information Quality in Information Systems, IQIS 2005. Baltimore, MA, USA.
  138. KAUFMAN, L., & ROUSSEEUW, PETER J. 1990. Finding Groups in Data: An Introduc- tion to Cluster Analysis. John Wiley.
  139. KNORR, EDWIN M., & NG, RAYMOND T. 1998. Algorithms for Mining Distance- Based Outliers in Large Datasets. Pages 392-403 of: Proceedings of 24rd Interna- tional Conference on Very Large Data Bases, VLDB 1998. New York City, NY, USA.
  140. KORN, FLIP, MUTHUKRISHNAN, S., & ZHU, YUNYUE. 2003. Checks and Balances: Monitoring Data Quality Problems in Network Traffic Databases. Pages 536-547 of: Proceedings of 29th International Conference on Very Large Data Bases, VLDB 2003. Berlin, Germany.
  141. LABRINIDIS, ALEXANDROS, & ROUSSOPOULOS, NICK. 2003. Balancing Perfor- mance and Data Freshness in Web Database Servers. Pages 393-404 of: Proceed- ings of 29th International Conference on Very Large Data Bases, VLDB 2003. Berlin, Germany.
  142. LACROIX, ZOE, & CRITCHLOW, TERENCE (eds). 2003. Bioinformatics: Managing Scientific Data. Morgan Kaufmann.
  143. LAKSHMANAN, LAKS V. S., & SADRI, FEREIDOON. 1994. Modeling Uncertainty in Deductive Databases. Pages 724-733 of: Proceedings of the 5th International Conference on Database and Expert Systems Applications, DEXA'94. Lecture Notes in Computer Science, vol. 856. Athens, Greece.
  144. LAVRA Č, NADA, FLACH, PETER A., & ZUPAN, BLAZ. 1999. Rule Evaluation Mea- sures: A Unifying View. Pages 174-185 of: Proceedings of the Intl. Workshop on Inductive Logic Programming, ILP 1999. Bled, Slovenia.
  145. LAZARIDIS, IOSIF, & MEHROTRA, SHARAD. 2004. Approximate Selection Queries over Imprecise Data. Pages 140-152 of: Proceedings of the 20th International Con- ference on Data Engineering, ICDE 2004. Boston, MA, USA.
  146. LEE, LILLIAN. 2001. On the Effectiveness of the Skew Divergence for Statistical Language Analysis. Artificial Intelligence and Statistics, 65-72.
  147. LEE, MONG-LI, HSU, WYNNE, & KOTHARI, VIJAY. 2004. Cleaning the Spurious Links in Data. IEEE Intelligent Systems, 19(2), 28-33.
  148. LEE, SUK KYOON. 1992. An Extended Relational Database Model for Uncertain and Imprecise Information. Pages 211-220 of: Proceedings of the 18th International Conference on Very Large Data Bases, VLDB 1992. Vancouver, Canada.
  149. LEMBO, DOMENICO, LENZERINI, MAURIZIO, & ROSATI, RICCARDO. 2002. Source Inconsistency and Incompleteness in Data Integration. In: Proceedings of the 9th International Workshop on Knowledge Representation meets Databases, KRDB 2002, vol. 54. Toulouse, France.
  150. LI, CHEN. 2003. Computing Complete Answers to Queries in the Presence of Lim- ited Access Patterns. VLDB J., 12(3), 211-227.
  151. LI, WEN-SYAN, PO, OLIVER, HSIUNG, WANG-PIN, CANDAN, K. SELÇUK, & AGRAWAL, DIVYAKANT. 2003. Freshness-Driven Adaptive Caching for Dy- namic Content Web Sites. Data Knowl. Eng., 47(2), 269-296.
  152. LIEPINS, GUNAR E., & UPPULURI, V. R. 1991. Data Quality Control: Theory and Pragmatics. New York, NY, USA: Marcel Dekker, Inc. 0-8247-8354-9.
  153. LIM, EE-PENG, SRIVASTAVA, JAIDEEP, PRABHAKAR, SATYA, & RICHARDSON, JAMES. 1993. Entity Identification in Database Integration. Pages 294-301 of: Proceedings of the 9th International Conference on Data Engineering, ICDE 1993. Vi- enna, Austria.
  154. LIN, JINXIN, & MENDELZON, ALBERTO O. 1998. Merging Databases Under Con- straints. Int. J. Cooperative Inf. Syst., 7(1), 55-76.
  155. LOSHIN, D. 2001. Enterprise Knowledge Management: The Data Quality Approach. Morgan Kaufmann.
  156. LOW, WAI LUP, LEE, MONG-LI, & LING, TOK WANG. 2001. A Knowledge-Based Approach for Duplicate Elimination in Data Cleaning. Inf. Syst., 26(8), 585-606.
  157. MANNINO, MICHAEL V., CHU, PAICHENG, & SAGER, THOMAS. 1988. Statistical Profile Estimation in Database Systems. ACM Comput. Surv., 20(3), 191-221.
  158. MARQUET, GWENAELLE, BURGUN, ANITA, MOUSSOUNI, FOUZIA, GUÉRIN, EM- ILIE, LE DUFF, FRANCK, & LORÉAL, OLIVIER. 2003. BioMeKe: an Ontology- Based Biomedical Knowledge Extraction System Devoted to Transcriptome Analysis. Studies in Health Technology and Informatics, 95, 80-86.
  159. MARTINEZ, ALEXANDRA, & HAMMER, JOACHIM. 2005. Making Quality Count in Biological Data Sources. Pages 16-27 of: Proceedings of the 2nd International ACM SIGMOD 2005 Workshop on Information Quality in Information Systems, IQIS 2005. Baltimore, MA, USA.
  160. MCCALLUM, ANDREW, NIGAM, KAMAL, & UNGAR, LYLE H. 2000. Efficient Clus- tering of High-Dimensional Data Sets with Application to Reference Matching. Pages 169-178 of: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2000. Boston, MA, USA.
  161. MCCALLUM, ANDREW, BELLARE, KEDAR, & PEREIRA, FERNANDO. 2005. A Con- ditional Random Field for Discriminatively-trained Finite-state String Edit Dis- tance. Pages 388-396 of: Proceedings of the 21rst Conference in Uncertainty in Arti- ficial Intelligence, UAI'05. Edinburgh, Scotland, UK.
  162. MCCLEAN, SALLY I., SCOTNEY, BRYAN W., & SHAPCOTT, MARY. 2001. Aggrega- tion of Imprecise and Uncertain Information in Databases. IEEE Trans. Knowl. Data Eng., 13(6), 902-912.
  163. MIHAILA, GEORGE A., RASCHID, LOUIQA, & VIDAL, MARIA-ESTHER. 2000. Us- ing Quality of Data Metadata for Source Selection and Ranking. Pages 93-98 of: Proceedings of the 3rd International Workshop on the Web and Databases, WebDB 2000. Dallas, TX, USA.
  164. MONGE, ALVARO E. 2000. Matching Algorithms within a Duplicate Detection System. IEEE Data Eng. Bull., 23(4), 14-20.
  165. MONGE, ALVARO E., & ELKAN, CHARLES. 1996. The Field Matching Problem: Algorithms and Applications. Pages 267-270 of: Proceedings of the 2nd Interna- tional Conference on Knowledge Discovery and Data Mining, KDD 1996. Portland, OR, USA.
  166. MOTRO, AMIHAI, & ANOKHIN, PHILIPP. 2006. FusionPlex: Resolution of Data Inconsistencies in the Integration of Heterogeneous Information Sources. Infor- mation Fusion, 7(2), 176-196.
  167. MOTRO, AMIHAI, & RAKOV, IGOR. 1998. Estimating the Quality of Databases. Pages 298-307 of: Proceedings of the 3rd International Conference on Flexible Query Answering Systems, FQAS'98. Roskilde, Denmark.
  168. MÜLLER, HEIKO, & NAUMANN, FELIX. 2003. Data Quality in Genome Databases. Pages 269-284 of: Proceedings of the 8th International Conference on Information Quality, ICIQ 2003. MIT, Cambridge, MA, USA.
  169. MÜLLER, HEIKO, LESER, ULF, & FREYTAG, JOHANN CHRISTOPH. 2004. Mining for Patterns in Contradictory Data. Pages 51-58 of: Proceedings of the 1st International ACM SIGMOD 2004 Workshop on Information Quality in Information Systems, IQIS 2004. Paris, France.
  170. MYLOPOULOS, JOHN, BORGIDA, ALEXANDER, JARKE, MATTHIAS, & KOUBARAKIS, MANOLIS. 1990. Telos: Representing Knowledge About Information Systems. ACM Trans. Inf. Syst., 8(4), 325-362.
  171. NAJJAR, FAÏZA, & SLIMANI, YAHYA. 1999. Cardinality Estimation of Distributed Join Queries. Pages 66-70 of: Proceedings of the 10th International DEXA Workshop on on Parallel & Distributed Databases: Innovative Applications & New Architectures. Florence, Italy.
  172. NASH, ALAN, & LUDÄSCHER, BERTRAM. 2004. Processing Unions of Conjunctive Queries with Negation under Limited Access Patterns. Pages 422-440 of: Pro- ceedings of the 9th International Conference on Extending Database Technology, EDBT 2004. Lecture Notes in Computer Science, vol. 2992. Heraklion, Crete, Greece.
  173. NAUMANN, FELIX. 2002. Quality-Driven Query Answering for Integrated Information Systems. Lecture Notes in Computer Science, vol. 2261. Springer-Verlag.
  174. NAUMANN, FELIX, LESER, ULF, & FREYTAG, JOHANN CHRISTOPH. 1999. Quality- Driven Integration of Heterogenous Information Systems. Pages 447-458 of: Proceedings of the 25th International Conference on Very Large Data Bases, VLDB 1999. Edinburgh, Scotland, UK.
  175. NAUMANN, FELIX, FREYTAG, JOHANN CHRISTOPH, & LESER, ULF. 2004. Com- pleteness of Integrated Information Sources. Inf. Syst., 29(7), 583-615.
  176. NAVARRO, GONZALO. 2001. A Guided Tour to Approximate String Matching. ACM Comput. Surv., 33(1), 31-88.
  177. NEWCOMBE, H.B., KENNEDY, J.M., AXFORD, S.J., & JAMES, A.P. 1959. Automatic Linkage of Vital Records. Science, 954-959.
  178. NEWCOMBE, HOWARD B., & KENNEDY, JAMES M. 1962. Record Linkage: Making Maximum Use of the Discriminating Power of Identifying Information. Com- mun. ACM, 5(11), 563-566.
  179. OLSON, JACK E. 2003. Data Quality: The Accuracy Dimension. Morgan Kaufmann.
  180. OLSTON, CHRISTOPHER, & WIDOM, JENNIFER. 2005. Efficient Monitoring and Querying of Distributed, Dynamic Data via Approximate Replication. IEEE Data Eng. Bull., 28(1), 11-18.
  181. OMG. 2003 (March). Common Warehouse Metamodel (CWM), Specification Version 1.1. Tech. rept. Object Management Group.
  182. PANG, HWEEHWA, JAIN, ARPIT, RAMAMRITHAM, KRITHI, & TAN, KIAN-LEE. 2005. Verifying Completeness of Relational Query Results in Data Publishing. Pages 407-418 of: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data. Baltimore, MD, USA.
  183. PAPADIMITRIOU, SPIROS, SUN, JIMENG, & FALOUTSOS, CHRISTOS. 2005. Stream- ing Pattern Discovery in Multiple Time-Series. Pages 697-708 of: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005. Trondheim, Norway.
  184. PARSONS, SIMON. 1996. Current Approaches to Handling Imperfect Information in Data and Knowledge Bases. IEEE Trans. Knowl. Data Eng., 8(3), 353-372.
  185. PASULA, HANNA, MARTHI, BHASKARA, MILCH, BRIAN, RUSSELL, STUART J., & SHPITSER, ILYA. 2002. Identity Uncertainty and Citation Matching. Pages 1401-1408 of: Proceedings of Advances in Neural Information Processing Systems 15, NIPS 2002. Vancouver, BC, Canada.
  186. PEARSON, RONALD K. 2005. Mining Imperfect Data: Dealing with Contamination and Incomplete Records. Philadelphia: SIAM.
  187. PEIM, MARTIN, FRANCONI, ENRICO, & PATON, NORMAN W. 2003. Estimating the Quality of Answers When Querying Over Description Logic Ontologies. Data Knowl. Eng., 47(1), 105-129.
  188. PERALTA, VERÓNIKA. 2006 (November). Data Quality Evaluation in Data Integra- tion Systems. Ph.D. thesis, Université de Versailles, France & Universidad de la República, Uruguay.
  189. PETROPOULOS, MICHALIS, DEUTSCH, ALIN, & PAPAKONSTANTINOU, YANNIS. 2006. Interactive Query Formulation over Web Service-Accessed Sources. Pages 253-264 of: Proceedings of the ACM SIGMOD International Conference on Manage- ment of Data. Chicago, IL, USA.
  190. POOLE, JOHN, CHANG, DAN, TOLBERT, DOUGLAS, & MELLOR, DAVID. 2003. Common Warehouse Metamodel Developer's Guide. New York: John Wiley & Sons Inc. PRADHAN, SHEKHAR. 2003. Argumentation Databases. Pages 178-193 of: Proceed- ings of 19th International Conference on Logic Programming, ICLP 2003. Mumbai, India.
  191. PYLE, DORIAN. 1999. Data Preparation for Data Mining. Morgan Kaufmann.
  192. QUASS, DALLAN, & STARKEY, P. 2003. A Comparison of Fast Blocking Methods for Record Linkage. Pages 40-42 of: Proceedings of the KDD 2003 Workshop on Data Cleaning, Record Linkage and Object Consolidation. Washington, DC, USA.
  193. RAHM, ERHARD, & DO, HONG HAI. 2000. Data Cleaning: Problems and Current Approaches. IEEE Data Eng. Bull., 23(4), 3-13.
  194. RAMAMRITHAM, KRITHI. 1993. Real-Time Databases. Distributed and Parallel Databases, 1(2), 199-226.
  195. RAMAMRITHAM, KRITHI, & CHRYSANTHIS, PANOS K. 1992. In Search of Accept- ability Citeria: Database Consistency Requirements and Transaction Correctness Properties. Pages 212-230 of: Proceedings of International Workshop on Distributed Object Management, IWDOM 1992. Edmonton, AL, Canada.
  196. RAMAN, VIJAYSHANKAR, & HELLERSTEIN, JOSEPH M. 2001. Potter's Wheel: An Interactive Data Cleaning System. Pages 381-390 of: Proceedings of 27th Interna- tional Conference on Very Large Data Bases, VLDB 2001. Roma, Italy.
  197. RE, CHRISTOPHER, DALVI, NILESH N., & SUCIU, DAN. 2006. Query Evaluation on Probabilistic Databases. IEEE Data Eng. Bull., 29(1), 25-31.
  198. REDMAN, THOMAS. 2001. Data Quality: The Field Guide. Digital Press, Elsevier.
  199. RISTAD, ERIC SVEN, & YIANILOS, PETER N. 1998. Learning String-Edit Distance. IEEE Trans. Pattern Anal. Mach. Intell., 20(5), 522-532.
  200. SAMPAIO, SANDRA DE F. MENDES, DONG, CHAO, & SAMPAIO, PEDRO. 2005. In- corporating the Timeliness Quality Dimension in Internet Query Systems. Pages 53-62 of: Proceedings of the International Workshop on Web Information Systems En- gineering, WISE 2005. New York, NY, USA.
  201. SANTIS, LUCA DE, SCANNAPIECO, MONICA, & CATARCI, TIZIANA. 2003. Trust- ing Data Quality in Cooperative Information Systems. Pages 354-369 of: Proceed- ings of CoopIS, DOA, and ODBASE -OTM Confederated International Conferences. Catania, Sicily, Italy.
  202. SARAWAGI, SUNITA, & KIRPAL, ALOK. 2004. Efficient Set Joins on Similarity Pred- icates. Pages 743-754 of: Proceedings of the 2004 ACM SIGMOD International Con- ference on Management of Data. Paris, France.
  203. SAYYADIAN, MAYSSAM, LEE, YOONKYONG, DOAN, ANHAI, & ROSENTHAL, ARNON. 2005. Tuning Schema Matching Software using Synthetic Scenarios. Pages 994-1005 of: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB 2005. Trondheim, Norway.
  204. SEGEV, ARIE, & FANG, WEIPING. 1990. Currency-Based Updates to Distributed Materialized Views. Pages 512-520 of: Proceedings of the 6th International Confer- ence on Data Engineering, ICDE 1090. Los Angeles, CA, USA.
  205. SHETH, AMIT P., WOOD, CHRISTOPHER, & KASHYAP, VIPUL. 1993. Q-Data: Using Deductive Database Technology to Improve Data Quality. Pages 23-56 of: Pro- ceedings of the International Workshop on Programming with Logic Databases, ILPS. Vancouver, BC, Canada.
  206. SIMITSIS, ALKIS, VASSILIADIS, PANOS, & SELLIS, TIMOS K. 2005. Optimizing ETL Processes in Data Warehouses. Pages 564-575 of: Proceedings of the 21st International Conference on Data Engineering, ICDE 2005. Tokyo, Japan.
  207. SINGLA, PARAG, & DOMINGOS, PEDRO. 2005. Collective Object Identification. Pages 1636-1637 of: Proceedings of the 19th International Joint Conference on Artifi- cial Intelligence, IJCAI-05. Edinburgh, Scotland, UK.
  208. STOCKINGER, KURT. 2002. Bitmap Indices for Speeding Up High-Dimensional Data Analysis. Pages 881-890 of: Proceedings of the 13th International Database and Expert Systems Applications Conference, DEXA 2002. Lecture Notes in Computer Science, vol. 2453. Aix-en-Provence, France.
  209. TAN, P.N., KUMAR, V., & SRIVASTAVA, J. 2002. Selecting the Right Interestingness Measure for Association Patterns. Pages 32-41 of: Proceedings of the 8th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2002. Edmon- ton,AL, Canada.
  210. TEJADA, SHEILA, KNOBLOCK, CRAIG A., & MINTON, STEVEN. 2001. Learning Object Identification Rules for Information Integration. Inf. Syst., 26(8), 607-633.
  211. TEJADA, SHEILA, KNOBLOCK, CRAIG A., & MINTON, STEVEN. 2002. Learning Domain-Independent String Transformation Weights for High Accuracy Object Identification. Pages 350-359 of: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2002. Edmonton, AL, Canada.
  212. THANGAVEL, ALPHONSE THANARAJ. 1999. A Clean Data Set of EST-confirmed Splice Sites from Homo Sapiens and Standards for Clean-up Procedures. Nucleic Acids Res., 27(13), 2627-2637.
  213. THEODORATOS, DIMITRI, & BOUZEGHOUB, MOKRANE. 1999. Data Currency Quality Factors in Data Warehouse Design. Page 15 of: Proceedings of the In- ternational Workshop on Design and Management of Data Warehouses, DMDW'99. Heidelberg, Germany.
  214. THEODORATOS, DIMITRI, & BOUZEGHOUB, MOKRANE. 2001. Data Currency Quality Satisfaction in the Design of a Data Warehouse. Int. J. Cooperative Inf. Syst., 10(3), 299-326.
  215. THOR, ANDREAS, & RAHM, ERHARD. 2007. MOMA -A Mapping-based Object Matching System. Pages 247-258 of: Proceedings of the 3rd Biennial Conference on Innovative Data Systems Research, CIDR 2007. Asilomar, CA, USA.
  216. VAILLANT, BENOÎT, LENCA, PHILIPPE, & LALLICH, STÉPHANE. 2004. A Cluster- ing of Interestingness Measures. Pages 290-297 of: Proceedings of the 7th Interna- tional Conference on Discovery Science, DS 2004. Padova, Italy.
  217. VASSILIADIS, PANOS, VAGENA, ZOGRAFOULA, SKIADOPOULOS, SPIROS, KARAYANNIDIS, NIKOS, & SELLIS, TIMOS K. 2001. ARKTOS: Towards the Mod- eling, Design, Control and Execution of ETL Processes. Inf. Syst., 26(8), 537-561.
  218. VASSILIADIS, PANOS, SIMITSIS, ALKIS, GEORGANTAS, PANOS, & TERROVITIS, MANOLIS. 2003. A Framework for the Design of ETL Scenarios. Pages 520-535 of: Proceedings of the 15th International Conference on Advanced Information Systems Engineering, CAiSE 2003. Klagenfurt, Austria.
  219. VERYKIOS, VASSILIOS S., MOUSTAKIDES, GEORGE V., & ELFEKY, MOHAMED G. 2003. A Bayesian Decision Model for Cost Optimal Record Matching. VLDB J., 12(1), 28-40.
  220. WANG, KE, ZHOU, SENQIANG, YANG, QIANG, & YEUNG, JACK MAN SHUN. 2005. Mining Customer Value: from Association Rules to Direct Marketing. Data Min. Knowl. Discov., 11(1), 57-79.
  221. WANG, RICHARD Y. 1998. A Product Perspective on Total Data Quality Manage- ment. Commun. ACM, 41(2), 58-65.
  222. WANG, RICHARD Y., STOREY, VEDA C., & FIRTH, CHRISTOPHER P. 1995. A Framework for Analysis of Data Quality Research. IEEE Trans. Knowl. Data Eng., 7(4), 623-640.
  223. WANG, RICHARD Y., ZIAD, MOSTAPHA, & LEE, YANG W. 2002. Data Quality. Advances in Database Systems, vol. 23. Kluwer Academic Publishers.
  224. WEIS, MELANIE, & MANOLESCU, IOANA. 2007. XClean in Action. Pages 259-262 of: Proceedings of the 3rd Biennial Conference on Innovative Data Systems Research, CIDR 2007. Asilomar, CA, USA.
  225. WEIS, MELANIE, & NAUMANN, FELIX. 2004. Detecting Duplicate Objects in XML Documents. Pages 10-19 of: Proceedings of the First International ACM SIGMOD 2004 Workshop on Information Quality in Information Systems, IQIS 2004. Paris, France.
  226. WEIS, MELANIE, NAUMANN, FELIX, & BROSY, FRANZISKA. 2006. A Duplicate Detection Benchmark for XML (and Relational) Data. In: Proceedings of the 3rd International ACM SIGMOD 2006 Workshop on Information Quality in Information Systems, IQIS 2006. Chicago, IL, USA.
  227. WIDOM, JENNIFER. 2005. Trio: A System for Integrated Management of Data, Accuracy, and Lineage. Pages 262-276 of: Proceedings of 2nd Biennial Conference on Innovative Data Systems Research. Asilomar, CA, USA.
  228. WIJSEN, JEF. 2003. Condensed Representation of Database Repairs for Consistent Query Answering. Pages 378-393 of: Proceedings of 9th International Conference on Database Theory, ICDT 2003. Siena, Italy.
  229. WINKLER, WILLIAM E. 1999. The State of Record Linkage and Current Research Prob- lems. Tech. Rept. Statistics of Income Division, Internal Revenue Service Publi- cation R99/04. U.S. Bureau of the Census, Washington, DC, USA.
  230. WINKLER, WILLIAM E. 2004. Methods for Evaluating and Creating Data Quality. Inf. Syst., 29(7), 531-550.
  231. WINKLER, WILLIAM E., & THIBAUDEAU, YVES. 1991. An Application of the Fellegi- Sunter Model of Record Linkage to the 1990 U.S. Decennial Census. Tech. Rept. Sta- tistical Research Report Series RR91/09. U.S. Bureau of the Census, Washington, DC, USA.
  232. WU, KESHENG, OTOO, EKOW J., & SHOSHANI, ARIE. 2006. Optimizing Bitmap Indices with Efficient Compression. ACM Trans. Database Syst., 31(1), 1-38.
  233. XIONG, MING, LIANG, BIYU, LAM, KAM-YIU, & GUO, YANG. 2006. Quality of Service Guarantee for Temporal Consistency of Real-Time Transactions. IEEE Trans. Knowl. Data Eng., 18(8), 1097-1110.
  234. ZHANG, TIAN, RAMAKRISHNAN, RAGHU, & LIVNY, MIRON. 1996. BIRCH: An Efficient Data Clustering Method for Very Large Databases. Pages 103-114 of: Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data. Montreal, Quebec, Canada.
  235. ZHAO, XIAOFEI, & HUANG, ZHIQIU. 2006. A Formal Framework for Reasoning on Metadata Based on CWM. Pages 371-384 of: Proceedings of 25th International Conference on Conceptual Modeling, ER 2006. Lecture Notes in Computer Science, vol. 4215. Tucson, AZ, USA.
  236. ZHU, YUNYUE, & SHASHA, DENNIS. 2002. StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time. Pages 358-369 of: Proceedings of 28th International Conference on Very Large Data Bases, VLDB 2002. Hong-Kong, China.
  237. ZHUGE, YUE, GARCIA-MOLINA, HECTOR, & WIENER, JANET L. 1997. Multiple View Consistency for Data Warehousing. Pages 289-300 of: Proceedings of the 13th International Conference on Data Engineering, ICDE 1997. Birmingham, UK.
  238. Data Sources Characteristics for MSISs . . . . . . . . . . . . . . . . . .
  239. 2 Applications and Data Types Coverage . . . . . . . . . . . . . . . . .
  240. 1 Taxonomy of Existing Techniques for Entity Resolution . . . . . . . . 16
  241. 2 Correlation Clustering Example from (Bansal et al., 2002) . . . . . . .
  242. 1 CWM Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  243. 2 Relational Metamodel of CRM_DB . . . . . . . . . . . . . . . . . . . .
  244. 3 CWM Relational Data Instance Metamodel (OMG, 2003) . . . . . . .
  245. 4 QoD Extension to CWM Data Instance Metamodel . . . . . . . . . . .
  246. 5 Example of QoD Metadata Associated to CRM_DB PRODUCT table 2.6 Example of Analytic Workflow for QoD evaluation of CRM_DB . . .
  247. 7 Syntax of Quality Contract Type Creation . . . . . . . . . . . . . . . .
  248. 8 Syntax of Call Specification in Quality Contract Type Declaration . .
  249. 9 Syntax of Quality Contract Creation . . . . . . . . . . . . . . . . . . .
  250. 10 Checking QoD Constraints on CRM_DB Granularity Levels . . . . .
  251. 11 Syntax of QWITH queries . . . . . . . . . . . . . . . . . . . . . . . . .
  252. 12 Quality-Extended Query Processing . . . . . . . . . . . . . . . . . . .
  253. 1 General Framework of Data Quality Awareness for the KDD Process 3.2 Different Levels for Measuring Data Quality . . . . . . . . . . . . . .
  254. 3 Classification Probabilities . . . . . . . . . . . . . . . . . . . . . . . . .
  255. 4 Decision Areas for Rule Post-Selection . . . . . . . . . . . . . . . . . .
  256. 5 Decision Costs for Rule Selection with a Priori Probability in [0.1,0.5] without Misclassification . . . . . . . . . . . . . . . . . . . . . . . . . .
  257. 6 Decision Costs for Rule Selection with Different Data Quality Varia- tions without Misclassification for the a Priori Probability ¼ ¼ ¾¼¼
  258. 7 Amplitude of Cost Variations Depending on Data Quality Variations without Misclassification for the a priori Probability ¼ ¼ ¾¼¼ . . .
  259. 8 Decision Status on Rule Selection for Data Quality Variations with- out Misclassification for ¼ ¼ ¾¼¼ . . . . . . . . . . . . . . . . . . . .
  260. 9 Decision Costs for Rule Selection with a Priori Probability in [0.1,0.5] with Misclassification . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  261. 1 Problems and Current Solutions for Data Quality Management . . .
  262. 2 Problem Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  263. 3 Decision Models for Handling Duplicates (Batini & Scannapieco, 2006)
  264. 4 Main Data Transformation Operators for ETL . . . . . . . . . . . . . .
  265. Data Cleaning Prototypes . . . . . . . . . . . . . . . . . . . . . . . . .
  266. 1 CRM_DB Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  267. 2 Join over FK attributes considered as correct . . . . . . . . . . . . . .
  268. 3 Join over incorrect FK attributes . . . . . . . . . . . . . . . . . . . . . .
  269. 4 Join over non FK attributes considered as correct . . . . . . . . . . . .
  270. 5 Join over incorrect non FK attributes . . . . . . . . . . . . . . . . . . .
  271. 6 Join over Deduplicated FK attributes . . . . . . . . . . . . . . . . . . .
  272. 7 Join over Deduplicated non FK attributes . . . . . . . . . . . . . . . .
  273. 8 Examples of Level I Basic Functions . . . . . . . . . . . . . . . . . . .
  274. 9 Examples of Level II Functions . . . . . . . . . . . . . . . . . . . . . .
  275. 10 Example of Statistics for Specifying SCs on Attributes Values . . . . .
  276. 11 Examples of Level III Synopses Techniques . . . . . . . . . . . . . . .
  277. 12 Examples of Level IV Classification and Partitioning Methods . . . .
  278. 13 Descriptive Metadata for Partitioning Methods . . . . . . . . . . . . .
  279. Assigning Probabilities to QoD Dimension for a DB Object Instance .
  280. 15 Range-Encoded Bitmap Index with Binning for QoD Measures . . . .
  281. 16 Examples of Quality Contract Type Declaration . . . . . . . . . . . . .
  282. 17 Example of Contract Declaration . . . . . . . . . . . . . . . . . . . . .
  283. 18 Examples of Acceptability Values per QoD Dimension . . . . . . . . .
  284. 19 Examples of Simple QWITH Queries with EXACT Constraint Checking Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  285. 20 Examples of Join QWITH Queries with EXACT Constraint Checking Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  286. 21 Examples of QWITH Queries in the Exact Constraint Checking Mode 3.
  287. Marketing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  288. 2 Fusion Function Examples for Scoring Quality Dimensions of Asso- ciation Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
  289. 3 Example of Costs of Various Decisions for Classifying Association Rules Based on Data Quality . . . . . . . . . . . . . . . . . . . . . . . .

Related papers

Quality Awareness for Data Managing and Mining
Laure Berti-Equille

2007

I would like to thank Tiziana Catarci, Helena Galhardas, and Mokrane Bouzeghoub for kindly accepting to serve as readers and reviewers for this dissertation of "Habilitation à Diriger des Recherches" and to participate to the jury.

View PDFchevron_right
Domain-Specific Characteristics of Data Quality
Ivo Odītis

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, 2017

The research discusses the issue how to describe data quality and what should be taken into account when developing an universal data quality management solution. The proposed approach is to create quality specifications for each kind of data objects and to make them executable. The specification can be executed step-by-step according to business process descriptions, ensuring the gradual accumulation of data in the database and data quality checking according to the specific use case. The described approach can be applied to check the completeness, accuracy, timeliness and consistency of accumulated data.

View PDFchevron_right
Data Quality: Developments and Directions
Eric Hughes

Integrity, Internal Control and Security in Information Systems, 2002

This paper first examines various issues on data quality and provides an overview of current research in the area. Then it focuses on research at the MITRE Corporation to use annotations to manage data quality. Next some of the emerging directions in data quality including managing quality for the semantic web and the relationships between data quality and data mining will be discussed. Finally some of the directions for data quality will be provided.

View PDFchevron_right
Developing Data Quality Aware Applications
Cesar Guerra

2009 Ninth International Conference on Quality Software, 2009

Inadequate levels of Data Quality (DQ) in Information Systems (IS) suppose a very important problem for organizations. In any case, they look for to assure data quality from earlier stages on information system developments. This paper proposes to incorporate mechanisms into software development methodologies, in order to integrate users DQ requirements aimed at assuring the data quality from the beginning of development. It brings a framework consisting of processes, activities and tasks, well defined, which would be incorporated in existent software development methodology, as METRICA V3; and therefore, to assure software product data quality created according to this methodology. The extension presented, is a guideline, and this can be extended and applied to other development methodologies like Unified Development Process.

View PDFchevron_right
User-Oriented Approach to Data Quality Evaluation
Ivo Odītis

JUCS - Journal of Universal Computer Science, 2020

The paper proposes a new data object-driven approach to data quality evaluation. It consists of three main components: (1) a data object, (2) data quality requirements, and (3) data quality evaluation process. As data quality is of relative nature, the data object and quality requirements are (a) use-case dependent and (b) defined by the user in accordance with his needs. All three components of the presented data quality model are described using graphical Domain Specific Languages (DSLs). In accordance with Model-Driven Architecture (MDA), the data quality model is built in two steps: (1) creating a platform-independent model (PIM), and (2) converting the created PIM into a platform-specific model (PSM). The PIM comprises informal specifications of data quality. The PSM describes the implementation of a data quality model, thus making it executable, enabling data object scanning and detecting data quality defects and anomalies. The proposed approach was applied to open data sets, ...

View PDFchevron_right

Related topics

  • Information Systems
  • Information Technology
  • Data Quality (Computer Science)
  • Database Management Systems
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved