Academia.eduAcademia.edu

Figure 1 – uploaded by Atsushi Fujita

See full PDFdownloadDownload figure
Table 1. Comparison of Prior Arts in Automatic Paraphrase Acquisition

Table 1 Comparison of Prior Arts in Automatic Paraphrase Acquisition

Related Figures (30)

Fig. 1. Overview of our method for expanding a given seed paraphrase lexicon.
Fig. 1. Overview of our method for expanding a given seed paraphrase lexicon.
Expanding Paraphrase Lexicons by Exploiting Generalities
Expanding Paraphrase Lexicons by Exploiting Generalities
In previous work, affix patterns have been acquired from lists of headwords in manually com- piled dictionaries. Here, we acquire them from actual paraphrase pairs by over-generation and filtering. First, candidate pairs of lexical variants are extracted from Sseeq using Algorithm 1, on the following assumption.
In previous work, affix patterns have been acquired from lists of headwords in manually com- piled dictionaries. Here, we acquire them from actual paraphrase pairs by over-generation and filtering. First, candidate pairs of lexical variants are extracted from Sseeq using Algorithm 1, on the following assumption.
Table 3. Examples of Filtering Affix Patterns
Table 3. Examples of Filtering Affix Patterns
Note: The numbers of unique stems are taken from our experimental results using the entire data for the Europarl English setting (Section 4).
Note: The numbers of unique stems are taken from our experimental results using the entire data for the Europarl English setting (Section 4).
Table 4. The Bilingual Data Table 5. The Additional Monolingual Data Used with the Corresponding Side of Bilingual Data
Table 4. The Bilingual Data Table 5. The Additional Monolingual Data Used with the Corresponding Side of Bilingual Data
Fig. 4. Number of seed paraphrase pairs.
Fig. 4. Number of seed paraphrase pairs.
Note: POS tags for English follow those used in the Penn Treebank (ftp://ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps. gz). Acronyms for French are as follows: “NC” for common noun, {“VW”, “VG”, “VK”, “VP”, “VF’} for infinitive, present participle, past participle, present indicative, and future indicative of verb, respectively, “AQ” for qualificative adjective, “Adv” for adverb, “m” for masculine, “f’ for feminine, “s” for single, “p” for plural, “1” for first person, and “3” for third person. See http://wwwllf.cnrs.fr/Gens/Abeille/French-Treebank-fr.php for the comprehensive list. Acronyms for Japanese are as follows: N for noun, V for verb, A for adjective, “IR” for irrealis form, “CO” for continuative form, “TE” for terminal form, and “HY” for hypothetical form. “x” indicates patterns specific to loan words written in katakana (the square forms of kana). Table 6. The 20 Most Frequent Suffix Patterns Obtained from the Entire Data in Each Setting
Note: POS tags for English follow those used in the Penn Treebank (ftp://ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps. gz). Acronyms for French are as follows: “NC” for common noun, {“VW”, “VG”, “VK”, “VP”, “VF’} for infinitive, present participle, past participle, present indicative, and future indicative of verb, respectively, “AQ” for qualificative adjective, “Adv” for adverb, “m” for masculine, “f’ for feminine, “s” for single, “p” for plural, “1” for first person, and “3” for third person. See http://wwwllf.cnrs.fr/Gens/Abeille/French-Treebank-fr.php for the comprehensive list. Acronyms for Japanese are as follows: N for noun, V for verb, A for adjective, “IR” for irrealis form, “CO” for continuative form, “TE” for terminal form, and “HY” for hypothetical form. “x” indicates patterns specific to loan words written in katakana (the square forms of kana). Table 6. The 20 Most Frequent Suffix Patterns Obtained from the Entire Data in Each Setting
Fig. 5. Paraphrase patterns (left: number; right: coverage). Note: Acronyms follow those in Table 6. (“generating”, “generators”) and (“X:fully”, “X:e”) from (“peacefully”, “peace”), and some regular suffixations, such as (“X:est”, “X:e”) for superlative and base forms of adjectives, that Catvar ig- nored. On the other hand, the 10 most frequent paraphrase patterns presented in Table 7 were all missing in Catvar. We also noticed the limitation of our method; apparently it cannot capture the generality that is exhibited less frequently. For instance, in the Europarl setting, we had only one pair of words, (“observation”, “observer”), of (“X:ation”, “X:er”), and no pair of words of (“X:sm”, “X:ze”), which was exhibited by (“criticism’”, “criticize”) and (“formalism”, “formalize”) in Catvar. Using the acquired affix patterns to capture lexical variants, we then induced paraphrase pat-
Fig. 5. Paraphrase patterns (left: number; right: coverage). Note: Acronyms follow those in Table 6. (“generating”, “generators”) and (“X:fully”, “X:e”) from (“peacefully”, “peace”), and some regular suffixations, such as (“X:est”, “X:e”) for superlative and base forms of adjectives, that Catvar ig- nored. On the other hand, the 10 most frequent paraphrase patterns presented in Table 7 were all missing in Catvar. We also noticed the limitation of our method; apparently it cannot capture the generality that is exhibited less frequently. For instance, in the Europarl setting, we had only one pair of words, (“observation”, “observer”), of (“X:ation”, “X:er”), and no pair of words of (“X:sm”, “X:ze”), which was exhibited by (“criticism’”, “criticize”) and (“formalism”, “formalize”) in Catvar. Using the acquired affix patterns to capture lexical variants, we then induced paraphrase pat-
Table 7. The 10 Most Frequent Prefix Patterns Obtained from the Entire Data in Each Setting
Table 7. The 10 Most Frequent Prefix Patterns Obtained from the Entire Data in Each Setting
Fig. 6. Percentage of the 1-variable, 2-variable, and 3-variable paraphrase patterns. NTCIR setting, we extracted more patterns for English than for Japanese. This could be a result of the fact that we missed some patterns by abandoning affix patterns that were supported by only short stems, such as those presented in (13). Accordingly, some seed paraphrase pairs failed to be generalized, and some partial patterns failed to be merged into a single general pattern. We are planning to address this shortcoming in our future work through an improved treatment of Japanese data, e.g., by using phonetic transcriptions of ideographical characters as well. Ae chnurn in Figure 6 moct of the acaiired naranhrace natterne cantained anlyvy ane variahle
Fig. 6. Percentage of the 1-variable, 2-variable, and 3-variable paraphrase patterns. NTCIR setting, we extracted more patterns for English than for Japanese. This could be a result of the fact that we missed some patterns by abandoning affix patterns that were supported by only short stems, such as those presented in (13). Accordingly, some seed paraphrase pairs failed to be generalized, and some partial patterns failed to be merged into a single general pattern. We are planning to address this shortcoming in our future work through an improved treatment of Japanese data, e.g., by using phonetic transcriptions of ideographical characters as well. Ae chnurn in Figure 6 moct of the acaiired naranhrace natterne cantained anlyvy ane variahle
Fig. 7. Number of seed and newly harvested paraphrase pairs and unique LHS phrases covered. Expanding Paraphrase Lexicons by Exploiting Generalities
Fig. 7. Number of seed and newly harvested paraphrase pairs and unique LHS phrases covered. Expanding Paraphrase Lexicons by Exploiting Generalities
Fig. 8. Leverage ratio (left: paraphrase pairs; right: unique LHS phrases).
Fig. 8. Leverage ratio (left: paraphrase pairs; right: unique LHS phrases).
Fig. 9. Average yield (left: seed paraphrase pairs; right: new paraphrase pairs). Table 8. Sample Paraphrase Patterns in the NTCIR English Setting and the Number of Corresponding Paraphrase Pairs
Fig. 9. Average yield (left: seed paraphrase pairs; right: new paraphrase pairs). Table 8. Sample Paraphrase Patterns in the NTCIR English Setting and the Number of Corresponding Paraphrase Pairs
Fig. 10. Number of new paraphrase pairs (cf. Figure 7).
Fig. 10. Number of new paraphrase pairs (cf. Figure 7).
Figure 12 compares the average yield of the three settings with different sizes of monolingual lata. As the sets of paraphrase patterns were almost the same in each setting, the yield of a given HS depends entirely on whether the RHS phrase of each corresponding pattern is found in the iven monolingual data. Nevertheless, the results indicate that the average yield is not significantly ncreased by scaling up the monolingual data. The main lesson is that the diversity of phrase con- tructions basically depends on the paraphrase patterns and, ultimately, on the size of underlying ilingual data.
Figure 12 compares the average yield of the three settings with different sizes of monolingual lata. As the sets of paraphrase patterns were almost the same in each setting, the yield of a given HS depends entirely on whether the RHS phrase of each corresponding pattern is found in the iven monolingual data. Nevertheless, the results indicate that the average yield is not significantly ncreased by scaling up the monolingual data. The main lesson is that the diversity of phrase con- tructions basically depends on the paraphrase patterns and, ultimately, on the size of underlying ilingual data.
Fig. 13. Number and coverage of paraphrase patterns (cf. Figure 5). Expanding Paraphrase Lexicons by Exploiting Generalities
Fig. 13. Number and coverage of paraphrase patterns (cf. Figure 5). Expanding Paraphrase Lexicons by Exploiting Generalities
Fig. 14. Number of new paraphrase pairs (cf Figure 7).
Fig. 14. Number of new paraphrase pairs (cf Figure 7).
Fig. 16. Average yield (left: seed paraphrase pairs; right: new paraphrase pairs; cf. Figure 9). induced from the entire Sseeq of Web-Def, while obvious pairs, such as (“Xy:ing X2:e”, “to X1:€ Xg:e”) and (“Xy:€ X2:ed”, “X>:ing X1:e”), were missing. Consequently, our method achieved a low leverage ratio for Web-Def only. On the other hand, the paraphrase pairs harvested with the PPDB showed the same tendencies as those of our Europarl setting. The 48.5 million paraphrase pairs in the Spys: acquired through the second smallest package (M) was larger than the 34.4 million pairs in the Sseeq of the largest package (XXXL), but, interestingly, they shared only 210 thousand pairs. In other words, 99.6% of the former were unseen even when all the available parallel data was used. This highlights the benefits of our generalization-and-instantiation approach in leveraging large-scale monolingual data.
Fig. 16. Average yield (left: seed paraphrase pairs; right: new paraphrase pairs; cf. Figure 9). induced from the entire Sseeq of Web-Def, while obvious pairs, such as (“Xy:ing X2:e”, “to X1:€ Xg:e”) and (“Xy:€ X2:ed”, “X>:ing X1:e”), were missing. Consequently, our method achieved a low leverage ratio for Web-Def only. On the other hand, the paraphrase pairs harvested with the PPDB showed the same tendencies as those of our Europarl setting. The 48.5 million paraphrase pairs in the Spys: acquired through the second smallest package (M) was larger than the 34.4 million pairs in the Sseeq of the largest package (XXXL), but, interestingly, they shared only 210 thousand pairs. In other words, 99.6% of the former were unseen even when all the available parallel data was used. This highlights the benefits of our generalization-and-instantiation approach in leveraging large-scale monolingual data.
Table 10. Classification Labels Table 11. Classification Results by the Authors
Table 10. Classification Labels Table 11. Classification Results by the Authors
Instead of numerical scoring, we asked our evaluators to classify each example into one of several predefined classes. Table 10 shows the lists of classes for each of two different granularity levels. To guide evaluators on the classification task, we provided them with decision trees branching on a set of basic questions (see Appendices A and B). Thus, assigning a class label to an example amounts to providing an answer to each of a series of basic questions. Table 11 shows classification results for the examples in (14) to (17) as determined by the authors.
Instead of numerical scoring, we asked our evaluators to classify each example into one of several predefined classes. Table 10 shows the lists of classes for each of two different granularity levels. To guide evaluators on the classification task, we provided them with decision trees branching on a set of basic questions (see Appendices A and B). Thus, assigning a class label to an example amounts to providing an answer to each of a series of basic questions. Table 11 shows classification results for the examples in (14) to (17) as determined by the authors.
Table 12. Number of Generated Paraphrases and Example Units
Table 12. Number of Generated Paraphrases and Example Units
Table 15. Precision of Paraphrase Substitution (cf. Table 14)
Table 15. Precision of Paraphrase Substitution (cf. Table 14)
Fig. 17. Decision tree for evaluating grammaticality. No, more than that. Label it “Major Problem.”
Fig. 17. Decision tree for evaluating grammaticality. No, more than that. Label it “Major Problem.”
Fig. 18. Decision tree for evaluating equivalence of meaning. Something else including both loss and addition. Label it “Ignorable Change.” ACM Trans. Asian Low-Resour. Lang. Inf. Process., Vol. 17, No. 2, Article 13. Publication date: January 2018.
Fig. 18. Decision tree for evaluating equivalence of meaning. Something else including both loss and addition. Label it “Ignorable Change.” ACM Trans. Asian Low-Resour. Lang. Inf. Process., Vol. 17, No. 2, Article 13. Publication date: January 2018.
Related topics:
Computer ScienceArtificial IntelligenceNatural Language Processing
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved