Fast Statistical Grammar Induction

2007

This paper describes an evolutionary approach to the problem of inferring non-stochastic context-free grammar (CFG) from natural language (NL) corpora. The approach employs Grammar-based Classifier System (GCS). GCS is a new version of Learning Classifier Systems in which classifiers are represented by CFG in Chomsky Normal Form. GCS has been tested on the NL corpora, and it provided comparable results to the pure genetic induction approach, but in a significantly shorter time. The efficient implementation for grammar induction is very important during analysis of large text corpora.

Applied Sciences, 2021

The split-based method in a weighted context-free grammar (WCFG) induction was formalised and verified on a comprehensive set of context-free languages. WCFG is learned using a novel grammatical inference method. The proposed method learns WCFG from both positive and negative samples, whereas the weights of rules are estimated using a novel Inside–Outside Contrastive Estimation algorithm. The results showed that our approach outperforms in terms of F1 scores of other state-of-the-art methods.

1995

We describe a corpus-based induction algorithm for probabilistic context-free grammars. The algorithm employs a greedy heuristic search within a Bayesian framework, and a post-pass using the Inside-Outside algorithm. We compare the performance of our algorithm to n-gram models and the Inside-Outside algorithm in three language modeling tasks. In two of the tasks, the training data is generated by a probabilistic context-free grammar and in both tasks our algorithm outperforms the other techniques. The third task involves naturally-occurring data, and in this task our algorithm does not perform as well as n-gram models but vastly outperforms the Inside-Outside algorithm.

Lecture Notes in Computer Science, 2013

There is a growing interest in unsupervised grammar induction, which does not require syntactic annotations, but provides less accurate results than the supervised approach. Aiming at improving the accuracy of the unsupervised approach, we have resorted to additional information, which can be obtained more easily. Shallow parsing or chunking identifies the sentence constituents (noun phrases, verb phrases, etc.), but without specifying their internal structure. There exist highly accurate systems to perform this task, and thus this information is available even for languages for which large syntactically annotated corpora are lacking. In this work we have investigated how the results of a pattern-based unsupervised grammar induction system improve as data on new kind of phrases are added, leading to a significant improvement in performance. We have analyzed the results for three different languages. We have also shown that the system is able to significantly improve the results of the unsupervised system using the chunks provided by automatic chunkers.

2010

This paper describes a new method for unsupervised grammar induction based on the automatic extraction of certain patterns in the texts. Our starting hypothesis is that there exist some classes of words that function as separators, marking the beginning or the end of new constituents. Among these separators we distinguish those which trigger new levels in the parse tree. If we are able to detect these separators we can follow a very simple procedure to identify the constituents of a sentence by taking the classes of words between separators. This paper is devoted to describe the process that we have followed to automatically identify the set of separators from a corpus only annotated with Part-of-Speech (POS) tags. The proposed approach has allowed us to improve the results of previous proposals when parsing sentences from the Wall Street Journal corpus.

2010

As we all are aware that "Evolutionary Algorithms" (EAs) are modern techniques used for searching for an optimum. One can establish communication via two medium first oral communications (Speech Processing) and second is written (Text Processing). This project focuses the second medium (text processing) of communication where we communicate using any language by writing some thing. Genetic Algorithms are developed as random search methods, which have not so sensitivity on primary data of the problems. It can be used in estimation of system parameter to get the best possible solution. Genetic Algorithms have been discussed for grammar induction. Grammar Inference or Language Learning is the process of learning grammar from training data.This paper mainly discussed the various methods for learning context-free grammar (CFG) from the corpus of string and presents the approach of informant learning in the form of result for two standard grammar problems first balanced parenthesis problem and second two symbol palindrome over {a, b}.

The learning of stochastic context-free grammars from bracketed corpora by means of reestimation algorithms was explored in this work. The Comparisons of the Inside-Outside algorithm (IO) and it's bracketed version (IOb), the Viterbi-Score algorithm (VS) and the bracketed version of the Viterbi-Score (VSb) algorithms was also carried out. A complete experiment with the Penn-Tree Bank was made in order to study the performance of the resulting model and the speed of convergence of the process. A combination of the IOb with the VSb and VS models was also studied.

Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02, 2001

We present a generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts. Parameter search with EM produces higher quality analyses than previously exhibited by unsupervised systems, giving the best published unsupervised parsing results on the ATIS corpus. Experiments on Penn treebank sentences of comparable length show an even higher F 1 of 71% on nontrivial brackets. We compare distributionally induced and actual part-of-speech tags as input data, and examine extensions to the basic model. We discuss errors made by the system, compare the system to previous models, and discuss upper bounds, lower bounds, and stability for this task.

2006

We investigate prototype-driven learning for primarily unsupervised grammar induction. Prior knowledge is specified declaratively, by providing a few canonical examples of each target phrase type. This sparse prototype information is then propagated across a corpus using distributional similarity features, which augment an otherwise standard PCFG model. We show that distributional features are effective at distinguishing bracket labels, but not determining bracket locations. To improve the quality of the induced trees, we combine our PCFG induction with the CCM model of , which has complementary stengths: it identifies brackets but does not label them. Using only a handful of prototypes, we show substantial improvements over naive PCFG induction for English and Chinese grammar induction.

2005

We present a generative probabilistic model for the unsupervised learning of hierarchical natural language syntactic structure. Unlike most previous work, we do not learn a context-free grammar, but rather induce a distributional model of constituents which explicitly relates ...