Nested term recognition driven by word connection strength

2002

This paper intends to investigate what contributions syntactic functions can make towards single word term extraction. It examines the probabilistic relations between medical terms and their syntactic functions. By probabilistic relations, it means the relations between term occurrence ratios and different paths of syntactic functions. An Automatic Term Extraction (ATE) system on the basis of such extended syntactic information is built up to find out which paths of syntactic functions are good indicators for terms after training on a large medical corpus drawn from MEDLINE. Accordingly, term candidates occurring in these syntactic paths will be assigned higher weights for better probabilistic estimates. As a result, the most helpful syntactic paths in identifying terms will be found out. One linguistic motivated method, SF-Value, is proposed to weight termhood of term candidates. Results of experiments show that single word terms are extracted dominantly at a fairly good recall besides multiword terms. In this way, syntactic behaviors of single word terms prove to be especially effective in selecting single word terms. All in all, this work studies the actual usage of terms in real texts rather than a static description of their internal structures. It dynamically characterizes patterns of term usage to a much deeper degree. And this information will in turn contribute to practical ATE system.

Natural Language Processing, 2001

Traditional methods of multi-word term extraction have used hybrid methods combining linguistic and statistical information. The linguistic part of these applications is often underexploited and consists of very shallow knowledge in the form of a simple syntactic lter. In most cases no interpretation of terms is undertaken and recognition does not involve distinguishing between di erent senses of terms, although ambiguity can be a serious problem for applications such as ontology building and machine translation. The approach described uses both statistical and linguistic information combining syntax and semantics to identify, rank and disambiguate terms. We describe a new thesaurus-based similarity measure which uses semantic information to calculate the importance of di erent parts of the context in relation to the term. Results show that making use of semantic information is bene cial for both theoretical and practical aspects of terminology.

2000

Technical terms (henceforth called terms), are important elements for digital libraries. In this paperwe present a domain-independent method for the automatic extraction of multi-word terms, from machinereadable special language corpora. The method, (C-value/NC-value), combines linguistic and statistical information. The rst part, C-value enhances the common statistical measure of frequency of occurrence for term extraction, making it sensitive t o a particular ty p e o f m ulti-word terms, the nested terms. The second part, NC-value, gives: 1) a method for the extraction of term context words (words that tend to appear with terms), 2) the incorporation of information from term context words to the extraction of terms. 2 The C-value Approach This section presents the C-value approach to multiword ATR. C-value is a domain-independent method for multi-word ATR which aims to improve t h e extraction of nested terms. The method takes as input an SL corpus and produces a list of candidate multi-word terms. These are ordered by t h e i r termhood, w h i c h w e also call C-value. The output list is evaluated by a domain expert. Since the candidate terms are ranked according to their termhood, the domain expert can scan the lists starting from the top, and go as far down the list as time/money allow. The C-value approach combines linguistic and statistical information, emphasis being placed on the sta

Proceedings of the “7th …, 2010

In this paper we present a novel approach to multi–word terminology extraction combining a well–known automatic term recognition approach, the C–NC value method, with a contrastive ranking technique, aimed at refining obtained results either by filtering noise due to ...

2016

In the paper, we address the problem of recognition of non-domain phrases in terminology lists obtained with an automatic term extraction tool. We focus on identification of multi-word phrases that are general terms and discourse function expressions. We tested several methods based on domain corpora comparison and a method based on contexts of phrases identified in a large corpus of general language. We compared the results of the methods to manual annotation. The results show that the task is quite hard as the inter-annotator agreement is low. Several tested methods achieved similar overall results, although the phrase ordering varied between methods. The most successful method with the precision about 0.75 at the half of the tested list was the context based method using a modified contextual diversity coefficient.

Information extraction needs text analysis prior to processing to identify main words and the frequently repeating words that combines them. This process is often done using mining tools with manual verification or fully manual, in this paper we propose an algorithm based on co-occurrences frequencies for identifying the most frequent important words in a specific domain and the technique for building their relations to create an informative words network that can be used in ontology construction or information retrieval processes. Introduction Text processing has wide range of applications, the common process between them is the pre-processing analysis that aims at knowing the main expressions used in the selected domain. This is a core prior step that produces the input for the following stages, as it helps in recognizing the text features and structure. Each domain has its own terms that are frequently used and has a main impact on forming the information about this domain. These terms comes in different sentences in many structures, it happens to have more than word that combines it in most occurrences. This shows that those words are related, and this relation can be represented numerically by the frequencies of their occurrences. The comparison between a main word occurrences frequency and another frequent word that often combines it could give an indication about a significant relation. This is a proposed technique to use for exploring the text corpus; it could be an aiding factor to be considered with other statistical approaches or to be validated by human experts. There are many applications that are based on similar input as starting point including: Ontology construction and updating, Information Extraction, Search, Classification, Summarization and Knowledge Base Building. All these applications are based on a structure that represents the text features and relations, in a form that could be a semantic network, keywords or rules which is considered our final targeted output from this proposed technique. The rest of this paper is organized as follows, the next section is presenting the related work, and section two contains detailed explanation for the proposed algorithm components. The third section shows the conclusion and future work.

Journal of Natural Language Processing, 1999

In this paper we present a domain-independent method for the automatic extraction of multi-word(technical)terms,from machine-readable special language corpora. The method,(C-value/NC-value),combines linguistic and statistical information. The first part,C-value enhances the common statistical measure of frequency of occurrence for term extraction,making it sensitive to a particular type of multi-word terms,the nested terms.Nested terms are those which also exist as substrings of other terms.The second part,NC-value,gives two things:1)a method for the extraction of term context words(words that tend to appear with terms),2)the incorporation of information from term context words to the extraction of terms.We apply the method to a medical corpus and compare the results with those produced by frequency of occurrence also applied on the same corpus.Frequency of occurrence was chosen for the comparison since it is the most commonly used statistical method for automatic term extraction to date.We show that using C-value we improve the extraction of nested multi-word terms,while using context information(NC-value) we improve the extraction of multi-word terms in general.In the evaluation sections, we give directions for the further improvement of the method.

Proceedings of CIKM���02, 2002

We describe a new probabilistic Sentence Tree Language Modeling approach that captures term dependency patterns in Topic Detection and Tracking's (TDT) Story Link Detection task. New features of the approach include modeling the syntactic structure of sentences in documents by a sentence-bin approach and a computationally efficient algorithm for capturing the most significant sentence level term dependencies using a Maximum Spanning Tree approach, similar to Van Rijsbergen's modeling of document- ...

1996

We consider the problem of learning co-occurrence information between two word categories, or more in general between two discrete random variables taking values in a hierarchically classified domain. In particular, we consider the problem of learning the 'association norm' defined by A(x, y) = p(x, y)/p(x)p(y), where p(x, y) is the joint distribution for x and y and p(x) and p(y) are marginal distributions induced by p(x, y). We formulate this problem as a sub-task of learning the conditional distribution p(x|y), by exploiting the identity p(x|y) = A(x, y) • p(x). We propose a two-step estimation method based on the MDL principle, which works as follows: It first estimates p(x) asp using MDL, and then estimates p(x|y) for a fixed y by applying MDL on the hypothesis class of {A•p|A ∈ A} for some given class A of representations for association norm. The estimation of A is therefore obtained as a side-effect of a near optimal estimation of p(x|y). We then apply this general framework to the problem of acquiring case-frame patterns, an important task in corpus-based natural language processing. We assume that both p(x) and A(x, y) for given y are representable by a model based on a classification that exists within an existing thesaurus tree as a 'cut,' and hence p(x|y) is represented as the product of a pair of 'tree cut models.' We then devise an efficient algorithm that implements our general strategy. We tested our method by using it to actually acquire case-frame patterns and conducted syntactic disambiguation experiments using the acquired knowledge. The experimental results show that our method improves upon existing methods.

2012