Automatic Evaluation

This paper describes an algorithm for automatically extracting multiword expressions (MWEs) from a corpus. The algorithm is nodebased, i.e. extracts MWEs that contain the item specified by the user, using a fixed window-size around the node. The main idea is to detect the frequency anomalies that occur at the starting and ending points of an ngram that constitutes a MWE. This is achieved by locally comparing matrices of observed frequencies to matrices of expected frequencies, and determining, for each individual input, one or more sub-sequences that have the highest probability of being a MWE. Top-performing sub-sequences are then combined in a scoreaggregation and ranking stage, thus producing a single list of score-ranked MWE candidates, without having to indiscriminately generate all possible sub-sequences of the input strings. The knowledge-poor and computationally efficient algorithm attempts to solve certain recurring problems in MWE extraction, such as the inability to deal with MWEs of arbitrary length, the repetitive counting of nested ngrams, and excessive sensitivity to frequency. Evaluation results show that the best-performing version generates top-50 precision values between 0.71 and 0.88 on Turkish and English data, and performs better than the baseline method even at n=1000.

One of the main reasons that justify the student's failure in (introductory) programming courses is the lack of motivation that impacts on the knowledge acquisition process, affecting learning results. As soon as students face the difficulties concerning the development of algorithms or the coding in a programming language, they give up and do not try harder to solve other problems; they think it is a demanding activity and feel frustrated. In this paper we describe in detail an experiment conducted to verify the effectiveness, in terms of the increase in motivation and in knowledge acquisition, of combining program Animation tools with the immediate feedback provided by Automatic Evaluations Systems.

) et de 12 textes calibrés de valence émotionnelle positive (joie et bonne surprise) et négative (peur, colère, dégoût, tristesse et mauvaise surprise). Les deux types de tests effectués confirment la pertinence psychologique d'EMOVAL. Cependant des limites quant à la possibilité de conclure sur la dimension d'activation sont discutées. La métanorme présentée dans cet article peut être obtenue auprès des auteurs.

This paper studies how granularity of machine translation evaluation can be extended from sentence to document level. While most state-of-the-art evaluation metrics focus on the sentence level, we emphasize the importance of document structure, showing that lexical cohesion is a critical feature to highlight the superior quality of human translation to machine translation, which uses cohesive devices to tie salient words between sentences together as a text. An experiment shows that this feature can bring forth a 3-5% improvement in the correlation of automatic evaluation results with human judgments of machine translation outputs at the document level.

Having students express their understanding of difficult, new material in their own words is an effective method to deepen their comprehension and learning. Summary Street® is a computer tutor that offers a supportive context for students to practice this activity by means of summary writing, guiding them through successive cycles of revising with feedback on the content of their writing. Automatic evaluation of the content of student summaries is enabled by Latent Semantic Analysis (LSA). This article describes an experimental study of the comprehension and writing tutor, in which 8th-grade students practiced summary writing over a 4-week period, either with or without the guidance of the tutor. Students using Summary Street® scored significantly higher on an independent comprehension test than the control group for test items that tapped gist level comprehension. Their summaries were also judged to be significantly superior in blind scoring on several measures of writing quality. ...

Question Answering (QA) is a specialized area in the field of Information Retrieval (IR). The QA systems are concerned with providing relevant answers in response to questions proposed in natural language. QA is therefore composed of three distinct modules, each of which has a core component beside other supplementary components. These three core components are: question classification, information retrieval, and answer extraction. Question classification plays an essential role in QA systems by classifying the submitted question according to its type. Information retrieval is very important for question answering, because if no correct answers are present in a document, no further processing could be carried out to find an answer. Finally, answer extraction aims to retrieve the answer for a question asked by the user. This survey paper provides an overview of Question-Answering and its system architecture, as well as the previous related work comparing each research against the oth...

Building Accessible Custom UI Controls: A Comprehensive Guide for 508 Compliance addresses the critical need for developing inclusive web applications through customized user interface components. This technical article explores the implementation of accessible UI controls that serve both standard users and those with

In this paper, we describe a News Story Gisting system that generates a 10-word short summary of a news story. This system uses a machine learning technique to combine linguistic, statistical and positional information in order to generate an appropriate summary. We also present the results of an automatic evaluation of this system with respect to the performance of other baseline summarisers using the new ROUGE evaluation metric.

In this paper, we present the HybridTrim system which uses a machine learning technique to combine linguistic, statistical and positional information to identify topic labels for headlines in a text. We compare our system with the Topiary system which, in contrast, uses a statistical learning approach to finding topic descriptors for headlines. The Topiary system, developed at the University of Maryland with BBN, was the top performing headline generation system at DUC 2004. Topiary-style headlines consist of a number of general topic labels followed by a compressed version of the lead sentence of a news story. The Topiary system uses a statistical learning approach to finding topic labels. The performance of these systems is evaluated using the ROUGE evaluation suite on the DUC 2004 news stories collection.

The relationship between QT interval dispersion and dipyridamole-induced, transient myocardial ischemia was assessed in 32 male patients with ischemic heart disease. A standardized, high dose dipyridamole-ECG stress test was used as dipyridamole infusion of 0,56 mg/kg applied i. v. for 4 min followed by 4 min interval of no-dose with ECG and blood pressure monitoring, and in negative test -by a dipyridamole infusion of 0,28mg/kg i. v. for 2 min. Seventeen patients (53%) developed a transient myocardial ischemia with duration of 20 ±7 (4-40) min during the dipyridamole infusion while 15 ones (47%) did not. No regular dynamics and significant differences in the values of total QT interval dispersion and maximum adjacent QT interval dis persion estimated before, during and after the dipyridamole infusion could be established. It was supposed that the severity, duration and time for development of dipyridamole-induced transient myocardial ischemia were not sufficient to generate a dispersion in ventricular repolarization detectable as changes in QT disper sion parameters on surface ECG. The combination of QT dispersion with various non-invasive markers of arrhythmogenic mechanisms could help the estimation of arrhythmogenic risk in the patients with ischemic heart disease.

Human assessment is often considered the gold standard in evaluation of translation systems. But in order for the evaluation to be meaningful, the rankings obtained from human assessment must be consistent and repeatable. Recent analysis by raised several concerns about the rankings derived from human assessments of English-Czech translation systems in the 2010 Workshop on Machine Translation. We extend their analysis to all of the ranking tasks from 2010 and 2011, and show through an extension of their reasoning that the ranking is naturally cast as an instance of finding the minimum feedback arc set in a tournament, a wellknown NP-complete problem. All instances of this problem in the workshop data are efficiently solvable, but in some cases the rankings it produces are surprisingly different from the ones previously published. This leads to strong caveats and recommendations for both producers and consumers of these rankings.

Graph and tree transducers have been applied in many NLP areas-among them, machine translation, summarization, parsing, and text generation. In particular, the successful use of tree rewriting transducers for the introduction of syntactic structures in statistical machine translation contributed to their popularity. However, the potential of such transducers is limited because they do not handle graphs and because they "consume" the source structure in that they rewrite it instead of leaving it intact for intermediate consultations. In this paper, we describe an open source tree and graph transducer interpreter, which combines the advantages of graph transducers and two-tape Finite State Transducers and surpasses the limitations of state-of-the-art tree rewriting transducers. Along with the transducer, we present a graph grammar development environment that supports the compilation and maintenance of graph transducer grammatical and lexical resources. Such an environment is indispensable for any effort to create consistent large coverage NLP-resources by human experts.

Automatic evaluation metrics for Machine Translation (MT) systems, such as BLEU, METEOR and the related NIST metric, are becoming increasingly important in MT research and development. This paper presents a significance testdriven comparison of n-gram-based automatic MT evaluation metrics. Statistical significance tests use bootstrapping methods to estimate the reliability of automatic machine translation evaluations. Based on this reliability estimation, we study the characteristics of different MT evaluation metrics and how to construct reliable and efficient evaluation suites.

शाबर मंत्र भारतीय धार्मिक एवं आध्यात्मिक परंपराओं में एक विशिष्ट स्थान रखते हैं जो लोक प्रथाओं तथा तांत्रिक विश्वासों में गहराई से निहित हैं। संस्कृत के शास्त्रीय मंत्रों के स्वरूप के विपरीत ये मंत्र क्षेत्रीय भाषाओं में रचित होते हैं जिससे आम जनता हेतु इनकी पहुँच तथा समझ आसान हो जाती है। प्रस्तुत आलेख शाबर मंत्रों के ऐतिहासिक विकास, सामाजिक स्वीकृति एवं इनके सांस्कृतिक महत्व का विश्लेषण करता है साथ ही संकट निवारण, सुरक्षा, उपचार तथा समृद्धि में इनकी भूमिका को उजागर करता है। इस आलेख में गुरु गोरखनाथ तथा नाथ संप्रदाय के योगदान पर भी प्रकाश डाला गया है जो शाबर मंत्रों के व्यापक धार्मिक परिदृश्य में एकीकृत होने की प्रक्रिया को दर्शाता है। यद्यपि कि शाबर मंत्रों के प्रभाव को लेकर विवाद भी बना हुआ है तथा इनके गूढ़ स्वभाव एवं वैज्ञानिक प्रमाणों की अनुपस्थिति के कारण भी इन्हें संदेह की दृष्टि से देखा जाता है। कुछ आलोचकों का तर्क है कि इन मंत्रों की प्रभावशीलता मुख्य रूप से मनोवैज्ञानिक होती है जो विश्वास, आत्म-सुझाव आदि से प्रभावित होती है न कि किसी अलौकिक शक्ति से तथापि डिजिटल मीडिया के प्रसार के साथ ही आधुनिक समय में इनकी निरंतर प्रासंगिकता एवं लोकप्रियता में वृद्धि देखी गई है। यह आलेख शाबर मंत्रों के व्यावसायीकरण, इनके कानूनी एवं नैतिक पहलुओं तथा आधुनिक समाज में इनकी भूमिका पर भी विचार करता है। समाजशास्त्रीय दृष्टिकोण से इन मंत्रों का अध्ययन यह समझने हेतु भी आवश्यक है कि परंपरागत आध्यात्मिक विश्वास किस प्रकार समकालीन समाज में संरक्षित एवं पुनः प्रतिष्ठित हो रहे हैं।

Abstract: This work focuses on how we can improve automatic evaluation based on guidelines inspection throughout the life cycle of Web applications by mapping guideline concepts to different artifacts produced during the development process. In order to support such an evaluation approach, we present a tool for automated evaluation based on guidelines reviews. Key words: automated usability evaluation, ergonomic inspection, usability, accessibility, Web sites, development process.

We present the application of Principal Component Analysis for data acquired during the design of a natural gesture interface. We investigate the concept of an eigengesture for motion capture hand gesture data and present the visualisation of principal components obtained in the course of conducted experiments. We also show the influence of dimensionality reduction on reconstructed gesture data quality.

World Wide Web content continuously grows in size and importance. Furthermore, users ask Web search engines to satisfy increasingly disparate information needs. New techniques and tools are constantly developed aimed at assisting users in the interaction with the Web search engine. Query recommender systems suggesting interesting queries to users are an example of such tools. Most query recommendation techniques are based on the knowledge of the behaviors of past users of the search engine recorded in query logs. A recent query-log mining approach for query recommendation is based on Query Flow Graphs (QFG). In this paper we propose an evaluation of the effects of time on this query recommendation model. As users interests change over time, the knowledge extracted from query logs may suffer an aging effect as new interesting topics appear. In order to validate experimentally this hypothesis, we build different query flow graphs from the queries belonging to a large query log of a real-world search engine. Each query flow graph is built on distinct query log segments. Then, we generate recommendations on different sets of queries. Results are assessed both by means of human judgments and by using an automatic evaluator showing that the models inexorably age.

Nowadays, the evaluation of the accessibility of Enterprise Web Information Systems is based on the Web Content Accessibility Guidelines (WCAG 2.0), created by the World Wide Web Consortium (W3C) in 2008 and adopted in 2012 by the International Organization of Standardization (ISO) as ISO/IEC 40500 standard. Web accessibility evaluation tools are software programs or online services that help developers determine if web content meets the accessibility guidelines. This paper shows the extension of an existing online tool, with the aim of exposing its functionality also as an Application Program Interface (API). The original tool is used through a Web user interface, and this extension exposes functionality through web services based on REST technology, so that developers can invoke them from their own applications for accessibility testing.

Life cycle monitoring of civil infrastructure such as bridges and buildings is critical to the long-term operational cost and safety of aging structures. The widespread use of Structural Health Monitoring (SHM) systems is limited due to unavailability of specialized data acquisition equipment, high cost of generic equipment, and absence of fully automatic decision support systems. The goals of the presented project include: first, design of a Wireless Intelligent Sensor and Actuator Network (WISAN) and creation of an inexpensive set of instrumentation for the tasks of structural health monitoring; second, development of a SHM method, which is suitable for autonomous structural health monitoring. The design of the wireless sensor network is aimed at applications of structural health monitoring, addressing the issues of achieving a low cost per sensor, higher reliability, sources of energy for the network nodes, energy-efficient distribution of the computational load, security and coexistence in the ISM radio bands. The practical applicability of the sensor network is increased through utilization of computational intelligence and support of signal generation capabilities. The automated SHM method is based on the method of modal strain energy, though other SHM methods will be supported as well. The automation tasks include automation of the modal identification through ambient vibrations, classification of the acquired mode shapes, and automatic evaluation of the structural health.

Abstract: This paper uses a neural theory of emotional consciousness to develop a novel account of conscience and moral intuition. Emotions are both cognitive appraisals and somatic perceptions, performed simultaneously by interacting brain areas. Conscience is a neural process that generates emotional intuitions combining bodily reactions with cognitive appraisal concerning a special subset of goals. This account explains how moral intuitions can be both cognitive and emotional, and why both moral agreement and disagreement are common. Our theory of conscience is both descriptive and normative, showing how to evaluate the ethical and epistemological validity of intuitions. Key words: Cognitive appraisal, conscience, consciousness, emotion, intuition, moral judgment, normative, somatic perception, Conscience is the internal sense of moral goodness or badness of the conduct of oneself or others. The products of conscience are moral intuitions, which are the feelings that some acts ar...

Considerable attention has been given to the problem of Multiword Expression (MWE) identification and treatment, for NLP tasks like parsing and generation, to improve the quality of results. Statistical methods have been often employed for MWE identification, as an inexpensive and language independent way of finding co-occurrence patterns. On the other hand, more linguistically motivated methods for identification, which employ information such as POS filters and lexical alignment between languages, can produce more targeted candidate lists. In this paper we propose a hybrid approach that combines the strenghts of different sources of information using a machine learning algorithm to produce more robust and precise results. Automatic evaluation on gold standards shows that the performance of our hybrid method is superior to the individual results of statistical and alignment-based MWE extraction approaches for Portuguese and for English. This method can be used to aid lexicographic work by providing a more targeted MWE candidate list.

This study delves into the relatively unexplored domain of natural language processing for the Kazakh languagea language with limited computational resources. The paper dissects the effectiveness of diffusion models and transformers in generating text, specifically paraphrases, which is a critical aspect of machine learning applications such as chatbots, virtual assistants, and automated translation services. The researchers methodically adapt these advanced models to understand and generate Kazakh text, tackling the unique challenges posed by the language's complex morphology. The paper is comprehensive in its approach, covering everything from the initial adaptation of the models to the Kazakh language context, to the creation of specialized tokenizer tools, to the translation and preparation of datasets for effective training. Through rigorous testing and performance analysis, the study identifies the strengths and weaknesses of each model type. This is critical as it informs the direction of future research and model development, with the goal of enhancing the fluency and accuracy of automated Kazakh text generation. The paper also discusses the broader impact of its findings, suggesting that the methodologies and insights gained could inform similar efforts in other low-resource languages, thereby contributing to the global field of NLP. The research concludes with reflections on the implications of their findings for the ongoing development of machine learning technologies, asserting the potential of these technologies to accommodate the intricacies of any language, given the right approach and resources. This work not only advances the technical capabilities for Kazakh text generation but also serves as a testament to the potential of machine learning to bridge language gaps and foster greater digital inclusivity.

Paraphrasing is an important aspect of language competence; however, EFL learners have long had difficulty paraphrasing in their writing owing to their limited language proficiency. Therefore, automatic paraphrase suggestion systems can be useful for writers. In this paper, we present PREFER1, a paraphrase reference tool for helping language learners improve their writing skills. In this paper, we attempt to transform the paraphrase generation problem into a graphical problem in which the phrases are treated as nodes ...

This paper studies how granularity of machine translation evaluation can be extended from sentence to document level. While most state-of-the-art evaluation metrics focus on the sentence level, we emphasize the importance of document structure, showing that lexical cohesion is a critical feature to highlight the superior quality of human translation to machine translation, which uses cohesive devices to tie salient words between sentences together as a text. An experiment shows that this feature can bring forth a 3-5% improvement in the correlation of automatic evaluation results with human judgments of machine translation outputs at the document level.

For a given piece of music, there often exist multiple versions belonging to the symbolic (e.g., MIDI representations), acoustic (audio recordings), or visual (sheet music) domain. Each type of information allows for applying specialized, domain-specific approaches to music analysis tasks. In this paper, we formulate the idea of a cross-version analysis for comparing and/or combining analysis results from different representations. As an example, we realize this idea in the context of harmonic analysis to automatically evaluate MIDI-based chord labeling procedures using annotations given for corresponding audio recordings. To this end, one needs reliable synchronization procedures that automatically establish the musical relationship between the multiple versions of a given piece. This becomes a hard problem when there are significant local deviations in these versions. We introduce a novel late-fusion approach that combines different alignment procedures in order to identify reliable parts in synchronization results. Then, the cross-version comparison of the various chord labeling results is performed only on the basis of the reliable parts. Finally, we show how inconsistencies in these results across the different versions allow for a quantitative and qualitative evaluation, which not only indicates limitations of the employed chord labeling strategies but also deepens the understanding of the underlying music material.

Over the past few years, tissue microarray (TMA) technology has been established as a standard method for assessing the expression of proteins or genes across large sets of tissue specimens. It is being adopted increasingly among leading research institutions around the world and utilized in cancer research in parallel with the cDNA microarray technology. This article summarizes various aspects of cancer understanding and diagnostics in which TMA has had great impact. Although tremendous advances continue to be made to facilitate imaging and archiving of TMA specimens, automatic evaluation and quantitative analysis of TMA still remains an important challenge for modern investigators.

This work presents a system for automatically evaluating the interaction that exists between the atmosphere and the ocean's surface. Monitoring and evaluating the ocean's carbon exchange process is a function that requires working with a great amount of data: satellite images and in situ vessel's data. The system presented in this study focuses on computational intelligence. The study presents an intelligent system based on the use of case-based reasoning (CBR) systems and offers a distributed model for such an interaction. Moreover, the system takes into account the fact that the working environment is dynamic and therefore it requires autonomous models that evolve over time. In order to resolve this problem, an intelligent environment has been developed, based on the use of CBR systems, which are capable of handling several goals, by constructing plans from the data obtained through satellite images and research vessels, acquiring knowledge and adapting to environmental changes. The artificial intelligence system has been successfully tested in the North Atlantic Ocean, and the results obtained will be presented in this study.

Web search evaluation is the process of measuring the effectiveness of a Web search system. Such an evaluation helps in identifying the most effective one and helps the users to find the required information with less effort. Web search systems have been evaluated in many different ways in the last 15 years. In this paper, we review some of the efforts made for the evaluation of Web search systems. We discuss these evaluation studies by classifying them into eight different categories. As the size and content of Web is changing rapidly, and hence, the Web search techniques, we mention the necessity of an automatic evaluation methodology. But, at the same time, we emphasize that the significance of user based evaluation can not be neglected. Finally, we conclude that an automatic evaluation method that models users' feedback based evaluation is required for the effective and realistic evaluation of Web search systems.

This paper reports an overview of the evaluation campaign results of the IWSLT 2005 workshop1. The BTEC corpus, which consists of typical travel domain phrases, was used. Data for the five language pairs Arabic/Chinese/Japanese/Korean to English and English to ...

Automatic lexical alignment is a vital step for empirical machine translation, and although good results can be obtained with existent models (e.g. Giza++), more precise alignment is still needed for successfully handling complex constructions such as multiword expressions. In this paper we propose an approach for lexical alignment combining statistical and linguistic information. We describe the development of a baseline discriminative aligner and a set of language dependent post-processing functions that allow the inclusion of shallow linguistic knowledge. The post-processing functions were designed to significantly improve word alignment mainly on verb-particle constructs both over our baseline and over Giza++.

Web Content Accessibility Guidelines (WCAG) from W3C consist of a set of 65 checkpoints or specifications that Web pages should accomplish in order to be accessible to people with disabilities or using alternative browsers. Many of these 65 checkpoints can only be checked by a human operator, thus implying a very high cost for full evaluation. However, some checkpoints can be automatically evaluated, thus spotting accessibility barriers in a very effective manner. Well known tools like Bobby, Tawdis or WebXACT evaluate Web accessibility by using a mixture of automated, manual and semiautomated evaluations. However, the automation degree of these Web evaluations is not the same for each of these tools. Since WCAG are not written in a formalized manner, these evaluators may have different "interpretations" of what these rules mean. As a result, different evaluation results might be obtained for a single page depending on which evaluation tool is being used. Taking into account that a fully automated Web accessibility evaluation is not feasible because of the nature of WCAG, this paper evaluates the WCAG automation coverage of some well known Web accessibility evaluation tools spotting their weaknesses and differences. We also provide a formalized specification for those checkpoints where these differences have been detected, thus challenging nowaday's tools for more automated coverage.

The principles of the Kohonen and counterpropagation artificial neural network (K-ANN and CP-ANN) learning strategy is described. The use of both methods (with the emphasis on CP-ANNs) is explained on several examples from analytical chemistry. The problems discussed in this presentation are: selection of a set of representative objects from a large number of multi-variate measurements, clustering of multi-variate experiments (multi-component analyses), generation of logical 'if-then' rules for an automatic decision making process, automatic evaluation of the quality of spectra based on their shape, spectra recording, quantitative decisions using weight maps, multivariate modelling of a property, generation of multi-variate response surfaces from a generated CP-ANN model, and estimation of missing variable-values.

La accesibilidad web es aquella característica que permite que cualquier persona sin importar sus condiciones pueda acceder a los contenidos de los sitios web. El uso de validadores automáticos permite realizar un primer análisis acerca del nivel de accesibilidad de un sitio web. Sin embargo, la selección de estas herramientas tiende a no ser trivial, debido a las diferencias técnicas que presenta cada validador. El objetivo de este trabajo fue realizar un análisis comparativo entre validadores automáticos de accesibilidad, buscando establecer criterios de selección a la hora de elegir un validador para realizar una evaluación de accesibilidad. Se propuso una metodología basada en tres etapas que permitieron seleccionar 14 validadores que fueron analizados bajo diferentes criterios. Además, permitió concluir que ninguno de los validadores analizados cumple con todas las características, por lo que no sería suficiente seleccionar solo uno para realizar un proceso de evaluación de accesibilidad. Por lo tanto, este proceso metodológico se constituye como una herramienta de gran utilidad para la selección de validadores de accesibilidad. Palabras clave: validadores de accesibilidad; validadores automáticos; accesibilidad; herramientas de evaluación de accesibilidad web; evaluación automática.

Some previous studies (e.g. that carried out by Van Bruggen et al. in 2004) have pointed to a need for additional research in order to firmly establish the usefulness of LSA (latent semantic analysis) parameters for automatic evaluation of academic essays. The extreme variability in approaches to this technique makes it difficult to identify the most efficient parameters and the optimum combination. With this goal in mind, we conducted a high spectrum study to investigate the efficiency of some of the major LSA parameters in small-scale corpora. We used two specific domain corpora that differed in the structure of the text (one containing only technical terms and the other with more tangential information). Using these corpora we tested different semantic spaces, formed by applying different parameters and different methods of comparing the texts. Parameters varied included weighting functions (Log-IDF or Log-Entropy), dimensionality reduction (truncating the matrices after SVD to a set percentage of dimensions), methods of forming pseudo-documents (vector sum and folding-in) and measures of similarity (cosine or Euclidean distances). We also included two groups of essays to be graded, one written by experts and other by non-experts. Both groups were evaluated by three human graders and also by LSA. We extracted the correlations of each LSA condition with human graders, and conducted an ANOVA to analyse which parameter combination correlates best. Results suggest that distances are more efficient in academic essay evaluation than cosines. We found no clear evidence that the classical LSA protocol works systematically better than some simpler version (the classical protocol achieves the best performance only for some combinations of parameters in a few cases), and found that the benefits of reducing dimensionality arise only when the essays are introduced into semantic spaces using the folding-in method.

We extend the original entity-based coherence model (Barzilay and Lapata, 2008) by learning from more fine-grained coherence preferences in training data. We associate multiple ranks with the set of permutations originating from the same source document, as opposed to the original pairwise rankings. We also study the effect of the permutations used in training, and the effect of the coreference component used in entity extraction. With no additional manual annotations required, our extended model is able to outperform the original model on two tasks: sentence ordering and summary coherence rating.

In this research the outcome of an aVective priming experiment is shown to critically depend on the frequency of occurrence of the target words used. Low frequency target words (5.7 occurrences per million words) resulted in an aVective congruency eVect, i.e., faster responses following aVectively congruent than incongruent primes. High frequency target words (32.6 occurrences per million) resulted in a reverse priming eVect, i.e., faster responses following incongruent than congruent primes. The size of the congruency eVect was larger than the size of the reverse priming eVect, thus masking its emergence when word frequency was not taken into account. We propose that target word frequency has its inXuence via an accessibility-related mechanism having to do with diVerences in observed changes in aVect between prime and target.

Statistical N-gram language modeling is used in many domains like spelling and syntactic verification, speech recognition, machine translation, character recognition and like others. This paper describes a system for sentence structure verification based on Ngram modeling of Bangla. An experimental corpus containing one million word tokens was used to train the system. The corpus was a part of the BdNC01 corpus, created in the SIPL lab. of Islamic university. Collecting several sample text from different newspapers, the system was tested by 1000 correct and another 1000 incorrect sentences. The system has successfully identified the structural validity of test sentences at a rate of 93%. This paper also describes the limitations of our system with possible solutions.

-Muitos alunos apresentam dificuldades na compreensão e desenvolvimento de algoritmos. Para tentar ajudar esses alunos foi criado o ambiente SICAS, um ambiente de trabalho individual baseado na animação e simulação de algoritmos. Neste artigo iremos apresentar o SICAS-COL, um sistema colaborativo destinado ao auxílio da aprendizagem da programação. O SICAS-COL é uma evolução deste ambiente que procura aliar a animação e simulação de algoritmos com actividades colaborativas de aprendizagem. Estas permitem que um grupo de alunos colabore na resolução de um problema através de actividades de desenvolvimento, comentário e argumentação por todos os elementos do grupo. Palavras Chave -aprendizagem da programação, animação de algoritmos, CSCL.

Ergonomik memainkan peranan yang penting dalam rekabentuk kerusi sekolah. Postur duduk yang janggal akibat daripada rekabentuk kerusi yang tidak sesuai mampu menyumbang ke arah kesan yang negatif kepada kesihatan kanak-kanak. Isu ini dapat dilihat melalui penyelidikan yang mempersoalkan tentang kesesuaian ukuran kerusi yang terdapat di sekolah rendah yang digunakan oleh Murid Sekolah Rendah Tahap Satu (MSRTS). Seramai 108 orang responden telah terlibat dalam penyelidikan ini yang terdiri daripada MSRTS yang berumur 7, 8 dan 9 tahun dari 12 buah Sekolah Rendah Kebangsaan di Perlis, Kedah dan Pulau Pinang. Penyelidikan ini menggabungkan kaedah kuantitatif dan kualitatif dalam mengenalpasti ketidak-sepadanan antropometri responden dengan ukuran kerusi yang digunakan di sekolah. Antropometer, goniometer, penimbang berat badan dan pita ukur digunakan bagi memperolehi data antropometri responden dan ukuran kerusi dari setiap sekolah. Data antropometri dan ukuran kerusi dianalisa menggunakan perisian SPSS (Statistical Package for Social Science). Postur duduk responden ketika sesi pembelajaran di dalam kelas turut dirakam dan dianalisa bagi menyokong data SPSS yang diperolehi. Dapatan kajian mencatatkan data antropometri MSRTS tidak sepadan dengan ukuran kerusi yang digunakan di sekolah. Oleh itu, pertimbangan terhadap faktor ergonomik dalam rekabentuk kerusi sekolah adalah perlu bagi memastikan kesihatan dan keselesaan MSRTS di sekolah. Kata Kunci: Ergonomik, Murid Sekolah Rendah Tahap Satu (MSRTS), Antropometri, Kerusi sekolah PENGENALAN "Ergonomics (or human factors) is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance".

From classic theory and research in psychology, we distill a broad theoretical statement that evaluative responding can be immediate, unintentional, implicit, stimulus based, and linked directly to approach and avoidance motives. This statement suggests that evaluative responses should be elicited by novel, nonrepresentational stimuli (e.g., abstract art, “foreign” words). We tested this hypothesis through combining the best features of relevant automatic-affect research paradigms. We first obtained explicit evaluative ratings of novel stimuli. From these, we selected normatively positive and negative stimuli to use as primes in a sequential priming paradigm. Two experiments using this paradigm demonstrated that briefly presented novel prime stimuli were evaluated automatically, as they facilitated responses to subsequently presented target stimuli of the same valence just as much as did pictures or names of real objects. A final experiment revealed that exposure to novel stimuli pr...

Some previous studies (e.g. that carried out by Van Bruggen et al. in 2004) have pointed to a need for additional research in order to firmly establish the usefulness of LSA (latent semantic analysis) parameters for automatic evaluation of academic essays. The extreme variability in approaches to this technique makes it difficult to identify the most efficient parameters and the optimum combination. With this goal in mind, we conducted a high spectrum study to investigate the efficiency of some of the major LSA parameters in small-scale corpora. We used two specific domain corpora that differed in the structure of the text (one containing only technical terms and the other with more tangential information). Using these corpora we tested different semantic spaces, formed by applying different parameters and different methods of comparing the texts. Parameters varied included weighting functions (Log-IDF or Log-Entropy), dimensionality reduction (truncating the matrices after SVD to a set percentage of dimensions), methods of forming pseudo-documents (vector sum and folding-in) and measures of similarity (cosine or Euclidean distances). We also included two groups of essays to be graded, one written by experts and other by non-experts. Both groups were evaluated by three human graders and also by LSA. We extracted the correlations of each LSA condition with human graders, and conducted an ANOVA to analyse which parameter combination correlates best. Results suggest that distances are more efficient in academic essay evaluation than cosines. We found no clear evidence that the classical LSA protocol works systematically better than some simpler version (the classical protocol achieves the best performance only for some combinations of parameters in a few cases), and found that the benefits of reducing dimensionality arise only when the essays are introduced into semantic spaces using the folding-in method.

We investigate the use of web search queries for detecting errors in non-native writing. Distinguishing a correct sequence of words from a sequence with a learner error is a baseline task that any error detection and correction system needs to address. Using a large corpus of error-annotated learner data, we investigate whether web search result counts can be used to

In this paper we present a system for automatic correction of errors made by learners of English. The system has two novel aspects. First, machine-learned classifiers trained on large amounts of native data and a very large language model are combined to optimize the precision of suggested corrections. Second, the user can access real-life web examples of both their original formulation and the suggested correction. We discuss technical details of the system, including the choice of classifier, feature sets, and language model. We also present results from an evaluation of the system on a set of corpora. We perform an automatic evaluation on native English data and a detailed manual analysis of performance on three corpora of nonnative writing: the Chinese Learners' of English Corpus (CLEC) and two corpora of web and email writing.

Some previous studies (e.g. that carried out by Van Bruggen et al. in 2004) have pointed to a need for additional research in order to firmly establish the usefulness of LSA (latent semantic analysis) parameters for automatic evaluation of academic essays. The extreme variability in approaches to this technique makes it difficult to identify the most efficient parameters and the optimum combination. With this goal in mind, we conducted a high spectrum study to investigate the efficiency of some of the major LSA parameters in small-scale corpora. We used two specific domain corpora that differed in the structure of the text (one containing only technical terms and the other with more tangential information). Using these corpora we tested different semantic spaces, formed by applying different parameters and different methods of comparing the texts. Parameters varied included weighting functions (Log-IDF or Log-Entropy), dimensionality reduction (truncating the matrices after SVD to a set percentage of dimensions), methods of forming pseudo-documents (vector sum and folding-in) and measures of similarity (cosine or Euclidean distances). We also included two groups of essays to be graded, one written by experts and other by non-experts. Both groups were evaluated by three human graders and also by LSA. We extracted the correlations of each LSA condition with human graders, and conducted an ANOVA to analyse which parameter combination correlates best. Results suggest that distances are more efficient in academic essay evaluation than cosines. We found no clear evidence that the classical LSA protocol works systematically better than some simpler version (the classical protocol achieves the best performance only for some combinations of parameters in a few cases), and found that the benefits of reducing dimensionality arise only when the essays are introduced into semantic spaces using the folding-in method.

In this paper we describe the METIS-II system and its evaluation on each of the language pairs: Dutch, German, Greek, and Spanish to English. The METIS-II system envisaged developing a data-driven approach in which no parallel corpus is required and in which no full parser or extensive rule sets are needed. We describe the evaluation on a development test set and on a test set taken from Europarl, and compare our results with SYSTRAN. We also provide some further analysis, namely researching the impact of the number and source of the reference translations and analysing the results according to test text type. The results are expectably lower for the METIS system, but not at an unattainable distance from a mature system like SYSTRAN.

In this paper we discuss our participation to the 2013 Semeval Semantic Textual Similarity task. Our core features include (i) a set of metrics borrowed from automatic machine translation, originally intended to evaluate automatic against reference translations and (ii) an instance of explicit semantic analysis, built upon opening paragraphs of Wikipedia 2010 articles. Our similarity estimator relies on a support vector regressor with RBF kernel. Our best approach required 13 machine translation metrics + explicit semantic analysis and ranked 65 in the competition. Our postcompetition analysis shows that the features have a good expression level, but overfitting and —mainly— normalization issues caused our correlation values to decrease.

Breasts are composed of a mixture of fibrous and glandular tissue as well as adipose tissue and breast density describes the prevalence of fibroglandular tissue as it appears on a mammogram. Over the past few years, evaluation and reporting of breast density as it appears on mammograms has received a lot of attention because it impacts one's risk of developing breast cancer but also the capability of detecting breast cancer on mammograms. In addition, mammography fails in the identification of breast cancer in almost half of the women with dense breasts. Different image analysis methods have been investigated for automatic breast density classification. The presented method investigates the use of Amplitude-Modulation Frequency-Modulation (AM-FM) multi-scale feature sets for characterization of breast density as the first step in the development of a density specific Computer Aided Detection System. AM-FM decompositions use different scales and bandpass filters to extract the instantaneous frequencies (IF), instantaneous amplitude (IA) and instantaneous phase (IP) components from an image. Normalized histograms of the maximum IA across all frequencies and scales are used to model the different breast density classes. Classification of a new mammogram into one of the breast density classes is achieved using the k-nearest neighbor method with k = 5 and the euclidean distance metric. The method is evaluated on the Medical Image Analysis Society (MIAS) mammographic database and the results are presented. The presented method allows breast density classification accuracy reaching over 84%. Future work will involve a new AM-FM methodology approach based on adaptive filterbank design and performance index decision.