Concept Map Mining: A definition and a framework for its evaluation

Essay writing and concept mapping are both learning activities that involve higher level thinking, moreover the latter has been used to support writing by presenting a different visualization of the essay, facilitating student’s reflection. However as concept mapping is a time consuming task, the immediacy of such feedback is impossible. Concept Map Mining (CMM) is the automatic extraction of concept maps from essays, which would allow immediate feedback on writing activities. CMM is currently work in progress in an advanced stage, and its evaluation requires a gold standard of concept maps extracted from essays by human annotators. This paper reports on the creation of such a gold standard, and analyzes patterns that will help understanding how humans summarize text using concept maps. Such patterns will inform the design of the CMM algorithms. This analysis shows that several interesting patterns arise when humans face the task of extracting concept maps from essays.

Concept maps are commonly used tools for organising and representing knowledge in order to assist meaningful learning. Although the process of constructing concept maps improves learners' cognitive structures, novice students typically need substantial assistance from experts. Alternatively, expertconstructed maps may be given to students, which increase the workload of academics. To overcome this issue, automated concept map extraction has been introduced. One of the key limitations is the lack of an evaluation framework to measure the quality of machine-extracted concept maps. At present, researchers in this area utilise human experts' judgement or expert-constructed maps as the gold standard to measure the relevancy of extracted knowledge components. However, in the educational context, particularly in course materials, the majority of knowledge presented is relevant to the learner, resulting in a large amount of information that has to be organised. Therefore, this paper introduces a machine-based approach which studies the relative importance of knowledge components and organises them hierarchically. We compare machine-extracted maps with human judgment, based on expert knowledge and perception. This paper describes three ranking models to organise domain concepts. The results show that the auto-generated map positively correlates with human judgment (r s~1 ) for well-structured courses with rich grammar (well-fitted contents).

KPI Science News, 2020

Background. The hypermedia environment, which has received a powerful technical infrastructure thanks to the WWW, has led to the study and emergence of new forms and tools of information and educational content presentation. Various methods and means of visualization of educational information increase clarity and facilitate the process of new knowledge perception. Such tools include a wide range of concept maps that have become widespread in many areas related to information technology and education. The challenge here is to find comprehensive solutions that will reduce computing and labor costs for the multifunctional use of concept maps in information and learning systems. Objective. The work is focused on the review of concept maps, research of preconditions for their origin, analysis of their types and purposes of application in the educational process and learning systems. The main task is to analyze the requirements for interactive concept maps in ontology-oriented information systems to support lifelong learning. Methods. The use of concept maps in educational systems is based on knowledge modeling, graph theory, ontological modeling of educational content and methods of adaptive e-learning systems. The paper reviews and analyzes the application of concept maps and requirements for them within the information and learning systems. Results. Concept maps have significant foundation in cognitive and educational psychology, computational linguistics, knowledge engineering and have widespread application in computer supported learning. Concept maps are used as a means of meaningful learning when constructed by students, used to assess knowledge, as well as a means of navigating information resources. Concept maps in educational systems are used to present knowledge to users and serve as interfaces in the process of exploratory search and review of study areas. Information systems provide enrichment of concept maps by means of interactivity, which allows diversifying and expanding its use. Conclusions. The key requirements for interactive concept maps in information and learning systems are formed, including automation of interactive concept map construction, availability of wide navigation capabilities together with cognitive load control, implementation of interdisciplinary links, media annotation of map elements, complex application of graph interfaces for different parts of the system and adaptation to mobile devices.

Proceedings of the Int …, 2006

, an adaptive multilingual free-text Computer-Assisted Assessment system, automatically evaluates students' free-text answers given a set of correct ones. This paper presents an extension of the system in order to generate the students' concept maps while they are being assessed. To that aim, a new module for the automatic identification of the terms of a particular knowledge field has been created. It identifies and keeps track of the terms that are being used in the students' answers, and calculates a confidence score of the student's knowledge about each term. An empyrical evaluation using the students' real answers show that it is robust enough to generate a good set of terms from a very small set of answers.

2012

Concept map is a graphical representation of its creator's knowledge structure, and it can be used as a tool for knowledge assessment. Even when considering only three factors-whether the structure, linking phrases and concept labels are given-there is a wide range of possible concept map-based tasks. Tasks with different demands allow assessing different aspects of knowledge, and thus, various sets of criteria are used for their assessment. Scoring of some of these criteria is easy to automate (e.g. count of concepts or propositions), but also more elaborate criteria are used that are more difficult to assess automatically (e.g. proposition's depth of explanation). This paper represents the results of a literature study on usage of concept map-based tasks for knowledge assessment purposes and criteria used to score them.

2011

This paper discusses a knowledge-based model for the design and development of units of learning and teaching aids. The idea behind this model originates from both the analysis of the open issues in instructional authoring systems, and the lack of a well-defined process able to merge pedagogical strategies with systems for the knowledge organization of the domain. In particular, it is presented the Educational Concept Map (ECM): a, pedagogically founded (derived from instructional design theories), abstract annotation system that was developed with the aim of guaranteeing the reusability of both teaching materials and knowledge structures. By means of ECMs, it is possible to design lessons and/or learning paths from an ontological structure characterized by the integration of hierarchical and associative relationships among the educational objectives. The paper also discusses how the ECMs can be implemented by means of the ISO/IEC 13250 Topic Maps standard. Based on the same model, it is also considered the possibility of visualizing, through a graphical model, and navigate, through an ontological browser, the knowledge structure and the relevant resources associated to them.

Unlike essays, concept maps provide a visual and holistic way to describe declarative knowledge relationships, often providing a clear measure of student understanding and most strikingly, highlighting student misconceptions. This poster session presents a computer-based approach that uses concept-map like Pathfinder network representations (PFNets; to make visual students' written text summaries of biological content. A software utility called ALA-Reader (www.personal.psu.edu/rbc4/) was used to translate students' written text summaries of the heart and circulatory system into raw proximity data, and then Pathfinder PCKNOT software (Schvaneveldt, 1990) was used to convert the proximity data into visual PFNets. The validity of the resulting PFNets as adequate representations of the students' written text was considered by simply asking the students and also by comparing the correlation of human rater scores to the PFNet agreement-with-an-expert scores, (PFNet text score Pearson r = 0.69, ranked 5 th out of 12). The concept-map like PFNet representations of texts provided students (and their instructor) with another way of thinking about their written text, especially by highlighting correct, incorrect, and missing propositions in their text. This paper provides an overview of the approach and the pilot experimental results. The actual poster session will in addition demonstration the free ALA-Reader software and will also how to procure and use PCKNOT software.

Education Sciences, 2019

Concept maps have been shown to have a positive impact on the quality of student learning in a variety of disciplinary contexts and educational levels from primary school to university by helping students to connect ideas and develop a productive knowledge structure to support future learning. However, the evaluation of concept maps has always been a contentious issue. Some authors focus on the quantitative assessment of maps, while others prefer a more descriptive determination of map quality. To our knowledge, no previous consideration of concept maps has evaluated the different types of knowledge (e.g., procedural and conceptual) embedded within a concept map, or the ways in which they may interact. In this paper we consider maps using the lens provided by the Legitimation Code Theory (LCT) to analyze concept maps in terms of semantic gravity and semantic density. Weaving between these qualitatively, different knowledges are considered necessary to achieve professional knowledge or expert understanding. Exemplar maps are used as illustrations of the way in which students may navigate their learning towards expertise and how this is manifested in their concept maps. Implications for curriculum design and teaching evaluation are included.

Writing assignments are ubiquitous in higher education. Writing develops not only communication skills, but also higher-level cognitive processes that facilitate deep learning. Cognitive visualizations, such as concept maps, can also be used as part of learning activities including as a form of scaffolding, or to trigger reflection by making conceptual understanding visible at different stages of the learning process. We present Concept Map Miner (CMM), a tool that automatically generates Concept Maps from students’ compositions, and discuss its design and implementation, its integration to a writing support environment and its evaluation on a manually annotated corpora of university essays (N=43). Results show that complete CM, with concepts and labeled relationships, are possible and its precision depends the level of summarization (number of concepts) chosen.

Writing assignments are ubiquitous in higher education. Writing develops not only communication skills, but also higher-level cognitive processes that facilitate deep learning. Cognitive visualizations, such as concept maps, can also be used as part of learning activities including as a form of scaffolding, or to trigger reflection by making conceptual understanding visible at different stages of the learning process. We present Concept Map Miner (CMM), a tool that automatically generates Concept Maps from students' compositions, and discuss its design and implementation, its integration to a writing support environment and its evaluation on a manually annotated corpora of university essays (N=43). Results show that complete CM, with concepts and labeled relationships, are possible and its precision depends the level of summarization (number of concepts) chosen.