Academia.eduAcademia.edu

Figure 5 – uploaded by ghader kurdi

See full PDFdownloadDownload figure
Table 5 Agreement between pairs of reviewers on questions appropriateness The following tables provides information about agreement between domain experts. Kappa values were interpreted according to Viera and Garrett’s guideline (Viera et al. 2005).

Table 5 Agreement between pairs of reviewers on questions appropriateness The following tables provides information about agreement between domain experts. Kappa values were interpreted according to Viera and Garrett’s guideline (Viera et al. 2005).

Related Figures (12)

Fig. 1 A small extraction from EMM€eT, illustrating the use of concepts and their relations which include their rankings and other associated data, such as sex and age
Fig. 1 A small extraction from EMM€eT, illustrating the use of concepts and their relations which include their rankings and other associated data, such as sex and age
Table 1 A description of the automated translation process from EMMeT-SKOS to EMMeT-OWL semrel = Semantic Relation, (aw) : Rank = a logical OWL axiom a annotated with a Rank (achievable in OWL 2), Cl = Class, s: = skos: and M = Match
Table 1 A description of the automated translation process from EMMeT-SKOS to EMMeT-OWL semrel = Semantic Relation, (aw) : Rank = a logical OWL axiom a annotated with a Rank (achievable in OWL 2), Cl = Class, s: = skos: and M = Match
The more information in the ontology, the more questions can be mapped to the emplate. Clip IAale With regards to medical question templates, experts from Elsevier identified four question templates that were representative of the type of questions used within their publications designed to help medical residents prepare for their board examinations. These publications, and therefore the questions used as a basis for the templates, were created by Elsevier authors who are practising medical doctors and/or professors of medicine and leading experts in their speciality area. All authors are acutely aware of the types of questions used on Board examinations. where all appropriate classes are disjoint, the question:
The more information in the ontology, the more questions can be mapped to the emplate. Clip IAale With regards to medical question templates, experts from Elsevier identified four question templates that were representative of the type of questions used within their publications designed to help medical residents prepare for their board examinations. These publications, and therefore the questions used as a basis for the templates, were created by Elsevier authors who are practising medical doctors and/or professors of medicine and leading experts in their speciality area. All authors are acutely aware of the types of questions used on Board examinations. where all appropriate classes are disjoint, the question:
Fig.2 The structure of the What is the most likely diagnosis template, using two symptoms and one history as stem entities
Fig.2 The structure of the What is the most likely diagnosis template, using two symptoms and one history as stem entities
Fig. 3 A model based on axioms from EMMeT-OWL showing the What is the most likely diagnosis question. Note that the greater the rank, the stronger the relation is. A hCF of rank 10 indicates a most common clinical finding while a hCF of rank 7 indicates a rare clinical finding
Fig. 3 A model based on axioms from EMMeT-OWL showing the What is the most likely diagnosis question. Note that the greater the rank, the stronger the relation is. A hCF of rank 10 indicates a most common clinical finding while a hCF of rank 7 indicates a rare clinical finding
International Journal of Artificial Intelligence in Education Fig.4 A modular system diagram showing each major module in EMCQG and their position in the entire system
International Journal of Artificial Intelligence in Education Fig.4 A modular system diagram showing each major module in EMCQG and their position in the entire system
Table 2. Number of questions per generated template T: Template, hCF: hasClinicalFinding, hRF: has RiskFactor, hDDx: hasDifferentialDiagnosis, hD: hasDrug, NR: Not relevant for the template
Table 2. Number of questions per generated template T: Template, hCF: hasClinicalFinding, hRF: has RiskFactor, hDDx: hasDifferentialDiagnosis, hD: hasDrug, NR: Not relevant for the template
Fig.5 Results of the evaluation of question appropriateness. Raw numbers are presented between paren- theses. Appr.|Appr. = appropriate by two reviewers; Appr. and Appr.|Inappr. = appropriate by one reviewer; Inappr. = inappropriate by one reviewer; Inappr.|Inaprp. = inappropriate by two reviewers
Fig.5 Results of the evaluation of question appropriateness. Raw numbers are presented between paren- theses. Appr.|Appr. = appropriate by two reviewers; Appr. and Appr.|Inappr. = appropriate by one reviewer; Inappr. = inappropriate by one reviewer; Inappr.|Inaprp. = inappropriate by two reviewers
Fig.6 Performance of different templates generated by the system. Template | = “What is the most likely diagnosis?”, Template 2 = “What is the drug of choice?”, Template 3 = “What is the most likely clinical finding?”, Template 4 = “What is the differential diagnosis?” We were also interested to know whether the quality of multi-term questions was related to the number of stem entities or not, for example, whether diagnos- tic questions with one history and one symptom are as useful as questions with more history and symptom elements. Although we suspected questions with one his- tory and one symptom to be vague compared to questions with a higher number of histories and symptoms, this was not the case. We broke down the number of appropriate/inappropriate questions by stem size but could not find any association between the two variables. pe
Fig.6 Performance of different templates generated by the system. Template | = “What is the most likely diagnosis?”, Template 2 = “What is the drug of choice?”, Template 3 = “What is the most likely clinical finding?”, Template 4 = “What is the differential diagnosis?” We were also interested to know whether the quality of multi-term questions was related to the number of stem entities or not, for example, whether diagnos- tic questions with one history and one symptom are as useful as questions with more history and symptom elements. Although we suspected questions with one his- tory and one symptom to be vague compared to questions with a higher number of histories and symptoms, this was not the case. We broke down the number of appropriate/inappropriate questions by stem size but could not find any association between the two variables. pe
Table 3 Statistics about correctness of answers given by domain experts The questions with the highest percentage of correct responses!? were the What s the differential diagnosis questions and the What is the most likely diagnosis ques- ions (84.6% and 83.2% respectively). This again highlights that multi-term questions ire sensible. Surprisingly, the percentage of correct responses to the What is the most ikely clinical finding questions was relatively low (51.5%). A possible interpretation s that these questions consist of one stem entity and therefore the number of hints hey provide is limited compared to questions with multiple stem entities.
Table 3 Statistics about correctness of answers given by domain experts The questions with the highest percentage of correct responses!? were the What s the differential diagnosis questions and the What is the most likely diagnosis ques- ions (84.6% and 83.2% respectively). This again highlights that multi-term questions ire sensible. Surprisingly, the percentage of correct responses to the What is the most ikely clinical finding questions was relatively low (51.5%). A possible interpretation s that these questions consist of one stem entity and therefore the number of hints hey provide is limited compared to questions with multiple stem entities.
Fig.7 Results of evaluating question distractors. Raw numbers are presented in parentheses
Fig.7 Results of evaluating question distractors. Raw numbers are presented in parentheses
Table6 Agreement between pairs of reviewers on distractors appropriateness
Table6 Agreement between pairs of reviewers on distractors appropriateness
Related topics:
Computer ScienceEducationArtificial Intelligence in Education
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved