Affect Detection Research Papers

Approximate natural language understanding for an intelligent tutor

2025, Proc. of the 12th Int'l …

Intelligent tutoring systems (ITS's) have a rich history of helping students in certain scientific domains, like geometry, chemistry, and programming. These domains are ideal for ITS's, because they can be easily represented and because... more

descriptionView Paper arrow_downwardDownload

How are feelings of difficulty and familiarity linked to learning behaviors and gains in a complex science learning task?

by Jaclyn Ocumpaugh

2025, European Journal of Psychology of Education

The feelings of difficulty and familiarity (FOD and FOF) are two types of metacognitive experiences. Both may influence student engagement and the application of metacognitive strategies, but these relationships are not well understood,... more

descriptionView Paper arrow_downwardDownload

Hart

by Jaclyn Ocumpaugh

2025, Proceedings of the 33rd Annual International Conference on the Design of Communication

This paper evaluates the Human Affect Recording Tool (HART), a Computer Assisted Direct Observation (CADO) application that facilitates scientific sampling. HART enforces an established method for systematic direct observation in... more

descriptionView Paper arrow_downwardDownload

AutoTutor detects and responds to learners affective and cognitive states

by Barry Kort

2025, Workshop on Emotional and Cognitive Issues at the International Conference on Intelligent Tutoring Systems

Abstract. This paper provides a synthesis of our research towards the development of an affect-sensitive Intelligent Tutoring System called AutoTutor. The affect-sensitive AutoTutor detects the emotions (boredom, flow/engagement,... more

descriptionView Paper arrow_downwardDownload

An evaluation of decision-theoretic tutorial action selection

by R. Charles Murray

2025

novel decision-theoretic architecture for intelligent tutoring systems, DT Tutor (DT), was fleshed out into a complete ITS and evaluated. DT uses a dynamic decision network to probabilistically look ahead to anticipate how its tutorial... more

novel decision-theoretic architecture for intelligent tutoring systems, DT Tutor (DT), was fleshed out into a complete ITS and evaluated. DT uses a dynamic decision network to probabilistically look ahead to anticipate how its tutorial actions will influence the student and other aspects of the tutorial state. It weighs its preferences regarding multiple competing objectives by the probabilities that they will occur and then selects the tutorial action with maximum expected utility. The evaluation was conducted in two phases. First, logs were recorded from interactions of students with a Random Tutor (RT) that was identical to DT except that it selected randomly from relevant tutorial actions. The logs were used to learn many of DT's key probabilities for its model of the tutorial state. Second, the logs were replayed to record the actions that DT and a Fixed-Policy Tutor (FT) would select for a large sample of scenarios. FT was identical to DT except that it selected tutorial actions by emulating the fixed policies of Cognitive Tutors, which are theoretically based, widely used, and highly effective. The possible action selections for each scenario were rated by a panel of judges who were skilled human tutors. The main hypotheses tested were that DT's action selections would be rated higher than FT's and higher than RT's. This was the first comparison of a decision-theoretic tutor with a non-trivial competitor. DT was rated higher than FT overall and for all subsets of scenarios except help requests, for which it was rated equally. DT was also rated much higher than RT. The judges preferred that the tutors provide proactive help and the study design permitted this information to be put to use right away to develop and evaluate enhanced versions of DT and FT. The enhanced versions of DT and FT were rated about equally and higher than non-enhanced DT except on help requests. The variability of the actions selected by both non-enhanced and enhanced versions of DT demonstrated more sensitivity to the tutorial state than the actions selected by non-enhanced and enhanced versions of FT.

descriptionView Paper arrow_downwardDownload

Exploring Personal Memories and Video Content as Context for Facial Behavior in Predictions of Video-Induced Emotions

by Mark Neerincx

2025

Empirical evidence suggests that the emotional meaning of facial behavior in isolation is often ambiguous in real-world conditions. While humans complement interpretations of others' faces with additional reasoning about context,... more

descriptionView Paper arrow_downwardDownload

A Blast From the Past: Personalizing Predictions of Video-Induced Emotions using Personal Memories as Context

by Mark Neerincx

2025, arXiv (Cornell University)

A key challenge in the accurate prediction of viewers' emotional responses to video stimuli in real-world applications is accounting for person-and situation-specific variation. An important contextual influence shaping individuals'... more

descriptionView Paper arrow_downwardDownload

Studying the Interactions between Components of Self Regulated Learning in Open Ended Learning Environments

by Jaclyn Ocumpaugh

2024

This paper investigates the interactions between learners’ cognitive strategies and affective states; both important components of self-regulated learning (SRL) processes that influence student learning. We study cognitive-affective... more

descriptionView Paper arrow_downwardDownload

Field Observations of Engagement in Reasoning Mind

by Jaclyn Ocumpaugh

2024, Lecture Notes in Computer Science

This study presents Quantitative Field Observations (QFOs) of educationally relevant affect and behavior among students at three schools using Reasoning Mind, a game-based software system designed to teach elementary-level mathematics.... more

descriptionView Paper arrow_downwardDownload

Predicting STEM and Non-STEM College Major Enrollment from Middle School Interaction with Mathematics Educational Software

by Jaclyn Ocumpaugh

2024

The worldwide increase in demand for qualified workers in science, technology, engineering, and mathematics (STEM) fields has resulted in a greater focus on preparing students to enroll in postsecondary STEM programs. The processes that... more

descriptionView Paper arrow_downwardDownload

‒I Feel Your Pain: A Selective Review of Affect-Sensitive Instructional Strategies

by Jaclyn Ocumpaugh

2024

that the reader is familiar with some basics of affect science, affective computing, and ITSs, so this section is relatively brief. Although some of the claims made below are generally accepted, others are still controversial and are... more

descriptionView Paper arrow_downwardDownload

Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra

by Jaclyn Ocumpaugh and

2024

In recent years, the usefulness of affect detection for educational software has become clear. Accurate detection of student affect can support a wide range of interventions with the potential to improve student affect, increase... more

descriptionView Paper arrow_downwardDownload

Affect detection and metaphor in e-drama

by John Barnden

2024, International Journal of Continuing Engineering Education and Life-Long Learning

We report work on adding affect-detection to an existing e-drama programme, a text-based software system for (human) dramatic improvisation in simple virtual scenarios, for use primarily in learning contexts. The system allows a human... more

descriptionView Paper arrow_downwardDownload

Artificial Intelligence in STEM Education: Current Developments and Future Considerations

by leo deng

2024

Artificial Intelligence in Education (AIEd) is an established interdisciplinary field of research with focus on deploying artificial intelligence (AI) algorithms to transform education. There has been a growing interest in harnessing the... more

Figure 1. Number of publications related to AIEd under three theoretical paradigms — \lthough these ducation more lirected instruc liscusses the t ipplications of ~ Ed in STI hree paradigms of AI] ~ — paradigmatic shifts are general educational frameworks, they can also be applied to STEM generally. This process involves gradual reshaping of STEM education from the teacher- ion mode to the student-centered learning mode (see Figure 2). The following section EM education in more detail. We also describe the he three paradigms in STEM education, the current AI applications for STEM education Inder thece three naradiome and related decion and recearch

Figure 2. The shifts of three paradigms of AIEd in STEM education under these three paradigms, and related design and research.

descriptionView Paper arrow_downwardDownload

Approximate natural language understanding for an intelligent tutor

by Peter Hastings

2024, Proc. of the 12th Int'l …

Intelligent tutoring systems (ITS's) have a rich history of helping students in certain scientific domains, like geometry, chemistry, and programming. These domains are ideal for ITS's, because they can be easily represented and because... more

descriptionView Paper arrow_downwardDownload

Developmental Differences in Self-Regulated Learning and Question Asking During Learning with Hypermedia

by Jeremiah Sullins

2024, Proceedings of the Annual Meeting of the Cognitive Science Society

descriptionView Paper arrow_downwardDownload

Simulating smooth tutorial dialogue with pedagogical value

by Peter Hastings

2024, Proceedings of the 11th International Florida Artificial Intelligence Research Symposium Conference

This paper describes the design of AutoTutor, a fully automated computer tutor that simulates dialogue moves of normal human tutors and that will eventually incorporate sophisticated tutoring strategies. Autolhtot follows a curriculum... more

descriptionView Paper arrow_downwardDownload

Affect-driven Learning Outcomes Prediction in Intelligent Tutoring Systems

by Beverly Woolf

2024, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)

Equipping an Intelligent Tutoring System (ITS) with the ability to interpret affective signals from students could potentially improve the learning experience of students by enabling the tutor to monitor the students' progress and provide... more

descriptionView Paper arrow_downwardDownload

Addressing Student Behavior and Affect with Empathy and Growth Mindset

by Beverly Woolf

2024

We present results of a randomized controlled study that compared different types of affective messages delivered by pedagogical agents. We used animated characters that were empathic and emphasized the malleability of intelligence and... more

descriptionView Paper arrow_downwardDownload

Affect-driven Learning Outcomes Prediction in Intelligent Tutoring Systems

by Ivon Arroyo

2024, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)

Equipping an Intelligent Tutoring System (ITS) with the ability to interpret affective signals from students could potentially improve the learning experience of students by enabling the tutor to monitor the students' progress and provide... more

descriptionView Paper arrow_downwardDownload

Automatic detection of uncertainty in spontaneous German dialogue

by Barbara Schuppler

2024, Interspeech 2015

Uncertainty is ubiquitous in natural human communication. Human listeners assess the speaker's degree of uncertainty at any time in communication and use this information to shape dialogue. In contrast, currently available computer... more

Table 3: Confusion table for the best model (64 features) show- ing raw accuracies.

Table 4: CNAs for acoustic, linguistic and all features, respec- tively. Remarkably, the class-wise performance differs consider- ably between acoustic and non-acoustic feature classes. The acoustic feature classes tend to perform particularly well on the least frequent class uncertain. Non-acoustic feature classes tend to perform about as well as acoustic feature classes on labels certain and rather certain (see Table 4).

Table 5: Summary of methods and results as reported in various publications. Reported results are always best results in F, if available and accuracy otherwise.

descriptionView Paper arrow_downwardDownload

The Future Belongs to the Curious: Towards Automatic Understanding and Recognition of Curiosity in Children

by Charlie Hughes

2024, 5th Workshop on Child Computer Interaction (WOCCI 2016)

Curiosity plays a crucial role in learning and education of children. Given its complex nature, it is extremely challenging to automatically understand and recognize it. In this paper, we discuss the contexts under which curiosity can be... more

Figure 1: /lustration of the five types of tasks performed by children in EmoReact dataset. The experimenters are behind the scenes and the children talk to them while their reactions are being recorded.

Figure 2: Example frames of videos in EmoReact showing the co-occurrence of fourteen states with curiosity. Note that people in all of these videos have been labeled as curious while they were expressing one or more emotions during the same video clip.

Figure 3: Co-.occurrence of curiosity with different affective states in 385 videos labeled as curiosity. This Figure shows the number of videos in which curiosity has co-occurred with each affective state which are expressed by females and males and the overall co- occurrence percentage of each emotion label with curiosity . Note that attentiveness, exploration, happiness and uncertainty are the top four co-occurring states that are co-occurring with curiosity.

Figure 4: Valence states of curiosity. This figure shows the bias towards positive valence in curiosity. Note that this analysis was done on 385 videos which were labeled as curious.

descriptionView Paper arrow_downwardDownload

A Systematic Literature Review of Intelligent Tutoring Systems With Dialogue in Natural Language

by Jaime Ramírez

2024, IEEE Access

Intelligent tutoring systems (ITSs) are computer programs that provide instruction adapted to the needs of individual students. Dialog systems are computer programs that communicate with human users by using natural language. This paper... more

descriptionView Paper arrow_downwardDownload

Student Emotions in Conversation-Based Assessments

by Diego Zapata-Rivera

2024, IEEE Transactions on Learning Technologies

Students can experience a variety of emotions while completing assessments. Some emotions can get in the way of students performing their best (e.g., anxiety, frustration), whereas other emotions can facilitate student performance (e.g.,... more

descriptionView Paper arrow_downwardDownload

Exploiting emotions to disambiguate dialogue acts

by Elisabeth Andre

2023, Proceedings of the 9th international conference on Intelligent user interfaces

This paper describes an attempt to reveal the user's intention from dialogue acts, thereby improving the effectiveness of natural interfaces to pedagogical agents. It focuses on cases where the intention is unclear from the dialogue... more

descriptionView Paper arrow_downwardDownload

Confusion can be beneficial for learning

by Blair Lehman

2023, Learning and Instruction

We tested key predictions of a theoretical model positing that confusion, which accompanies a state of cognitive disequilibrium that is triggered by contradictions, conflicts, anomalies, erroneous information, and other discrepant events,... more

Fig. 2. Condition x confusion interaction for performance on the posttest (Experiment 1). An alternate method to probe the interaction is to regress posttest performance on confusion independently for each condi- tion (see Fig. 2). This yielded significant models for the true—false,

Fig. 3. Condition x Trial 4 interaction for performance on far transfer test (Experiment 2).

Fig. 4. Examples of confused faces from participants in Experiments 1 and 2. It is difficult to identify whether any of the specific factors listed above contributed to the lack of sensitivity of the self-reports in the present study. What is clear, however, is that future research should consider alternate methods to investigate emotions in addition to self-reports. Some possibilities include online observations by external judges (Rodrigo & Baker, 2011), retrospective coding of There are three important limitations with this research that need to be addressed in subsequent research activities. First, critics might object to the manipulations on the grounds that intentionally confusing learners by providing misleading information and contradictions is not in the best interests of the learner. We acknowledge this and similar reactions to the manipulations, but

Fig. 5. Observed emotion transitions and their hypothesized causes. Image adapted from D’Mello and Graesser (2012a).

Sample case study and excerpt of trialogue.

Notes. Tr: true; Fl: false. Tr—Tr was the reference group for the models, hence, coefficients for this condition are not shown in the table. Bolded cells refer to significant effects at p <.05. Proportional occurrence of affective states from Experiment 1.

Proportion of forced-choice questions correctly answered (Experiment 1). Notes. Tr: true; Fl: false; Tr—Tr was the reference group for the models, hence, coefficients for this condition are not shown in the table. Bolded cells refer to significant effects at p< .05.

Performance on the multiple-choice posttest (Experiment 2). Notes. Tr: true; Fl: false; Tr—Tr was the reference group for the models, hence, coefficients for this condition are not shown in the table. Bolded cells refer to significant effects at p< .05. Table 6 Table 7

descriptionView Paper arrow_downwardDownload

A Time for Emoting: When Affect-Sensitivity Is and Isn’t Effective at Promoting Deep Learning

by Blair Lehman

2023, Springer eBooks

descriptionView Paper arrow_downwardDownload

Confusion and complex learning during interactions with computer learning environments

by Blair Lehman

2023, Internet and Higher Education

Folk wisdom holds that being confused is detrimental to learning. However, research on emotions and learning suggest a somewhat more complex relationship between confusion and learning outcomes. In fact, it has been proposed that impasses... more

descriptionView Paper arrow_downwardDownload

Computational affect detection for education and health

by David Cooper

2023

helped frame my perspective on computer science. I have benefited from many lunch conversations with fellow students as well as conversations in the labs that I have had a desk. I would especially like to thank Megan Olsen and Yariv Levy,... more

Figure 1.1. The Emotion Scale on the Left [50] and the Personal Interaction Coding Inventory on the right [15].

Figure 3.1. An example problem presented by the Wayang system. Jake is on the lower right corner. The Hint Toolbar is on the right.

Table 3.5. This table shows the results of the best classifier of each emotional response (confident, frustrated, excited, and interested). Accuracy of no classifier (last column) is a prediction that the emotional state is always low. Values in parentheses include the middle values in the testing set as negative examples.

Table 4.1. The mapping of tags to text in Fig. 4.1 above. Figure 4.1. An example of the Emotion query. Table 4.1 below has the values for each <> enclosed word, except for (<Name>), which is the name of the student.

The third chair feature is meant to determine if the student is sitting forward. From the three pressure values from the back of the chair, each having the range and the net change in pressure of the back: 0, 1023], we compute the Sit Forward feature as follows the most useful features from [38]. These are the net change in pressure of the seat, We compute three features from the 6 chair sensors. The first two are based on

Figure 3.2. A student at the client computer puts on a bracelet and starts the two client programs indicating the wrist ID of the bracelet. The bracelet sends Skin Conductance data to the Wrist Node, then logs bracelet data from all of the students in the classroom. The User Model System (UMS) receives the bracelet data through the Wrist Stream. The UMS client performs the same task as the Wrist Node for each of the other three sensor sources. The ITS logs student interactions, and sends Tutor Data to the UMS. The data is time synced based on the client’s system time. The UMS uses all available streams of data to make user predictions to improve the ITS Client interaction. Figure 3.2. A student at the client computer puts on a bracelet and starts the

Minimum Pause Length for Utterance Figure 5.1. The effect of pause length on extracting useful utterances for the clas- sification of gender. The red line with boxes expresses utterances extracted with no buffer, and the blue line with diamonds expresses utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.

significantly for 500 ms and above. Figure 5.2. The effect of pause length on extracting useful utterances for the two class classification of culture (American vs. Asian). The left shows the results for male subjects. The red line with boxes expresses male utterances extracted with no buffer, the blue line with diamonds expresses male utterances extracted with a 100ms silence buffer on each side. The right shows results for female subjects. The red line with down triangles expresses female utterances extracted with no buffer, the blue line with up triangles expresses female utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.

Figure 6.1. Left: The key points for Group A with one simulated patient. The key points in this room were 1) The monitor, 2) the patient, and 3) the telephone. In this frame, the student is closest to the monitor, but actually interacting with the patient. Right: The key points for Group B with two simulated patients. The key points in this room were 1) patient 1, 2) patient 2, and 3) the telephone. In this frame the student is closest to patient 1.

Figure 6.2. Two examples of the Reading People Tracker. Yellow rectangles repre- sent objects. Pink rectangles represent upper body. Red rectangles with a small inner rectangle represent heads. top: Multiple objects are found for one person, bottom: the foreground detection of the person begins to fade as the student stands very still. when the person stopped for an extended period of time, the track was lost. sent objects. Pink rectangles represent upper body. Red rectangles with a small inner a motion detection algorithm based on background estimation and subtraction, anc

Figure 6.3. Two examples of the CAMSHIFT tracker. The blue box is the center of the track, representing the head position. The red ellipse is the area being tracked. top: the tracker center stays near the student, but the size of the track is most of the frame, bottom: the tracker center goes to the wrong side of the patient.

of one of the students from our study. Figure 6.4. An example of the Lucas-Kanade tracker. The head region is selected by hand. As the student moves around the ‘good features to track’ (in green) disperse away from the head. from the head. Figure 6.4 shows an example of the Lucas-Kanade tracker on the head

Figure 6.5. Two examples of the distribution field tracker. Blue box represents the area being tracked, in this case, the head of the student. top: tracker incorrectly selects filing cabinet, bottom: tracker incorrectly selects a box with dials

Boxplot of distances for ALL students in ALL rooms. Num points: 2027 Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right: Box plot of distances from ground truth for top ten trackers. Trackers beginning with ‘n’ use only initialization, trackers beginning with ‘a’ are variants of the CAMSHIFT tracker, and trackers beginning with ‘i’ are the DF trackers. Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right

Figure 6.7. The predicted head positions of one student for two different trackers. Blue points predict closest to monitor. Green points predict closest to patient, and Orange points predict student closest to the telephone. Left: heuristic detection (no tracking). Right: distribution field tracker of switches to another bin are computed.

Table 3.2. The ten sensor features below are summarized by their mean, standard deviation, min and max values and then these 40 summarized features are selected by using regression models to predict confidence, frustration, excitement, and interest. This table defines the abbreviations for each feature. In addition to predicting frustration, our model is meant to predict excitement, interest, and confidence. The sensor features considered in our analysis are described

Table 3.4. This table lists the variables that the stepwise regression method selected as relevant, for each of the regression models in Table 3.3. Each of these features significantly contribute to the prediction of emotion self-reports (p < 0.01), and are listed in order of relevance (The feature at the top is the best predictor.) The abbreviations of these features are defined in Tables 3.1 and 3.2. Table 3.3. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using stepwise linear least squares regression, and variables entered into the model are shown in Table 3.4. The top row lists the feature sets that are available. The left column lists the emotional self-reports being predicted. R values correspond to the fit of the model (best fit models for each emotion are in bold). N values vary because some students are missing data for a sensor.

Table 4.2. Features used for each problem that includes an affective state label in order to train the emotion classifiers (features are shown in abbreviated form). The nine tutor features are shown on the left and the ten sensor features are on the right. Features used in a classifier that are significantly better than the baseline (p < 0.05) are in bold. are in bold.

Table 4.4. These are the results of the feature selection. The baseline classifier for each emotion is just a linear model trained on a constant. The classifier names are the concatenation of an abbreviated emotion (e.g., confident, interested, and excited) and the contributing sensor features. If there are no sensor features, then Tutor comes after the emotion, and when there is more than one classifier with the same feature set a letter is added to disambiguate the names (e.g. TutorA is only tutor data, and TutorM is only tutor data, but the stepwise selection process had the mouse data available, but no mouse features were selected). Names in bold are for classifiers that performed significantly better than the baseline for that emotion in at least one way.

Table 4.5. Classifier ranking using cross-validation data (p < 0.05) results are shown in Table 4.5. Given these results, our findings suggest that the tutor could generate interven-

Table 4.6. This shows validation results of all classifiers that performed better than the baseline classifier during training. All values are the mean value per student. Fall specifies the training set based on the leave-one-student-out cross-validation, and Spring specifies the results of the classifiers trained on the training set (Fall 2008 Data), and tested on the validation set (Spring 2009 Data). Values in bold are significantly better (p < 0.05) than the baseline. Table 4.7. Classifier ranking using validation data from the Spring of 2009. All differences indicated by ‘>’ are significant with p < 0.01.

Table 5.1. The ten acoustical features below are selected by using a linear least squares stepwise regression models to predict gender, American vs. Asian culture, or one of four cultures. This table lists each feature with an abbreviation and a definition. the Youoicea feature is between 20% and 80%, then it is considered natural speech.

Table 6.1. Cross-validated mean R? values for the five best trackers. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using a branch and bound linear regression. The top row lists the tracker. The left column lists the emotional self-reports being predicted and the group that reports are predicted for. R? values correspond to the fit of the model (best fit models for each emotion are in bold). Group A had one simulated patient and Group B had two simulated patients. Each group has 23 students reporting emotions. Table 6.2 shows results for the best accuracy classifiers. Note that the many of

Table 6.2. The ten best classifiers, one for each emotion for each group. Trackers starting with an i are DF trackers using variant 4. f indicates using variant 3. g indicates using variant 5. a indicates the CAMSHIFT tracker, and b indicates using variant lb. t indicates using variant la. s indicates using the faster variant of the DF tracker, and n indicates that just initialization is done.

descriptionView Paper arrow_downwardDownload

The Effect of Emotions on Self-Regulated-Learning (SRL) and Story Comprehension in Emerging Readers

by Earl Woodruff

2023, Frontiers in Education

In the early-elementary years, emerging readers experience many emotions due to the high value that is placed on the development of reading skills. If emotions are not regulated using effective learning strategies, they can negatively... more

FIGURE 2 | Standardized means of individual SRL strategies used. FIGURE 1 | Standardized means of individual SRL strategies used.

FIGURE 3 | Model with indirect mediated effects. Pathways are standardized estimates, standard errors in parentheses. Significant pathways are represented by a bold line, not significant pathways a dotted line. *p < 0.05, *“p < 0.001.

*n < 0.01, *p < 0.05, Girls = 0, Boys = 1, N = 150. TABLE 2 | Covariates, IV, mediator and dependent variable.

“p < 0.01, *p < 0.05, Girls = 0, Boys = 1, N = 150, R&R, reaction and reflection. TABLE 1 | Emotions, SRL strategies and storytelling scores.

TABLE 3 | Model path coefficients, standard errors, p-values, and Cls. Dependent variable: Storytelling score. Estimates are unstandardized. X, frustration; M, planning. R°y, x represents the R® value for the model without a mediator (c pathway); R°M, x represents the R° value for the relationship between the independent variable and mediator (a pathway); R° Y, MX represents the R? value for the total model including the mediator (c’ pathway). N = 150.

descriptionView Paper arrow_downwardDownload

Toward an Affect-Sensitive AutoTutor

by Rosalind Picard

2023, IEEE Intelligent Systems

This paper investigates the reliability of detecting a learner's affective states in an attempt to augment an Intelligent Tutoring System (AutoTutor) with the ability to incorporate such states into its pedagogical strategies to improve... more

descriptionView Paper arrow_downwardDownload

AutoTutor detects and responds to learners affective and cognitive states

by Rosalind Picard

2023, Workshop on Emotional and Cognitive Issues at the International Conference on Intelligent Tutoring Systems

Abstract. This paper provides a synthesis of our research towards the development of an affect-sensitive Intelligent Tutoring System called AutoTutor. The affect-sensitive AutoTutor detects the emotions (boredom, flow/engagement,... more

descriptionView Paper arrow_downwardDownload

Integrating affect sensors in an intelligent tutoring system

by Rosalind Picard

2023, Affective Interactions: The Computer in the Affective Loop Workshop at

This project augments an existing intelligent tutoring system (AutoTutor) that helps learners construct explanations by interacting with them in natural language and helping them use simulation environments. The research aims to develop... more

descriptionView Paper arrow_downwardDownload

To feel or not to feel: The role of affect in human–computer interaction

by Eva Hudlicka

2023, International journal of human-computer studies

The past decade has witnessed an unprecedented growth in user interface and humancomputer interaction (HCI) technologies and methods. The synergy of technological and methodological progress on the one hand, and changing user expectations... more

descriptionView Paper arrow_downwardDownload

Inducing and Tracking Confusion with Contradictions during Critical Thinking and Scientific Reasoning

by Blair Lehman

2023, Lecture Notes in Computer Science

Cognitive disequilibrium and its affiliated affective state of confusion have been found to be beneficial to learning due to the effortful cognitive activities that accompany their experience. Although confusion naturally occurs during... more

descriptionView Paper arrow_downwardDownload

Artificial Intelligence in Education

by Charles Fadel

2023, Artificial Intelligence in Education

From International Organizations "Artificial intelligence will be disruptive, but few people understand that education is going to be in the first frontline. This brilliantly reflective and forward-looking book helps the education... more

descriptionView Paper arrow_downwardDownload

Evaluating Gaming Detector Model Robustness Over Time

by Nidhi Nasiar

2023, Zenodo (CERN European Organization for Nuclear Research)

Research into "gaming the system" behavior in intelligent tutoring systems (ITS) has been around for almost two decades, and detection has been developed for many ITSs. Machine learning models can detect this behavior in both real-time... more

descriptionView Paper arrow_downwardDownload

Beyond the Short Answer Question with Research Methods Tutor

by Peter Hastings

2023, Lecture Notes in Computer Science

Research Methods Tutor is a new intelligent tutoring system created by porting the existing implementation of the AutoTutor system to a new domain, Research Methods in Behavioural Sciences, which allows more interactive dialogues. The... more

descriptionView Paper arrow_downwardDownload

Assessing User Bias in Affect Detection within Context-Based Spoken Dialog Systems

by Juan M Montero

2023, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing

This paper presents an empirical evidence of user bias within a laboratory-oriented evaluation of a Spoken Dialog System. Specifically, we addressed user bias in their satisfaction judgements. We question the reliability of this data for... more

descriptionView Paper arrow_downwardDownload

Supporting the review of student proposal drafts in information technologies

by Jair Lopez

2023, Proceedings of the 13th annual conference on Information technology education

In many cases, academic programs or courses conclude with a thesis or research proposal text, elaborated by students. The review of such texts is a heavy load, especially at initial stages of drafting. This paper proposes a model that... more

descriptionView Paper arrow_downwardDownload

Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

by Aurora Constantin

2023, Proceedings of the 22nd Annual ACM Interaction Design and Children Conference

The present study aims to investigate the relationship between emotions experienced during learning and metacognition in typically developing (TD) children and those with autism spectrum disorder (ASD). This will assist us in using... more

Figure 1: Our expected interaction between ITS and children’s metacognition workflow Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

Figure 2: Neural network’s attention on six basic emotions from the validation set of the ChildEFES. The chin raiser in anger expression in Fig.2a; the lip corner depressor and the lower lip depressor in disgust expression in Fig.2b; the brow lower and upper lid raiser in fear expression in Fig.2c; the cheek raiser and lip corner puller in happy expression in Fig.2d; the brow lower and lip corner depressor in sad expression in Fig.2e; the jaw drop in surprise expression in Fig.2f. Table 1: Recognition Rate (Accuracy) and F1 score of AD-Loss (our proposed) and three state-of-art loss functions on the test se’ (10% of the subjects) of PUZZLED. (CE: Cross Entropy; AD: AD-Loss (we proposed); AM: Additive Margin Softmax; AF: ArcFace

Figure 3: The example of different numbers of patches on facial emotion expressions: Origin image in (a) and different numbers of patches in (b) and (c). Figure 4: A play flow of ‘Meta-BrainHood’. 1: welcome page; 2: the JOL questionnaire of game rules; 3: the JOL questionnaire of the current action (shoot or do nothing); 4: customized game; Dash lines to server: transmit children’s behaviour (emotion and actions in game); Dash line to researcher: transmit results of emotion recognition and JOL answers.

Table 2: Recognition Rate and F1 score of AU-based ViT (AU-ViT) on the test set of the ChildEFES.

descriptionView Paper arrow_downwardDownload

Using system and user performance features to improve emotion detection in spoken tutoring dialogs

by Mihai Rotaru

2023, Interspeech 2006

In this study, we incorporate automatically obtained system/user performance features into machine learning experiments to detect student emotion in computer tutoring dialogs. Our results show a relative improvement of 2.7% on... more

descriptionView Paper arrow_downwardDownload

Preliminary Results On Dialogue Act and Subact Classification in Chat-based Online Tutorial Dialogues

by Michael Yudelson

2023

We present in this paper preliminary results with dialogue act classification in human-to-human tutorial dialogues. Dialogue acts are ways to characterize the intentions and actions of the speakers in dialogues based on the... more

descriptionView Paper arrow_downwardDownload

The integration of an emotional system in the Intelligent tutoring system

by Mohamed Ammar

2023, The 3rd ACS/IEEE International Conference onComputer Systems and Applications, 2005.

In this article, we present a new architecture of the Intelligent Tutoring System (ITS) and we suggest an original method, which allows recognizing the expression of the learner's face during exercise. It helps evaluate his affective... more

This system (figure 2) should give an automatic interpretation of the emotional expression of the face of the learner.

The extraction of the eye and mouth contours were tested on coloured images with different emotional expressions and the lightening is priorily anything. The task of the recognition of expression generally requires the knowledge of the contour and the precise position of the characteristic features of the face, such as the eyes, the nose or the mouth (figure 6). We started by detecting the eyes and the mouth as the first step and search to make this phase of extraction of the face features more pertinent and tougher. As shown in figure 6 we started to detect the features of the face (eyes, nose and mouth) after searching these contours and extract its places in X and Y in order to calculate the different distances which may be useful for the recognition of the emotional expression of the face. Se ee Se ee ee ee Seren ym Once the characteristic features localized, we should define the attributes from which we establish the recognition. Concerning the starting vector given by the texture of the face, it has a very high dimension

then we extract the two extreme points and their places.(figure 4). Figure 4: Mouth detection

descriptionView Paper arrow_downwardDownload

Learner characteristics and feedback in tutorial dialogue

by Robert Phillips

2023, Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications - EANL '08

Tutorial dialogue has been the subject of increasing attention in recent years, and it has become evident that empirical studies of human-human tutorial dialogue can contribute important insights to the design of computational models of... more

descriptionView Paper arrow_downwardDownload

An investigation of eureka and the affective states surrounding eureka moments

by JULIANA ANDRES

2023

This paper continues prior work conducted on the analysis of moments of student learning in Physics Playground, a learning environment for qualitative physics. The study analyzed data from 60 tenth-grade students who used Physics... more

descriptionView Paper arrow_downwardDownload

The potential of learning with (and not from) artificial intelligence in education

by Tanya Chichekian

2023, Frontiers in Artificial Intelligence

AI-powered technologies are increasingly being developed for educational purposes to contribute to students' academic performance and overall better learning outcomes. This exploratory review uses the PRISMA approach to describe how... more

A study needed to meet all our criteria to be included in the review. After deleting duplicates and applying these selection criteria to the remaining 438 publications, we narrowed down the review to 48 articles (see Figure 1). The most common reasons for which studies did not qualify for inclusion were that they focused primarily on the description of the design or development of a system, they addressed non-AlI-powered educational technologies, and they evaluated systems’ (not learner’) outcomes. We coded and analyzed studies based on the following elements: the workplace and departments where the authors of the selected articles worked, theoretical framework, teachers’ attributed role in the study, learning outcomes, as well as theoretical and practical contributions. In the following section, we present the current achievements in AIED followed by a summary of the findings from this literature search.

descriptionView Paper arrow_downwardDownload

AI Base E-Learning Solution to Motivate and Assist Primary School Students

by Manori Gamage

2023, International Conference on Advancements in Computing (ICAC)

E-learning is a form of providing education by using electronic devices. Lack of proper mechanisms for encouraging and assisting students are key issues faced by many students in an e-learning environment. The 'Vidu Mithuru' is a... more

the percentage of the progress that student has achieved over time as shown in the “Fig. 1.b.” Fig. 1. ULof the app

For getting the semantic similarity between the student's descriptive answer and sample answer, the created model used more than 5000 pairs of sentences that are related to primary education level in Sri Lanka. To get a good result we have used the cosine similarity algorithm. The model used 70 % of dataset for the training process and 30% of the dataset for testing process. The model nearly achieved 86% of accuracy and the loss is 36% as shown in the “Fig. 2.”

The dataset has created using collection of speech commands gathered from the students age between 8 — 10 years. The dataset has audio files in 5 different emotions as happy, sad, surprise, drowsiness and neutral in the form of .wav format. The feature extraction in speech emotion detection model has been done using MEL Spectrogram Frequency (mel), chroma, mfcc, contrast and tonnetz and model training was done using MLP classifier. The trained accuracy we have obtained was 78% and the accuracy can be further improved by scaling up the dataset collection. We plot he speech signal form “Fig. 4” and spectrogram “Fig. 5” to get the visual representation of the voice commands based on particular emotion. Fig. 4. Feature extracted speech signal form of a voice command with ‘happy' emotion

More than 1500 of facial expression images of the students in age group of 8 — 10 years old are gathered to create the dataset collection. The model has been trained using CNN in order to obtain a higher accuracy. The accuracy gained by training the CNN model with 80 epochs is 99% and loss is 2% as shown in the “Fig. 3.” The accuracy can be improved by feeding more training data to the neural network. By using the trained model, we are detecting 5 main emotions of students as happy, sad, surprise, drowsiness and neutral. Fig. 3. Loss and Accuracy graph of Face emotion detection model

Fig. 5. Spectrogram of the voice command with 'happy' emotion

sLE I. GENERATING SCORES FOR DESCRIPTIVE ANSWERS As an example, if the submitted descriptive answering question contains 5 marks and the percentage of similarity is 60%. Then the actual score for the student answer is given as 3.75 out of 5 (5 * 0.75 = 3.75).

descriptionView Paper arrow_downwardDownload

Affect-driven Learning Outcomes Prediction in Intelligent Tutoring Systems

by Danielle A Allessio

2023, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)

Equipping an Intelligent Tutoring System (ITS) with the ability to interpret affective signals from students could potentially improve the learning experience of students by enabling the tutor to monitor the students' progress and provide... more

descriptionView Paper arrow_downwardDownload

To feel or not to feel: The role of affect in human–computer interaction

by Eva Hudlicka

2023, International Journal of Human-Computer Studies

The past decade has witnessed an unprecedented growth in user interface and humancomputer interaction (HCI) technologies and methods. The synergy of technological and methodological progress on the one hand, and changing user expectations... more

descriptionView Paper arrow_downwardDownload