Academia.eduAcademia.edu

Figure 11 – uploaded by David Cooper

See full PDFdownloadDownload figure
Table 4.2. Features used for each problem that includes an affective state label in order to train the emotion classifiers (features are shown in abbreviated form). The nine tutor features are shown on the left and the ten sensor features are on the right. Features used in a classifier that are significantly better than the baseline (p < 0.05) are in bold. are in bold.

Table 4 2. Features used for each problem that includes an affective state label in order to train the emotion classifiers (features are shown in abbreviated form). The nine tutor features are shown on the left and the ten sensor features are on the right. Features used in a classifier that are significantly better than the baseline (p < 0.05) are in bold. are in bold.

Related Figures (31)

Figure 1.1. The Emotion Scale on the Left [50] and the Personal Interaction Coding Inventory on the right [15].
Figure 1.1. The Emotion Scale on the Left [50] and the Personal Interaction Coding Inventory on the right [15].
Figure 3.1. An example problem presented by the Wayang system. Jake is on the lower right corner. The Hint Toolbar is on the right.
Figure 3.1. An example problem presented by the Wayang system. Jake is on the lower right corner. The Hint Toolbar is on the right.
3.2.2 Sensor Features
3.2.2 Sensor Features
Table 3.2. The ten sensor features below are summarized by their mean, standard deviation, min and max values and then these 40 summarized features are selected by using regression models to predict confidence, frustration, excitement, and interest. This table defines the abbreviations for each feature. In addition to predicting frustration, our model is meant to predict excitement, interest, and confidence. The sensor features considered in our analysis are described
Table 3.2. The ten sensor features below are summarized by their mean, standard deviation, min and max values and then these 40 summarized features are selected by using regression models to predict confidence, frustration, excitement, and interest. This table defines the abbreviations for each feature. In addition to predicting frustration, our model is meant to predict excitement, interest, and confidence. The sensor features considered in our analysis are described
The third chair feature is meant to determine if the student is sitting forward. From the three pressure values from the back of the chair, each having the range and the net change in pressure of the back: 0, 1023], we compute the Sit Forward feature as follows the most useful features from [38]. These are the net change in pressure of the seat, We compute three features from the 6 chair sensors. The first two are based on
The third chair feature is meant to determine if the student is sitting forward. From the three pressure values from the back of the chair, each having the range and the net change in pressure of the back: 0, 1023], we compute the Sit Forward feature as follows the most useful features from [38]. These are the net change in pressure of the seat, We compute three features from the 6 chair sensors. The first two are based on
Figure 3.2. A student at the client computer puts on a bracelet and starts the two client programs indicating the wrist ID of the bracelet. The bracelet sends Skin Conductance data to the Wrist Node, then logs bracelet data from all of the students in the classroom. The User Model System (UMS) receives the bracelet data through the Wrist Stream. The UMS client performs the same task as the Wrist Node for each of the other three sensor sources. The ITS logs student interactions, and sends Tutor Data to the UMS. The data is time synced based on the client’s system time. The UMS uses all available streams of data to make user predictions to improve the ITS Client interaction. Figure 3.2. A student at the client computer puts on a bracelet and starts the
Figure 3.2. A student at the client computer puts on a bracelet and starts the two client programs indicating the wrist ID of the bracelet. The bracelet sends Skin Conductance data to the Wrist Node, then logs bracelet data from all of the students in the classroom. The User Model System (UMS) receives the bracelet data through the Wrist Stream. The UMS client performs the same task as the Wrist Node for each of the other three sensor sources. The ITS logs student interactions, and sends Tutor Data to the UMS. The data is time synced based on the client’s system time. The UMS uses all available streams of data to make user predictions to improve the ITS Client interaction. Figure 3.2. A student at the client computer puts on a bracelet and starts the
Table 3.4. This table lists the variables that the stepwise regression method selected as relevant, for each of the regression models in Table 3.3. Each of these features significantly contribute to the prediction of emotion self-reports (p < 0.01), and are listed in order of relevance (The feature at the top is the best predictor.) The abbreviations of these features are defined in Tables 3.1 and 3.2. Table 3.3. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using stepwise linear least squares regression, and variables entered into the model are shown in Table 3.4. The top row lists the feature sets that are available. The left column lists the emotional self-reports being predicted. R values correspond to the fit of the model (best fit models for each emotion are in bold). N values vary because some students are missing data for a sensor.
Table 3.4. This table lists the variables that the stepwise regression method selected as relevant, for each of the regression models in Table 3.3. Each of these features significantly contribute to the prediction of emotion self-reports (p < 0.01), and are listed in order of relevance (The feature at the top is the best predictor.) The abbreviations of these features are defined in Tables 3.1 and 3.2. Table 3.3. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using stepwise linear least squares regression, and variables entered into the model are shown in Table 3.4. The top row lists the feature sets that are available. The left column lists the emotional self-reports being predicted. R values correspond to the fit of the model (best fit models for each emotion are in bold). N values vary because some students are missing data for a sensor.
Table 3.5. This table shows the results of the best classifier of each emotional response (confident, frustrated, excited, and interested). Accuracy of no classifier (last column) is a prediction that the emotional state is always low. Values in parentheses include the middle values in the testing set as negative examples.
Table 3.5. This table shows the results of the best classifier of each emotional response (confident, frustrated, excited, and interested). Accuracy of no classifier (last column) is a prediction that the emotional state is always low. Values in parentheses include the middle values in the testing set as negative examples.
Table 4.1. The mapping of tags to text in Fig. 4.1 above. Figure 4.1. An example of the Emotion query. Table 4.1 below has the values for each <> enclosed word, except for (<Name>), which is the name of the student.
Table 4.1. The mapping of tags to text in Fig. 4.1 above. Figure 4.1. An example of the Emotion query. Table 4.1 below has the values for each <> enclosed word, except for (<Name>), which is the name of the student.
Table 4.4. These are the results of the feature selection. The baseline classifier for each emotion is just a linear model trained on a constant. The classifier names are the concatenation of an abbreviated emotion (e.g., confident, interested, and excited) and the contributing sensor features. If there are no sensor features, then Tutor comes after the emotion, and when there is more than one classifier with the same feature set a letter is added to disambiguate the names (e.g. TutorA is only tutor data, and TutorM is only tutor data, but the stepwise selection process had the mouse data available, but no mouse features were selected). Names in bold are for classifiers that performed significantly better than the baseline for that emotion in at least one way.
Table 4.4. These are the results of the feature selection. The baseline classifier for each emotion is just a linear model trained on a constant. The classifier names are the concatenation of an abbreviated emotion (e.g., confident, interested, and excited) and the contributing sensor features. If there are no sensor features, then Tutor comes after the emotion, and when there is more than one classifier with the same feature set a letter is added to disambiguate the names (e.g. TutorA is only tutor data, and TutorM is only tutor data, but the stepwise selection process had the mouse data available, but no mouse features were selected). Names in bold are for classifiers that performed significantly better than the baseline for that emotion in at least one way.
Table 4.5. Classifier ranking using cross-validation data (p < 0.05) results are shown in Table 4.5. Given these results, our findings suggest that the tutor could generate interven-
Table 4.5. Classifier ranking using cross-validation data (p < 0.05) results are shown in Table 4.5. Given these results, our findings suggest that the tutor could generate interven-
Table 4.6. This shows validation results of all classifiers that performed better than the baseline classifier during training. All values are the mean value per student. Fall specifies the training set based on the leave-one-student-out cross-validation, and Spring specifies the results of the classifiers trained on the training set (Fall 2008 Data), and tested on the validation set (Spring 2009 Data). Values in bold are significantly better (p < 0.05) than the baseline. Table 4.7. Classifier ranking using validation data from the Spring of 2009. All differences indicated by ‘>’ are significant with p < 0.01.
Table 4.6. This shows validation results of all classifiers that performed better than the baseline classifier during training. All values are the mean value per student. Fall specifies the training set based on the leave-one-student-out cross-validation, and Spring specifies the results of the classifiers trained on the training set (Fall 2008 Data), and tested on the validation set (Spring 2009 Data). Values in bold are significantly better (p < 0.05) than the baseline. Table 4.7. Classifier ranking using validation data from the Spring of 2009. All differences indicated by ‘>’ are significant with p < 0.01.
Table 5.1. The ten acoustical features below are selected by using a linear least squares stepwise regression models to predict gender, American vs. Asian culture, or one of four cultures. This table lists each feature with an abbreviation and a definition. the Youoicea feature is between 20% and 80%, then it is considered natural speech.
Table 5.1. The ten acoustical features below are selected by using a linear least squares stepwise regression models to predict gender, American vs. Asian culture, or one of four cultures. This table lists each feature with an abbreviation and a definition. the Youoicea feature is between 20% and 80%, then it is considered natural speech.
Minimum Pause Length for Utterance Figure 5.1. The effect of pause length on extracting useful utterances for the clas- sification of gender. The red line with boxes expresses utterances extracted with no buffer, and the blue line with diamonds expresses utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.
Minimum Pause Length for Utterance Figure 5.1. The effect of pause length on extracting useful utterances for the clas- sification of gender. The red line with boxes expresses utterances extracted with no buffer, and the blue line with diamonds expresses utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.
significantly for 500 ms and above. Figure 5.2. The effect of pause length on extracting useful utterances for the two class classification of culture (American vs. Asian). The left shows the results for male subjects. The red line with boxes expresses male utterances extracted with no buffer, the blue line with diamonds expresses male utterances extracted with a 100ms silence buffer on each side. The right shows results for female subjects. The red line with down triangles expresses female utterances extracted with no buffer, the blue line with up triangles expresses female utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.
significantly for 500 ms and above. Figure 5.2. The effect of pause length on extracting useful utterances for the two class classification of culture (American vs. Asian). The left shows the results for male subjects. The red line with boxes expresses male utterances extracted with no buffer, the blue line with diamonds expresses male utterances extracted with a 100ms silence buffer on each side. The right shows results for female subjects. The red line with down triangles expresses female utterances extracted with no buffer, the blue line with up triangles expresses female utterances extracted with a 100ms silence buffer on each side. 95% confidence intervals are shown.
Figure 6.1. Left: The key points for Group A with one simulated patient. The key points in this room were 1) The monitor, 2) the patient, and 3) the telephone. In this frame, the student is closest to the monitor, but actually interacting with the patient. Right: The key points for Group B with two simulated patients. The key points in this room were 1) patient 1, 2) patient 2, and 3) the telephone. In this frame the student is closest to patient 1.
Figure 6.1. Left: The key points for Group A with one simulated patient. The key points in this room were 1) The monitor, 2) the patient, and 3) the telephone. In this frame, the student is closest to the monitor, but actually interacting with the patient. Right: The key points for Group B with two simulated patients. The key points in this room were 1) patient 1, 2) patient 2, and 3) the telephone. In this frame the student is closest to patient 1.
Figure 6.2. Two examples of the Reading People Tracker. Yellow rectangles repre- sent objects. Pink rectangles represent upper body. Red rectangles with a small inner rectangle represent heads. top: Multiple objects are found for one person, bottom: the foreground detection of the person begins to fade as the student stands very still. when the person stopped for an extended period of time, the track was lost. sent objects. Pink rectangles represent upper body. Red rectangles with a small inner a motion detection algorithm based on background estimation and subtraction, anc
Figure 6.2. Two examples of the Reading People Tracker. Yellow rectangles repre- sent objects. Pink rectangles represent upper body. Red rectangles with a small inner rectangle represent heads. top: Multiple objects are found for one person, bottom: the foreground detection of the person begins to fade as the student stands very still. when the person stopped for an extended period of time, the track was lost. sent objects. Pink rectangles represent upper body. Red rectangles with a small inner a motion detection algorithm based on background estimation and subtraction, anc
Figure 6.3. Two examples of the CAMSHIFT tracker. The blue box is the center of the track, representing the head position. The red ellipse is the area being tracked. top: the tracker center stays near the student, but the size of the track is most of the frame, bottom: the tracker center goes to the wrong side of the patient.
Figure 6.3. Two examples of the CAMSHIFT tracker. The blue box is the center of the track, representing the head position. The red ellipse is the area being tracked. top: the tracker center stays near the student, but the size of the track is most of the frame, bottom: the tracker center goes to the wrong side of the patient.
of one of the students from our study. Figure 6.4. An example of the Lucas-Kanade tracker. The head region is selected by hand. As the student moves around the ‘good features to track’ (in green) disperse away from the head. from the head. Figure 6.4 shows an example of the Lucas-Kanade tracker on the head
of one of the students from our study. Figure 6.4. An example of the Lucas-Kanade tracker. The head region is selected by hand. As the student moves around the ‘good features to track’ (in green) disperse away from the head. from the head. Figure 6.4 shows an example of the Lucas-Kanade tracker on the head
Figure 6.5. Two examples of the distribution field tracker. Blue box represents the area being tracked, in this case, the head of the student. top: tracker incorrectly selects filing cabinet, bottom: tracker incorrectly selects a box with dials
Figure 6.5. Two examples of the distribution field tracker. Blue box represents the area being tracked, in this case, the head of the student. top: tracker incorrectly selects filing cabinet, bottom: tracker incorrectly selects a box with dials
Boxplot of distances for ALL students in ALL rooms. Num points: 2027 Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right: Box plot of distances from ground truth for top ten trackers. Trackers beginning with ‘n’ use only initialization, trackers beginning with ‘a’ are variants of the CAMSHIFT tracker, and trackers beginning with ‘i’ are the DF trackers. Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right
Boxplot of distances for ALL students in ALL rooms. Num points: 2027 Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right: Box plot of distances from ground truth for top ten trackers. Trackers beginning with ‘n’ use only initialization, trackers beginning with ‘a’ are variants of the CAMSHIFT tracker, and trackers beginning with ‘i’ are the DF trackers. Figure 6.6. Left: Top ten trackers in order of increasing mean squared error. Right
Figure 6.7. The predicted head positions of one student for two different trackers. Blue points predict closest to monitor. Green points predict closest to patient, and Orange points predict student closest to the telephone. Left: heuristic detection (no tracking). Right: distribution field tracker of switches to another bin are computed.
Figure 6.7. The predicted head positions of one student for two different trackers. Blue points predict closest to monitor. Green points predict closest to patient, and Orange points predict student closest to the telephone. Left: heuristic detection (no tracking). Right: distribution field tracker of switches to another bin are computed.
Table 6.1. Cross-validated mean R? values for the five best trackers. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using a branch and bound linear regression. The top row lists the tracker. The left column lists the emotional self-reports being predicted and the group that reports are predicted for. R? values correspond to the fit of the model (best fit models for each emotion are in bold). Group A had one simulated patient and Group B had two simulated patients. Each group has 23 students reporting emotions. Table 6.2 shows results for the best accuracy classifiers. Note that the many of
Table 6.1. Cross-validated mean R? values for the five best trackers. Each cell corresponds to a linear model to predict emotion self-reports. Models were generated using a branch and bound linear regression. The top row lists the tracker. The left column lists the emotional self-reports being predicted and the group that reports are predicted for. R? values correspond to the fit of the model (best fit models for each emotion are in bold). Group A had one simulated patient and Group B had two simulated patients. Each group has 23 students reporting emotions. Table 6.2 shows results for the best accuracy classifiers. Note that the many of
Table 6.2. The ten best classifiers, one for each emotion for each group. Trackers starting with an i are DF trackers using variant 4. f indicates using variant 3. g indicates using variant 5. a indicates the CAMSHIFT tracker, and b indicates using variant lb. t indicates using variant la. s indicates using the faster variant of the DF tracker, and n indicates that just initialization is done.
Table 6.2. The ten best classifiers, one for each emotion for each group. Trackers starting with an i are DF trackers using variant 4. f indicates using variant 3. g indicates using variant 5. a indicates the CAMSHIFT tracker, and b indicates using variant lb. t indicates using variant la. s indicates using the faster variant of the DF tracker, and n indicates that just initialization is done.
Related topics:
PsychologySpeech ProsodyIntelligent Tutoring SystemsAffect Detection
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved