Figure 1. Layout of the recording area used for the experiment to be left alone in the room since it might suggest to them that they were bein experiment is monitored. This was a concer at the beginning of my project, so I askec Figure 2. Experiment setting Carrying out all the sessions in the same setting helps to avoid effects that occur collected in the form of voice-recordings and eye- tracking data. when a remote eye tracker needs to be moved, such as variations in the distance of the participants to the monitor and in the distance between the monitor and the eye tracker Additionally, it makes it easier to control lighting conditions. The participants came tc the laboratory individually and I made sure they were seated approximately 60 cm away from the eye tracker. I calibrated the eye tracker for each participant before the ax periment: participants focused their gaze on a dot that stopped at nine data points in < 3x3 grid displayed on the screen. Lighting was kept relatively constant by closing the olinds and turning on the same lights for all sessions. All of the participants wore over sar headphones during the eye-tracking session and the volume was maintained at the Figure 3. Sequence of presentation for the tests Tobii Studio tests consist of a sequence of media elements structured along a the order, timing and duration of the stimulus presentation. The order in which th Alt64/Tobii in Barcelona, I discovered it was impossible to have this type of control established method for the randomization feature. After consulting with the distributor Figure 4. Tobii Studio Track Status box Figure 5. Fixation and saccades Among the different possible classifications of the movements the human eye can Figure 6. Areas of Interest For analyzing the data in my pilot study, I created interactive AOlIs for the subtitles. The area would activate and deactivate every time a subtitle was shown on the the duration and number of fixations enclosed within each area. Table 1. Type of eye-tracking data and methods of exploration used (adapted from Saldanha and O ’Brier 2013) of the types of data and the methods of exploration used in this project is given in Table Figure 7. Scanpath as displayed in ESE Another method used to explore visual attention data is the identification of the Figure 8. Heat map with colored areas representing the areas most fixated upon The change of focus of attention is another measure that gives information about Figure 9. Close-up on ESE summary information for subtitle segment an additional column with the shifts between the two A OIs. attention shifts by exporting the data to Excel and then use a Python script that created Table 2. Factorial table for the main experiment with the professional version of the subtitles. Thus, the design required at least eight Table 3. Participants included in the main experiment Table 4. Duration of clips and number of words in the original dialogue another character participated in a discussion. In each of the excerpts the conversatio! revolved about two different topics, with all characters participating in the scene. another character participated in a discussion. In each of the excerpts the conversation Table 5. Subtitle density per condition and clip 4.5. Ethical considerations created the resolution appropriate to fit the screen: 1920x1080. objectives of the pilot experiment were to validate the combination of the data collected Table 7. Sequence randomization for the pilot experiment a sequence to each participant and showed them the three excerpts, alternating the three Table 8. Means fixation (in milliseconds) by area of interest, group and subtitle condition (PRO, NP1 anc NP2) this number rises to 68% in the HLE group. Thus, there is a significant difference Figure 11. Percentage of fixations on areas of interest by Level of L2/L3 and Type of subtitle (F(1,18)=13.39 p<0.01). A paired t-test showed there is a significant difference between Figure 12. Percentage of duration of fixations on areas of interest by Level of L2/L3 and Type of subtitle ‘ixations on the subtitle area than on the image, but their fixations are longer on th image and on the subtitle area, on average, participants in the LLE group have more Table 9. Percentage of correct questions by type of question and condition Table 10. Self-reported comprehension and Subtitle-reading effort by Type of subtitle Table 11. Weekly consumption of audiovisual products in terms of time and means of access (pre- experiment questionnaire) declared that they watch more than 7 hours of audiovisual content on the Internet. towards the Internet as their means to access content. About 50% of all participants Table 13. Use of audiovisual translation modalities for the pre-experiment questionnaire Table 14. Use of audiovisual translation modalities for the main experiment The results of this question for the group of participants in the experimental stage are Table 12. Weekly consumption of audiovisual products in terms of time and means of access (main experiment) 5.1.3. Audiovisual translation modalities Table 15. Use of the Internet per day for participants in the pre-experiment questionnaire and the main experiment 2. Experiment data exploration take part in an eye-tracking and interview session. In this section I will describe the that about 35% of the participants spend between three and four hours using the Internet Table 16. Mean percentage of fixations by area of interest, Level of L2/L3 and Type of subtitle The boxplot in Figure 13 gives a graphic representation of the percentages of Figure 13. Percentage of fixations by area of interest, Level of L2/L3 and Type of subtitle Table 17. Mean percentage of duration of fixations by area of interest, Level of L2/L3 and Type of subtitle Once again, the values for the percentage of duration of fixation seem to suggest Once again, the values ror the percentage Or durauon oF txation seem to suggest differences in the behavior of the participants. The same pattern observed for the percentage of fixations can be observed in the percentage of duration of fixations: a high the time looking at the subtitle area, while in the case of the participants in the HLE Figure 14. Percentage of duration of fixation by area of interest, Level of L2/L3 and Type of subtitle degree of dispersion in the HLE group and a tighter grouping of the data of the LLE group. Figure 15. Mean fixation duration by area of interest, Level of L2/L3 and Type of subtitle Table 18. Mean fixation duration by area of interest, Level of L2/L3 and Type of subtitle Table 20. Self-reported comprehension per Clip, Level of L2/L3 and Type of subtitle found to have a significant effect on some dependent variables. because, as will be shown in the statistical analysis per variable, the Clip variable we Table 19. Percentage of correct answers for different types of attention and Reception capacity * 1=boring to 4=great fun Table 22. Audience enjoyment per Clip, Level of L2/L3 and Type of subtitle the highest for HLE, 3.08, while NP1 has the highest score in the LLE group, of 2.58. Table 21. Subtitle-reading effort per Clip, Level of L2/L3 and Type of subtitle positive and similar in general, the lowest in the HLE group was 3.44 for the NP2, but Table 23. List of variables with codes and descriptions Figure 16. Use of subtitling, original and transformed o=very easy to follow. In the case of Subtitle-reading effort, the level that was causing Figure 17. Subtitle-reading effort, original and transformed Table 24. Estimated marginal means of Total fixation duration on the subtitle area (ms) with the effect of Use of subtitling Figure 18 shows a graphical representation of the variable Use of subtitling per the HLE group declared they never use subtitles. This explains the significant Table 25. Pairwise contrast of estimated marginal means of Total duration of fixations on the subtitle area (ms) with the effect of Use of subtitling Figure 18. Use of subtitling per Level of L2/L3 Table 27. Estimated marginal means of Total duration of fixations on the subtitle area (ms) with the interaction effect between Level of L2/L3 and Use of subtitling. The comparison of the estimated means for the interaction is presented in Table can be seen that the means for the HLE group are smaller in all cases except in those Table 26. Estimated marginal means of Total fixation duration on the subtitle area (ms) with the effect of Level of L2/L3 in the LLE group spent over 28400 ms more looking at the subtitle area than the participants in the HLE group. Figure 19. Estimated marginal means of Total duration of fixations on the subtitle area (ms) with the interaction effect between Level of L2/L3 and Use of subtitling Table 28. Pairwise contrasts of estimated marginal means of Total duration of fixations on the subtitle area (ms) with the interaction effect between Level of L2/L3 and Use of subtitling Figure 20. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of Clip Table 30. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of Clip Table 29. Estimated marginal means of Percentage of fixations with the effect of Level of L2/L3 proficiency in English made while looking at the subtitle area. For participants in the LLE group, this means that 59% of all the fixations during the screening time were made within the subtitle area. Table 31. Pairwise contrasts of estimated marginal means of Percentage fixations on the subtitle area with the effect of Clip * indicates significant results (a=0.05) Table 33. Pairwise contrasts of estimated marginal means of Percentage of fixations on the subtitle area with the effect of Use of subtitling Table 32. Estimated marginal means of Percentage of fixations on the subtitle area with the effect of Use of subtitling A close analysis of Table 33 comparing the estimated means shows the leve Never has significant differences with respect to all the other levels (p<0.01). Referrin to Figure 18, only participants in the HLE group say they never use subtitles whe watching audiovisual material. This indicates that participants who declare to never us subtitles are also the ones who fixated the least frequently on the subtitle area. Thes results show that these participants in the Never group made about 31% of thei fixations on the subtitle area. The differences between this value and the percentage o fixations for participants in all other groups range between 22 and 35 points. Figure 21. Estimated marginal means of Percentage of fixations with the effect of Use of subtitling Level of L2/L3 and Prior knowledge of the clips Table 34. Estimated marginal means of Percentage of fixations on the subtitle area with the interaction effect between Level of L2/L3 and Prior knowledge of the clips * indicates significant results (a=0.05) Table 35. Pairwise contrasts of estimated marginal means of Percentage of fixations on the subtitle area with the interaction effect between Level of L2/L3 and Prior knowledge of the clips 5.3.2.3. Other attention-allocation variables Figure 23. Percentage of duration of fixations and Percentage of fixations on the subtitle area This high correlation indicated that the statistical analysis of Percentage of Juration of fixations on the subtitle area would yield results similar to those obtained This high correlation indicated that the statistical analysis of Percentage of duration of fixations on the subtitle area would yield results similar to those obtained Table 36. Final model of Percentage of fixations on the subtitle area applied to Percentage of fixations on the image area, with the same main effects and interactions Figure 24. Skipped subtitle (%) per Order of presentation and Level of L2/L3 A detailed revision of the pairwise contrasts of the means in Table 40 and Figure Table 39. Estimated marginal means of Skipped subtitles with the effect of order of presentation Figure 25. Estimated marginal means of Skipped subtitles (%) with the effect of Order of presentation Table 40. Pairwise contrasts of estimated marginal means of Skipped subtitles with the effect of Order of presentation Figure 26. Estimated marginal means of Skipped subtitles (%) with the effect of Use of subtitling The conclusions that can be drawn from the analysis for Skipped subtitles are summarized as follows: Table 42. Pairwise contrasts of estimated marginal means of Skipped subtitles with the effect of Use of subtitling Table 41. Estimated marginal means of Skipped subtitles with the effect of Use of subtitling Table 44. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect of Type of subtitle version (t=-5.49; p<0.0001). No significant differences were found between the two non-professional versions (t=-1.56; p=0.122). Table 45. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect of Use of subtitling Table 46 presents the pairwise comparisons between the levels of Use of subtitling. Only the participants who said they Never use subtitling have significantly Figure 28. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the effect of Use of subtitling Table 46. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle aree (ms) with the effect of Use of subtitling Table 48. Pairwise contrasts of estimated marginal means of Mean fixation duration on the subtitle area (ms) with the interaction effect between Clip and Type of subtitle (Type of subtitle as reference factor) The results of the pairwise comparisons taking Clip as the reference factor show Table 47. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the interaction effect between Clip and Type of subtitle The figures show that mean fixations for NP2 are very similar in all three clips, while the mean fixations for PRO and NP1 have more variation between clips. fixations for PRO had more variation. There is no significant difference between the Figure 29. Estimated marginal means of Mean fixation duration on the subtitle area (ms) with the interaction effect between Clip and Type of subtitle Figure 30. Estimated marginal means of Mean fixation duration on the image area (ms) with the effect of Type of subtitle means. and NP1 subtitles (t=3.32; p<0.01). Figure 30 shows the graphical representation of the Table 49. Estimated marginal means of Mean fixation duration on the image area (ms) with the effect of Type of subtitle Table 50. Pairwise contrasts of estimated marginal means of Mean fixation duration on the image area (ms) with the effect of Type of subtitle Table 54. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of Clip Table 53. Estimated marginal means of Attention shift ratio with the effect of Clip Table 52. Estimated marginal means of Attention shift ratio with the effect of Level of L2/L3 make 2.09 shifts from the image to the subtitle or vice versa per subtitle, while the meat Figure 31. Estimated marginal means of Attention shift ratio with the effect of Clip Table 55. Estimated marginal means of Attention shift ratio with the effect of Use of subtitlin, Figure 32 shows the mean attention shifts per subtitle for each level of Use of subtitling. Figure 32. Estimated marginal means of Attention shift ratio with the effect of Use of subtitling Table 56. Pairwise contrasts of estimated marginal Attention shift ratio with the effect of Use of subtitling Table 57. Estimated marginal means of Attention shift ratio with the effect of Type of subtitle Figure 33. Estimated marginal means of Attention shift ratio with the effect of Type of subtitle Table 59. Estimated marginal means of Attention shift ratio with the interaction between Clip and Type ot subtitle Table 60 presents the mean values for attention shifts taking Clip as the reference factor. In Clip 2, mean differences are significantly shorter for NP2. A graphical representation of the means is shown in Figure 34. Considering Clip as the reference Table 58. Pairwise contrasts of estimated marginal means of Attention shift ratio with the effect of Type of subtitle Figure 34. Estimated marginal means of Attention shift ratio with the interaction between Clip and Type of subtitle * indicates significant results (a=0.05) Table 60. Pairwise contrasts of estimated marginal means of Attention shift ratio with the interaction effect between Clip and Type of subtitle (Clip as reference factor) Table 61. Final model with significant main effects and interactions of Subtitle-reading effort [he participants in the experiment were asked to rate the difficulty they had following he subtitles, using a scale ranging from O=very difficult to 5=very easy. Table 61 yresents the model with the variables and interactions that had a significant effect on subtitle-reading effort. The variables were Prior knowledge of the clips (F=9.51; <0.01) and Use of subtitling (F=3.06; p<0.05). The interaction between Clip and Type yf subtitle was also found to have a significant effect (F=3.10; p<0.05). The Type of subtitle variable did not have a significant effect, and neither did Level of L2/L3. The atter is not included in the final model because, although by itself it is not significant, it iffects the results of the significant variables. latter is not included in the final model because, although by itself it is not significant, it Table 62. Estimated marginal means of Subtitle-reading effort with the effect of Prior knowledge of the clips Figure 35. Estimated marginal means of Subtitle-reading effort with the effect of Use of subtitling Table 64. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with the effect of Use of subtitling Table 63. Estimated marginal means of Subtitle-reading effort with the effect of Use of subtitling Table 66. Pairwise contrasts of estimated marginal means of Subtitle-reading effort with the interaction effect between Clip and Type of subtitle (Clip as reference factor) In the case of the NP2 version, the subtitle-reading difficulty score for Clip 3 i: Table 65. Estimated marginal means of Subtitle-reading effort with the interaction effect between Clip and Type of subtitle Table 66 shows the pairwise comparisons considering Clip as the reference Figure 36. Estimated marginal means of Subtitle-reading effort with the interaction effect between Clip and Type of subtitle Table 67. Final model with significant main effects on Audience enjoyment Table 69. Estimated marginal means of Audience enjoyment with the effect of Clip these means. The mean score of Clip 3 is significantly different from the other two Table 68. Estimated marginal means of Audience enjoyment with the effect of Level of L2/L3 Table 70. Pairwise contrasts of estimated marginal means of Audience enjoyment with the effect of Level of L2/L3 Table 72. Pairwise contrasts of estimated marginal means of Audience enjoyment with the effect of Subtitle-reading effort Table 71. Estimated marginal means of Audience enjoyment with the effect of Subtitle-reading effort Table 72 shows the pairwise contrasts between the means of Audience enjoyment levels 2 and 3, nor 2 and 4. The difference between 3 and 4 was 0.28 points (t=1.99; Figure 37. Estimated marginal means of Audience enjoyment with the effect of Subtitle-reading effort The main highlights resulting from the analysis of Audience enjoyment are: Table 73. Final model with significant main effects and interactions of Self-reported comprehension Table 75. Estimated marginal means of Self-reported comprehension with the effect of Use of subtitling participants at the Never level reported significantly greater comprehension than did the Table 75 shows the mean Self-reported comprehension scores for all the levels of Use of Table 74. Estimated marginal means of Self-reported comprehension with the effect of Prior knowledg of the clips Figure 38. Estimated marginal means of Self-reported comprehension with the effect of Use of subtitling Table 77. Estimated marginal means of Self-reported comprehension with the effect of Audience enjoyment Table 76. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect of Use of subtitling participants in all three other groups (p<0.05 in all cases). Figure 38 shows the mean self-reported comprehension ratings for all groups. The lowest mean self-reported comprehension score is 3.0 for the participants who did Table 78. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect of Use of subtitling The pairwise contrasts of the estimated means of the self-reported comprehension Table 80. Pairwise contrasts of estimated marginal means of Self-reported comprehension with the effect of Narrative attention points (t=3.89; p<0.001) and the difference between the participants with two correct Table 79. Estimated marginal means of Self-reported comprehension with the effect of Narrative attention not answer any of the questions correctly. The scores for the participants who answered one or two questions correctly are very similar: 4.0 and 3.8 respectively. not answer any of the questions correctly. The scores for the participants who answered Figure 40. Estimated marginal means of Self-reported comprehension with the interaction effect between Clip and Type of subtitle 5.3.9.6. Self-reported comprehension - Summary Table 82. Pairwise contrasts of Self-reported comprehension with the interaction effect between Clip and Type of subtitle (Clip as reference factor) Table 81. Estimated marginal means of Self-reported comprehension with the interaction effect between Clip and Type of subtitle Figure 41. Estimated marginal means of Reception capacity with the effect of Clip Table 85. Pairwise contrasts of estimated marginal means of Reception capacity with the effect of Clip of the means is included in Figure 41. Table 84. Estimated marginal means of Reception capacity with the effect of Cli Figure 42. Estimated marginal means of Reception capacity with the effect of Type of subtitle Table 87. Pairwise contrasts of Reception capacity with the effect of Type of subtitle * indicates significant results (a=0.05) 5.3.10.3. Prior knowledge of the clips Table 86. Estimated marginal means of Reception capacity with the effect of Type of subtitle lowest mean is 5.37 points for the NP2 version, and the means for PRO and NP1 are very similar: 5.91 and 5.96 respectively. Table 90. Pairwise contrasts of estimated marginal means of Iconic attention with the effect of Clip Table 89. Estimated marginal means of Iconic attention with the effect of Clip different from the average of correct answers for Clip 2. This indicates that the participants answered only half of the Iconic attention questions of Clip 2 correctly. Table 92. Estimated marginal means of Narrative attention with the effect of Subtitle-reading effort Table 93. Pairwise contrasts of estimated marginal means of Narrative attention with the effect of Subtitle-reading effort The main highlights resulting from the analysis of Narrative attention are summarized as follows: Table 91. Estimated marginal means of Narrative attention with the effect of Prior knowledge of the clips Table 96. Pairwise contrasts of Verbal attention with the effect of Type of subtitle Table 98. Pairwise contrasts of Verbal attention with the effect of Clip Table 97. Estimated marginal means of Verbal attention with the effect of Clip difference between Clip 3 and the other two clips is statistically significant (p<0.05). Table 95. Estimated marginal means of Verbal attention with the effect of Type of subtitle Table 99. Summary of dependent variables with main effects and interactions marked as x supported with interview data in Section 6.2.4. Significant effects were also found for Figure 43. Example of attention shifts when subtitles overlap The results for Hi and its sub-hypotheses are summarized in Table 100. The data do not r TT a | “4 fT a1 provide enough evidence to confirm H:. Only two of the seven postulated sub- obtained results that are not significantly different from the professional version. Table 101. Summary of the hypothesis testing for H2 6.1.3. Hypothesis 3: Type of non-professional subtitles Table 102. Summary of the hypothesis testing for H3 6.1.4. Hypotheses based on qualitative data the self-reported measurements assessing comprehension and Subtitle-reading effort. Table 103. Summary of the hypothesis testing for Hi4 and His 2, Findings from the qualitative data as shown in Table 103, itis not possible to confirm these hypotheses. but this did not transpire during the interviews. Considering the participants’ comments, Figure 44. Heat maps of the fixations of HLE and LLE participants in Order of presentation 5 There are some interesting aspects about the behavior of these participants. It is have more fixations at the center of the screen on the image area. important to point out here that these behavior patterns are not restricted to the Figure 45. Example of HLE participants’ fixations Figure 46. Heat maps with fixations durations made by all participants in Order 1 Figure 47. Percentage of duration of fixations per Clip and Level of L2/L3 image. This trade-off in attention allocation is evident in the results of the statistica increases for all participants in Clip 2. The mean percentages of duration of fixation on Figure 48. Heat maps for participants who use subtitles never (left) and occasionally (right) from the action on the screen and that this bothers her. She realizes that even if she is Figure 49. Layout of the subtitles on the screen Table 104. Confirmation of sub-hypotheses for Hi: Type of subtitle Considering these results, it is safe to assume that the reception of professional instructions for subtitles which are more appropriate for reading. Among these sub indicate that the type of subtitle was a relevant factor from the participants’ perspective. Table 105. Confirmation of sub-hypotheses for H3: Type of non-professional subtitles Table 106. Confirmation of sub-hypotheses for Hz: Level of L2/L3 subtitles, as suggested by the results for H1. Table 107. Confirmation of Quantitative data sub-hypotheses: Hi4 and His 7.2. Complementary findings American version and the Iberian version, but they did not liken that situation to the subtitles they had just seen. Clip 1: The Big Bang Theory, season 2, episode 4 “The Griffin Equivalency” Clip 3: The Big Bang Theory, season 2, episode 15 “The Maternal Capacitance” Preg.5.- Idiomas hablados (enumeéralos en orden del que mas usas al que menos usas): “Gracias por aceptar participar en nuestro estudio. Por favor, responde las siguientes preguntas de la forme mas precisa posible. Pag.2.- Datos personales Pag.1.- ;Bienvenido a esta encuesta! Preg.8.- Indica la frecuencia con la que has usado los siguientes tipos de traduccion en los ultimos seis (6) meses al ver material audiovisual producido originalmente en idiomas extranjeros:
