Multimodal Reading Comprehension

Este artigo é uma resenha bibliográfica derivada de um projeto de pesquisa focando na multimodalidade e seu possível impacto na aprendizagem de línguas estrangeiras, com especial atenção aos desenhos de textos (KRESS & VAN LEEUWEN, 2001) de acordo com o informado pelos modelos de aprendizado multimodal (SCHNOTZ, 2005; MAYER, 2001). Em primeiro lugar, o artigo resgata estes modelos e os relaciona ao campo da aquisição de um segundo idioma. Em continuação, descreve-se alguns pontos de possível encontro e diálogos entre estes dois campos de pesquisa. Finalmente, apresenta alguns estudos que se realizaram em torno ao aprendizado multimodal de línguas como exemplos deste diálogo.

Although ubiquitous learning enhances students' access to learning materials, it is crucial to find out which media presentation modes produce the best results for English listening comprehension. The present study examined the effect of media presentation mode (sound and text versus sound) on English listening comprehension and cognitive load. Participants were 162 students majoring in Applied Foreign Language at a university in Taiwan. The students were randomly assigned to either single mode (sound) or double mode (sound and text). The research questions are (a) whether students learning with double mode outperformed students learning with single mode in listening comprehension; and (b) whether students learning with double mode encountered less cognitive load than students learning with single mode. If the answers to these questions are affirmative, then the modality effect occurs and the redundancy effect does not occur. The results demonstrated that (a) text significantly ...

This research aims to describe the perception on the use of Wattpad to improve students' ability and interest in writing. This study applied descriptive qualitative method. Data and data sources in this study were grade X senior high school teachers and students. Data collection techniques using interview and observation techniques. The data analysis technique used is interactive data analysis techniques. The results showed that most students argued learning by using components in Wattpad such as picture for cover could improve their motivation in writing. teacher also claimed that the use of Wattpad as learning media had an impact in improving the students writing ability.

Resumen El principio y el efecto de redundancia parecieran ser un impedimento para el aprendizaje en campos diversos. Este trabajo da cuenta de una investigación que intenta probar si tal redundancia tiene también efectos negativos en el aprendizaje de expresiones idiomáticas en inglés como lengua extranjera. Dos grupos de estudiantes universitarios fueron expuestos a dos tipos de presentaciones multimodales: uno, a texto en pantalla, narración e imagen fija; y el otro a texto en pantalla y narración. Según el principio de redundancia, la presentación del primer grupo duplicaría la información y conllevaría a una sobrecarga cognitiva. Aunque los resultados indican que no hay mayor variación entre los dos grupos en cuanto a retención, en transferencia hay variaciones atribuibles a las características semánticas del vocabulario presentado. Palabras clave: aprendizaje multimodal, principio y efecto de redundancia, expresiones idiomáticas, inglés como lengua extranjera Abstract The prin...

The multimedia elements need to be carefully integrated together to maximize the impact of the multimedia elements on learning. Redundancy information presented through audio and visual channels can inhibit learning for individuals diagnosed with ADHD who may experience challenges in the processing of information through visuospatial and phonological loop channels in the memory system. This study explores how redundancy affects the individuals’ with selfreported higher levels of ADHD symptoms ability to process information presented using multimedia presentations. Individuals with higher-reported ADHD symptoms had lower performance levels when using the multimedia presentation with redundancy. Prevalence and Diagnoses Prevalence rates of individuals who have been diagnosed with attention deficit/ hyperactivity disorder (ADHD), have not varied significantly over the past 3 decades (Polanczyk, Willcutt, Salum, Kieling, & Rohde, 2014) with parents reporting that approximately 11% of th...

This study was designed to examine the effect of the redundancy principle in a multimedia presentation constructed for foreign language vocabulary learning on undergraduate students' retention. During the experiment, students received a multimedia presentation on Turkish vocabulary and answered questions on retention test. Some students received animation, concurrent narration and concurrent text (ANT group); other students received only animation and concurrent narration (AN group). According to the redundancy principle, adding redundant on-screen text to a multimedia explanation where it is already narrated, results in poorer learning (Kalyuga, Chandler & Sweller, 1999; Mayer, Heiser & Lonn, 2001). That is, when making a multimedia presentation with a narration and animation, the designers should not add on-screen text that duplicates words that are already spoken in the narration although students have more exposure to the multimedia presentation when it is delivered in three ways (animation, narration and text). The reason for removing on-screen text from the multimedia presentation is because of the cognitive theory of multimedia learning that is based on three assumptions: dual-channel (Clark & Paivio, 1991), limited capacity (Baddeley, 1992) and generative learning assumption (Mayer, 1996) or active processing. On the other hand, the reason for adding on-screen text is based on the information delivery hypothesis, which states that students learn more when the same information is delivered by means of more paths rather than fewer paths (Mayer et al, 2001). That is, "adding on-screen text to a narrated animation will result in better performance on tests of learning that focus on remembering verbal explanation (ie, retention test)" (Mayer et al, 2001, p. 190).Therefore, the underlying hypothesis of this study is that when the students are exposed to the material in multiple ways through ANT, the learning and the retention will have better results in foreign language learning. Early research on vocabulary learning with text and pictures has consistently found that when the written vocabulary words are paired with pictures, there is a better retention of vocabulary words (Paivio, Clark & Lambert, 1988; Plass & Jones, 2005). Besides, presenting both on-screen text and narration at the same time is better than presenting just on-screen text or narration because students might choose the one that best fits their learning style (Mayer et al, 2001). This small study is expected to contribute to the cognitive theory of multimedia learning literature in three ways. First, the study was realized in ecologically more valid environment than Mayer's experiments, because the experiment was done through authentic classroom experience. Unlike the experiments done in Mayer's cognitive laboratory, this study was done within regular classroom settings. Classroom research is an important way to establish external validity of findings when using multimedia. Second, the study was done in a different content area, foreign language learning. It was suggested by Plass and Jones (2005, p. 483) that an area for future research is "the integration of second-language acquisition theory and cognitive theories of multimedia learning." Third, the study was conducted with non-Turkish speaking students, which is also another contribution to the literature on multimedia learning. Besides, students'

Resumen El principio y el efecto de redundancia parecieran ser un impedimento para el aprendizaje en campos diversos. Este trabajo da cuenta de una investigación que intenta probar si tal redundancia tiene también efectos negativos en el aprendizaje de expresiones idiomáticas en inglés como lengua extranjera. Dos grupos de estudiantes universitarios fueron expuestos a dos tipos de presentaciones multimodales: uno, a texto en pantalla, narración e imagen fija; y el otro a texto en pantalla y narración. Según el principio de redundancia, la presentación del primer grupo duplicaría la información y conllevaría a una sobrecarga cognitiva. Aunque los resultados indican que no hay mayor variación entre los dos grupos en cuanto a retención, en transferencia hay variaciones atribuibles a las características semánticas del vocabulario presentado. Palabras clave: aprendizaje multimodal, principio y efecto de redundancia, expresiones idiomáticas, inglés como lengua extranjera Abstract The prin...

If as literacy educators we are to be successful brokers of our own destinies and those of our students in mat- ters of policy and practice, we have to communicate in clear and consistent language. Currently, the M word (multiliteracies) is often conflated with its so-called near synonyms—multimodal reading and writing, new literacies, digital literacies, and multiple literacies—an issue that needs rectifying.

This literature review article approaches the topic of information and communications technologies from the perspective of their impact on the language learning process, with particular emphasis on the most appropriate designs of multimodal texts as informed by models of multimodal learning. The first part contextualizes multimodality within the fields of discourse studies, the psychology of learning and CALL; the second, deals with multimodal conceptions of reading and writing by discussing hypertextuality and literacy. A final section outlines the possible implications of multimodal learning models for foreign language teaching and learning. Key words: multimodal discourse, literacies, multimodal principles and design Resumen En este artículo se hace una reseña de la literatura en torno al impacto de las tecnologías de la información y comunicaciones desde la perspectiva de su impacto en el proceso de aprendizaje de la(s) lengua(s), con particular énfasis en los diseños multimodal...

This literature review article approaches the topic of information and communications technologies from the perspective of their impact on the language learning process, with particular emphasis on the most appropriate designs of multimodal texts as informed by models of multimodal learning. The first part contextualizes multimodality within the fields of discourse studies, the psychology of learning and CALL; the second, deals with multimodal conceptions of reading and writing by discussing hypertextuality and literacy. A final section outlines the possible implications of multimodal learning models for foreign language teaching and learning. Key words: multimodal discourse, literacies, multimodal principles and design Resumen En este artículo se hace una reseña de la literatura en torno al impacto de las tecnologías de la información y comunicaciones desde la perspectiva de su impacto en el proceso de aprendizaje de la(s) lengua(s), con particular énfasis en los diseños multimodal...

Listening plays a vital role in EFL classes. With the advance of technology nowadays, teachers can use more than one means-namely multimodality-as a tool of instruction. This study is aimed to discover whether audio or video modality is more beneficial for EFL students in listening comprehension. It used the post-test control group design to measure the participants' performances with regard to the different types of modality in listening comprehension test. The participants, 50 students of English Department of a private university in Greater Jakarta area, were given two types of test: Audio listening test (ALT) and Video listening test (VLT). The analysis used quantitative method-t-test in SPSS program-which compared the results of two kinds of the test. The findings show that there is a significant difference between the results of ALT and those of VLT. It suggests that VLT provides better scores compared to ALT scores.

This study aims to discover the view, opinion, perception and experiences of Turkish teachers and secondary school students towards Wattpad, an online literacy community, to interpret them educationally, to determine the reasons for their use and to interpret the main plot of youth literature on the axis of changing reading perception/behavior and expectations of youth. It is foreseen that the study has comparatively discussed literature-education and new reading types on the axis of the teacher and student, the direct participants of learning-teaching within this scope, and therefore fills the gap in the literature. The research design is phenomenology, one of the qualitative research designs. Data obtained through semi-structured interviews from Turkish teachers and secondary school students were subjected to content analysis. The findings of the study are as follows: Youth enjoy spending long time on Wattpad via their phones and feel themselves free. They think that the application develops reading and writing habits; read and write mainly about love and science fiction; prefer digital reading over printed books and/or classics; comment and criticize books/articles with their peers. Turkish teachers have educational concerns and do not find appropriate for youth to spend time on this platform; because it includes violence, sexuality etc. Therefore, it can be said that there is a conflict and a difference of opinion between teachers and youth.

Education is one of the main needs that must be met. Education has become a top priority after the very advanced developments in the era of globalization, where everyone must use science to compete. The development of the world is very fast and fast, requiring one to prioritize education so as not to be excluded from social life. Because education has influenced various aspects of life. In the era of globalization that is increasingly sophisticated with the advancement of information technology that continues to develop, and is used as the main factor in globalization, information technology, as well as information technology, has made it possible to realize effective and fun learning, which involves students actively. In the field of education, information technology has changed the delivery of subject matter to students and can help teachers in teaching. There are several types of technology, one of which is called Wattpad. Wattpad was developed by Allen Lau and Ivan Yuen in 2006. Wattpad is also one of the largest media platforms for reading and writing. Wattpad facilitates people to share their novels and who want to read novels. Several researchers conduct research on exploring Wattpad to improve students' abilities, especially writing skills.

This literature review article approaches the topic of information and communications technologies from the perspective of their impact on the language learning process, with particular emphasis on the most appropriate designs of multimodal texts as informed by models of multimodal learning. The first part contextualizes multimodality within the fields of discourse studies, the psychology of learning and CALL; the second, deals with multimodal conceptions of reading and writing by discussing hypertextuality and literacy. A final section outlines the possible implications of multimodal learning models for foreign language teaching and learning.

Palabras clave: Aprendizaje multimodal, modelos sociosemióticos, multimedia, comprensión multimodal, adquisición de una segunda lengua. a La elaboración de este artículo ha sido posible con financiamiento del Proyecto DICYT N° 030951FF, USACH.

Evaluación de tipos y combinaciones de presentaciones multimodales en la retención y la transferencia de vocabulario concreto en el aprendizaje de EFL Abstract Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

Kim, SoHee. (2019). The effects of using images and video clips for vocabulary learning: Single words vs. phrasal verbs. STEM Journal, 20(2), 19-42. This study investigated how Korean college EFL learners' interactions with different types of media modes-images with texts and video clips with texts-affected vocabulary learning. Consistent with Paivio's dual coding theory, this research tries to examine different visual input; thus, sound was excluded on both modes. Participants studied the given words with definitions and example sentences in English. Participants then took a pre-test, an immediate post-test, and a delayed post-test to assess their vocabulary learning for 30 terms: 20 single words and 10 phrasal verbs. To understand participants' attitudes towards the two modes, one online survey was also conducted. The results show that images with texts provides better outcomes than video clips with texts, especially for single words in the immediate post-test, while video clips with texts can help learners retain phrasal verbs longer. Participants responded that using images with texts provided a strong association with the words while using video clips with texts could motivate them to learn vocabulary in real-life settings. Some also believed that English explanations and examples were useful when guessing the meaning of words in context. This study also suggests that using media can foster learner autonomy in vocabulary learning.

This study aimed to determine the in-school and out-of-school reading and writing experiences of middle school students. This study used phenomenology from qualitative research methods. All 12 participants (8 female, 4 male) were middle school grade 6 students. Various data collection techniques were employed: a survey and a semi-structured interview about the use of Wattpad, a semi-structured interview about the Turkish language subject, a semi-structured interview about the relationship between in-school and out-of-school reading and writing experiences, a diary of the students' Wattpad reading list, a diary of the students' Wattpad writing list, and the texts written by the students on Wattpad. Data collection lasted 16 weeks. The data collected from the survey questionnaire were analyzed descriptively. The data from the semi-structured interviews and diaries were analyzed using content analysis. The results indicated that most of the participants had a negative perception of in-school reading. Similar conclusions were drawn from the results in-school writing because most of the participants felt restricted and under pressure during the in-school writing process. Wattpad was considered a venue where participants can express themselves comfortably. The participants shared positive experiences characterized by willingness, diversity, meaningfulness, and entertainment. By contrast, in-school reading evoked negative experiences, such as feeling obliged, bored, and restricted, and perceiving the activity as meaningless. Out-of-school writing experiences were determined to promote internal motivation, to develop self-expression of one's anxiety and enjoyment alike, and to develop digital writing skills in out-of-school writing experiences. Overall, no relationship was found between in-school and out-of-school reading and writing experiences.

Evaluación de tipos y combinaciones de presentaciones multimodales en la retención y la transferencia de vocabulario concreto en el aprendizaje de EFL Abstract Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

Evaluación de tipos y combinaciones de presentaciones multimodales en la retención y la transferencia de vocabulario concreto en el aprendizaje de EFL Abstract Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

Palabras clave: Aprendizaje multimodal, modelos sociosemióticos, multimedia, comprensión multimodal, adquisición de una segunda lengua. a La elaboración de este artículo ha sido posible con financiamiento del Proyecto DICYT N° 030951FF, USACH.

Palabras clave: Aprendizaje multimodal, modelos sociosemióticos, multimedia, comprensión multimodal, adquisición de una segunda lengua. a La elaboración de este artículo ha sido posible con financiamiento del Proyecto DICYT N° 030951FF, USACH.

In the current teaching methodology, computer-assisted language instruction is considered to be an important tool that helps extend education beyond the classroom and facilitates both interactive and individualised learning. This paper is concerned with the use of on-line activities in teaching a foreign language vocabulary. Vocabulary learning includes complex processes such as the acquisition of words' pronunciations, meanings, as well as their stylistic, morphological and syntactic properties. This paper demonstrates how on-line activities not only help students to memorise new words but also motivate them to practice foreign language vocabulary. These activities allow students to choose exercises with different levels of difficulty and, therefore, adjust the material according to their needs. They also bring together various multimedia elements essential for vocabulary learning, such as sound, images and text.

Many Japanese high school teachers of English do not make use of images in classes, despite
the fact that images have been shown to be more memorable than text. This article covers
research in memory, L1 (first language) reading, and multimedia learning to see understand how
images can aid in the acquisition of written language and reading comprehension ability in
preparation for designing studies on the use of images in EFL (English as a foreign language)
teaching and learning situations.

Evaluación de tipos y combinaciones de presentaciones multimodales en la retención y la transferencia de vocabulario concreto en el aprendizaje de EFL Abstract Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

Although multimodality has been addressed in the educational psychology arena, this study evaluates the effects of presentation modality in the retention and transfer of concrete vocabulary in students of English as a second language. The participants were 104 second year university students belonging to three groups. Group 1 (n=32) was exposed to vocabulary through on screen text and narration (Group TN); group 2 (n=42) was exposed to on screen text, narration and video (Group TNV); and group 3 (n=30) was presented vocabulary via on screen text, narration and still image (Group TNI). All three groups were given a retention test (RT), a transfer test (TT), and a questionnaire to evaluate the type of presentation (TPQ). Results indicate that in the RT there are statistically significant differences among groups (ANOVA), being the TNI group the one that retains more lexical items. In the TT, using the t-student, results show that there are significant differences between groups TNI and TN, being group TNI the one with the highest scores. Results from the TPQ suggest that still images helped more than text and video in vocabulary learning, that actions are better represented through videos than through still images, and that more attention is paid to narration in group TN than in groups TNV and TNI.

ABSTRACT: This article is a partial review of the literature that supports an investigation on the possible
impact of multimodality on foreign language learning, with particular emphasis on text design (KRESS & VAN
LEEUWEN, 2001) and models of multimedia learning (SCHNOTZ, 2005; MAYER, 2001). To start, the article
describes these models and establishes their relation to the field of second language acquisition. To continue, we
discuss some meeting points that may justify our interest in making multimodal leaning and foreign language
education engage in dialog. The article concludes with a brief account of some studies dealing with multimodal
foreign language learning that illustrate such dialogue.
Keywords: Multimodal learning, sociosemiotic models, multimodal second/foreign language teaching-learning

Resumen
La introducción de nuevas tecnologías de información y comunicación (TICs) en el sistema educacional hace evidente la preocupación por los distintos procesos sociocognitivos asociados al manejo y comprensión de los textos llamados multimodales, que tales tecnologías hacen posible. En su desarrollo histórico, la representación del lenguaje como medio de comunicación ha estado centrada en el código escrito. Sin embargo, la llegada de las nuevas TICs ha hecho posible la incorporación de la dimensión visual no plana y la hipertextualidad, lo que ha afectado los contextos de aprendizaje relacionados con el desarrollo de habilidades en un segundo idioma. En este artículo se hace una reseña del estado del arte en torno de la literatura relacionada con el impacto de las tecnologías de información y comunicaciones en el proceso enseñanza-aprendizaje de la (s) lengua (s), con particular acento en los diseños multimodales. En la primera parte se contextualiza la multimodalidad en los modelos sociosemióticos, la psicología cognitiva y CALL y, en la segunda, se abordan los procesos de comprensión y de escritura del discurso escrito desde una perspectiva del aprendizaje multimodal. Finalmente, se plantean algunas posibles implicaciones de los modelos de aprendizaje multimodal para la enseñanza y aprendizaje de lenguas extranjeras.
Palabras clave: Aprendizaje multimodal, modelos sociosemióticos, multimedia, comprensión multimodal, adquisición de una segunda lengua.
a

Abstract
This literature review article approaches the topic of information and communications technologies
from the perspective of their impact on the language learning process, with particular emphasis on the
most appropriate designs of multimodal texts as informed by models of multimodal learning. The first part
contextualizes multimodality within the fields of discourse studies, the psychology of learning and CALL;
the second, deals with multimodal conceptions of reading and writing by discussing hypertextuality and
literacy. A final section outlines the possible implications of multimodal learning models for foreign language
teaching and learning.
Key words: multimodal discourse, literacies, multimodal principles and design