Academia.eduAcademia.edu

Figure 36 – uploaded by Ioannis Deliyannis

See full PDFdownloadDownload figure
Multimedia Approach in Teaching Mathematics — Examples of Interactive Lessons from Mathematical Analysis and Geometry Fig. 10. Homepage of animation about isometric transformations

Figure 10 Multimedia Approach in Teaching Mathematics — Examples of Interactive Lessons from Mathematical Analysis and Geometry Fig. 10. Homepage of animation about isometric transformations

Related Figures (164)

Fig. 1. Drawing made by chronicler Felipe Guaman Poma The use of multimedia applications in schools is quite common. However, according to Alfonso Gutiérrez (2003: 42), these are often attributed educational advantages, which perhaps they do not have and just like that, it is assumed it favors learning. From a creative design perspective, how can we make educational multimedia resources really contribute to an effective teaching-learning process? We believe a key aspect has to do with the possibility of digitally recreating or recycling some traditional teaching materials that have proven to be effective teaching tools. Among them, we find the so-called “Inca abacus” known in Peru and other countries of the region as “yupana’”.
Fig. 1. Drawing made by chronicler Felipe Guaman Poma The use of multimedia applications in schools is quite common. However, according to Alfonso Gutiérrez (2003: 42), these are often attributed educational advantages, which perhaps they do not have and just like that, it is assumed it favors learning. From a creative design perspective, how can we make educational multimedia resources really contribute to an effective teaching-learning process? We believe a key aspect has to do with the possibility of digitally recreating or recycling some traditional teaching materials that have proven to be effective teaching tools. Among them, we find the so-called “Inca abacus” known in Peru and other countries of the region as “yupana’”.
In the lower left corner of a 16th century drawing (figure 1), made by chronicler Felipe suaman Poma, there is a sketch of a yupana next to the quipu; therefore, it is believed they vere complementary calculation tools. The term yupana comes from the Quechua word ‘yupay” that means “to count”. Originally, it consisted of a clay or stone tablet with several ‘columns and boxes with small grooves for placing corn kernels. Researchers have not ‘eached an agreement on how it was used in the olden days; however, for its use in schools, he interpretation made by the engineer William Burns has been chosen, which is based on he decimal system (Bousany, 2008: 18). The tablet has been turned into a horizontal osition, where each column has the value of a multiple of ten and each circle has the value of one multiplied by the value in its column. The upper boxes are used as a memory or for xchange and underneath, there are ten circles, grouped in two, three and five units to acilitate counting (figure 2). There are research and testimonies certifying the educational capacity of the yupana in facilitating the understanding of the positional value of numbers in the decimal system and the execution of immediate arithmetic calculations (Vargas & Ldpez, 2000: 75). It is stated this is an educational and historic tool “[...] that helps students understand certain mathematical algorithms that are many times applied in a mechanical way without knowing the logical part.” (Torres, 2009). Taking into account the importance of mathematics in basic education and the difficulties and resistance generated in its learning, we decided to investigate the feasibility to digitally recycle the yupana with the purpose of strengthening its qualities and make it more attractive for digital natives. For this interdisciplinary work, we counted with the advisory services of a specialist in mathematics education, David Palomino, and of a multimedia application designer, David Chura.
In the lower left corner of a 16th century drawing (figure 1), made by chronicler Felipe suaman Poma, there is a sketch of a yupana next to the quipu; therefore, it is believed they vere complementary calculation tools. The term yupana comes from the Quechua word ‘yupay” that means “to count”. Originally, it consisted of a clay or stone tablet with several ‘columns and boxes with small grooves for placing corn kernels. Researchers have not ‘eached an agreement on how it was used in the olden days; however, for its use in schools, he interpretation made by the engineer William Burns has been chosen, which is based on he decimal system (Bousany, 2008: 18). The tablet has been turned into a horizontal osition, where each column has the value of a multiple of ten and each circle has the value of one multiplied by the value in its column. The upper boxes are used as a memory or for xchange and underneath, there are ten circles, grouped in two, three and five units to acilitate counting (figure 2). There are research and testimonies certifying the educational capacity of the yupana in facilitating the understanding of the positional value of numbers in the decimal system and the execution of immediate arithmetic calculations (Vargas & Ldpez, 2000: 75). It is stated this is an educational and historic tool “[...] that helps students understand certain mathematical algorithms that are many times applied in a mechanical way without knowing the logical part.” (Torres, 2009). Taking into account the importance of mathematics in basic education and the difficulties and resistance generated in its learning, we decided to investigate the feasibility to digitally recycle the yupana with the purpose of strengthening its qualities and make it more attractive for digital natives. For this interdisciplinary work, we counted with the advisory services of a specialist in mathematics education, David Palomino, and of a multimedia application designer, David Chura.
Fig. 3. Use of the traditional yupana in school Educational Digital Recycling: Design of Videogame Based on “Inca Abacus”
Fig. 3. Use of the traditional yupana in school Educational Digital Recycling: Design of Videogame Based on “Inca Abacus”
Educational Digital Recycling: Design of Videogame Based on “Inca Abacus” On the other hand, there are recent projects in Bolivia (Murillo, 2010) and Colombia (I.E. Once de Noviembre, 2010) of digital or virtual yupanas (figure 5). In all of these cases, they are multimedia resources that use digital technology but not the digital “culture”; that is, they are applications that do not take advantage of communication formats or styles that have developed around the new mediums.
Educational Digital Recycling: Design of Videogame Based on “Inca Abacus” On the other hand, there are recent projects in Bolivia (Murillo, 2010) and Colombia (I.E. Once de Noviembre, 2010) of digital or virtual yupanas (figure 5). In all of these cases, they are multimedia resources that use digital technology but not the digital “culture”; that is, they are applications that do not take advantage of communication formats or styles that have developed around the new mediums.
In our case, we decided to creatively transform the yupana into a space videogame, which we called “Yupi 10” (figure 6). We only included four columns in the prototype (up to thousands) and we aligned all the circles vertically with the purpose of simplifying the design and facilitating its use, not only in PCs, but in mobile phones and tablets as well. The game was developed with Adobe Flash. In the case of platforms or devices that do not support Adobe Flash Player, we would have to develop versions of the game with other tools, such as Objective-C, Java or HTML5.
In our case, we decided to creatively transform the yupana into a space videogame, which we called “Yupi 10” (figure 6). We only included four columns in the prototype (up to thousands) and we aligned all the circles vertically with the purpose of simplifying the design and facilitating its use, not only in PCs, but in mobile phones and tablets as well. The game was developed with Adobe Flash. In the case of platforms or devices that do not support Adobe Flash Player, we would have to develop versions of the game with other tools, such as Objective-C, Java or HTML5.
Fig. 6. Main screen of the Yupi 10 videogame
Fig. 6. Main screen of the Yupi 10 videogame
In the case of Yupi 10, the basic rules of the game correspond to the following tasks. First the user has to choose a mission for the ship. In the prototype, we include nine missions (figure 7), with increasing levels of difficulty that correspond to second grade. All missions have an “audio-problem” form and represent acted out situations in which the captain and lieutenant of the ship take part. For example, there is an emergency and the number of oxygen tanks has to be determined through a subtraction or addition operation.
In the case of Yupi 10, the basic rules of the game correspond to the following tasks. First the user has to choose a mission for the ship. In the prototype, we include nine missions (figure 7), with increasing levels of difficulty that correspond to second grade. All missions have an “audio-problem” form and represent acted out situations in which the captain and lieutenant of the ship take part. For example, there is an emergency and the number of oxygen tanks has to be determined through a subtraction or addition operation.
Fig. 8. Color Board with data and option menu
Fig. 8. Color Board with data and option menu
Fig. 10. Subtraction example (23 - 5 = 18) In a subtraction operation, the process is the other way around (figure 10). Let’s suppose we want to subtract 23 - 5. First we represent number twenty-three (figure 10a). Then we subtract number five, but as there are only three circles available (figure 10b), we have to exchange a ten for ten units. For this purpose, we turn off a blue circle and we light up the entire red column with the upper circle (figure 10c). Then we continue turning off the two remaining red circles and at the end we have the subtraction result: eighteen (figure 10d).
Fig. 10. Subtraction example (23 - 5 = 18) In a subtraction operation, the process is the other way around (figure 10). Let’s suppose we want to subtract 23 - 5. First we represent number twenty-three (figure 10a). Then we subtract number five, but as there are only three circles available (figure 10b), we have to exchange a ten for ten units. For this purpose, we turn off a blue circle and we light up the entire red column with the upper circle (figure 10c). Then we continue turning off the two remaining red circles and at the end we have the subtraction result: eighteen (figure 10d).
Fig. 1. Conceptual framework of combination of two instructional design models EC Lab was developed by the researcher by using the combination of two instructiona design models: the Kemp Model and Gerlach and Ely Model. The reasons for using th combination of these two models are that they are classroom-oriented models (Gustafson é Branch, 1997) with their own strengths. The Kemp Model describes elements, not ‘step stage, level or sequential item’ in an instructional design (Kemp et al., 2004). The oval shap of the model indicates the independency of the elements in the model. It is a non-linea model with no starting and ending point. All the processes of designing, developing implementing and evaluating can be done concurrently and continuously. The Gerlach anc Ely Model is suitable for the novice instructional designers who have knowledge anc expertise in a specific context (Qureshi, 2001, 2003, 2004). This model is classroom-orientec and is suitable for teachers at secondary schools and higher education institutions. Th: Gerlach and Ely Model focuses more on the instructional materials and resources withou identifying the instructional problems. Hence, the researchers combined the two models a: the instructional design model to develop the EC Lab. The conceptual framework of th combination of these two models used in the study is presented in Figure 1 below.
Fig. 1. Conceptual framework of combination of two instructional design models EC Lab was developed by the researcher by using the combination of two instructiona design models: the Kemp Model and Gerlach and Ely Model. The reasons for using th combination of these two models are that they are classroom-oriented models (Gustafson é Branch, 1997) with their own strengths. The Kemp Model describes elements, not ‘step stage, level or sequential item’ in an instructional design (Kemp et al., 2004). The oval shap of the model indicates the independency of the elements in the model. It is a non-linea model with no starting and ending point. All the processes of designing, developing implementing and evaluating can be done concurrently and continuously. The Gerlach anc Ely Model is suitable for the novice instructional designers who have knowledge anc expertise in a specific context (Qureshi, 2001, 2003, 2004). This model is classroom-orientec and is suitable for teachers at secondary schools and higher education institutions. Th: Gerlach and Ely Model focuses more on the instructional materials and resources withou identifying the instructional problems. Hence, the researchers combined the two models a: the instructional design model to develop the EC Lab. The conceptual framework of th combination of these two models used in the study is presented in Figure 1 below.
explains new concepts to the students. Professor T speaks slowly in a formal way with little body gestures and facial expressions. On the other hand, Lisa is a fifteen-year old female youth who speaks with an energetic voice. She is a learning companion in the EC Lab. She learns together with the students, gives motivation and encouragement to the students to complete the tasks and exercises in the module. Students are free to choose the PA they want to accompany them in the learning of Electrochemistry after they key in their name in the module. The interface for the choosing of PAs is shown in the Figure 2. The main menu for the EC Lab consists of tutorial, experiment, exercise, quiz, memo and game. There are five sub units in the EC Lab: (1) Electrolytes and Non-Electrolytes, (2) Electrolysis of Molten Compounds, (3) Electrolysis of Aqueous Solutions, (4) Voltaic Cells and (5) Types of Voltaic Cells. All the information delivery for the sub units is presented in the tutorial session. The experiment session consists of five experiments in Electrochemistry. The first experiment about the concept of electrolyte and non-electrolyte is done through the simulation. Another three experiments investigating the factors that determine the ions to be discharged at the electrodes and experiment for simple voltaic cell are hands on investigation. The students were guided by the PAs to carry out the experiments in the chemistry laboratory and they need to apply scientific process skills and manipulative skills during the investigations. There is a session named ‘Micro-World’ (Figure 3) in the module. When students click on magnifying glass button, they will be shown the moving of electrons and ions in microscopic level. This ‘Micro-World’ session describe the detail process happened in the cell during the electrolysis process. After the information delivery process, students will do some exercises to enhance their understanding on the concepts learnt. Quiz will be given at the end of every sub unit. Each quiz is divided into three levels. The first level is to let the students do some reflections on what they have learnt in the sub unit. The students then need to compare their prior idea with the new idea to review whether the conceptual change has occurred. The second level of the quiz consists of five simple multiple-choice questions and some elementary structured questions. Students need to click the answers for the multiple-choice questions and write down the answer for the structure
explains new concepts to the students. Professor T speaks slowly in a formal way with little body gestures and facial expressions. On the other hand, Lisa is a fifteen-year old female youth who speaks with an energetic voice. She is a learning companion in the EC Lab. She learns together with the students, gives motivation and encouragement to the students to complete the tasks and exercises in the module. Students are free to choose the PA they want to accompany them in the learning of Electrochemistry after they key in their name in the module. The interface for the choosing of PAs is shown in the Figure 2. The main menu for the EC Lab consists of tutorial, experiment, exercise, quiz, memo and game. There are five sub units in the EC Lab: (1) Electrolytes and Non-Electrolytes, (2) Electrolysis of Molten Compounds, (3) Electrolysis of Aqueous Solutions, (4) Voltaic Cells and (5) Types of Voltaic Cells. All the information delivery for the sub units is presented in the tutorial session. The experiment session consists of five experiments in Electrochemistry. The first experiment about the concept of electrolyte and non-electrolyte is done through the simulation. Another three experiments investigating the factors that determine the ions to be discharged at the electrodes and experiment for simple voltaic cell are hands on investigation. The students were guided by the PAs to carry out the experiments in the chemistry laboratory and they need to apply scientific process skills and manipulative skills during the investigations. There is a session named ‘Micro-World’ (Figure 3) in the module. When students click on magnifying glass button, they will be shown the moving of electrons and ions in microscopic level. This ‘Micro-World’ session describe the detail process happened in the cell during the electrolysis process. After the information delivery process, students will do some exercises to enhance their understanding on the concepts learnt. Quiz will be given at the end of every sub unit. Each quiz is divided into three levels. The first level is to let the students do some reflections on what they have learnt in the sub unit. The students then need to compare their prior idea with the new idea to review whether the conceptual change has occurred. The second level of the quiz consists of five simple multiple-choice questions and some elementary structured questions. Students need to click the answers for the multiple-choice questions and write down the answer for the structure
Fig. 5. Pictures shown in Think about it session in Sub Unit 4 The complete flow of each sub unit follows the five phases in the learning process created b Needham (1987). The five phases are orientation, elicitation of ideas, restructuring of ideas pplication of ideas and review. In the EC Lab, the Think about it session (Figure 5) is th orientation phase. The students will be shown some pictures that are familiar to them. Thos dictures are related to the concepts to be learnt in every sub unit. Then, in the Do you sti remember session, the students will be reminded of some concepts that they have learn efore. Those concepts are related to the new concepts to be learnt in the sub unit. Next, it he Give me your ideas session (Figure 6), the students are given the chance to give their idea ‘egarding some activities that are related to the concepts to be learnt. Students need to type lick or drag the pictures to give their ideas regarding the concepts being tested. Student ‘an then check their answers to evaluate themselves based on the feedback given by th PAs. Then, in the Are you sure session, the students need to give some ideas, make som suesses or predictions on some outcomes of the situations. In order to examine their ideas suesses and predictions, the students need to carry out some investigations in Let’s do it 0 watch related videos in Show time sessions. In these two sessions, the students will b -xposed to the conflict situations if their ideas, guesses or predictions are different fron what is being shown in the experiments or videos. Hence, conceptual change should happer rere and the students need to modify, extend or replace their existing ideas. Then ‘einforcement of the constructed ideas will be done in the Practice makes perfect session. Th students will apply the concepts learnt in new situations and examples. Students need to di exercises related to the concepts learnt in each sub unit. Lastly, Before and after session i created to enable the students to reflect upon the extent to which their ideas have changed [The students need to answer certain activity questions again and compare their prio answers to the new answers. Testing yourself and Challenge yourself sessions contain multiple choice questions, structured questions and essay questions to let the students evaluat themselves on the concepts learnt.
Fig. 5. Pictures shown in Think about it session in Sub Unit 4 The complete flow of each sub unit follows the five phases in the learning process created b Needham (1987). The five phases are orientation, elicitation of ideas, restructuring of ideas pplication of ideas and review. In the EC Lab, the Think about it session (Figure 5) is th orientation phase. The students will be shown some pictures that are familiar to them. Thos dictures are related to the concepts to be learnt in every sub unit. Then, in the Do you sti remember session, the students will be reminded of some concepts that they have learn efore. Those concepts are related to the new concepts to be learnt in the sub unit. Next, it he Give me your ideas session (Figure 6), the students are given the chance to give their idea ‘egarding some activities that are related to the concepts to be learnt. Students need to type lick or drag the pictures to give their ideas regarding the concepts being tested. Student ‘an then check their answers to evaluate themselves based on the feedback given by th PAs. Then, in the Are you sure session, the students need to give some ideas, make som suesses or predictions on some outcomes of the situations. In order to examine their ideas suesses and predictions, the students need to carry out some investigations in Let’s do it 0 watch related videos in Show time sessions. In these two sessions, the students will b -xposed to the conflict situations if their ideas, guesses or predictions are different fron what is being shown in the experiments or videos. Hence, conceptual change should happer rere and the students need to modify, extend or replace their existing ideas. Then ‘einforcement of the constructed ideas will be done in the Practice makes perfect session. Th students will apply the concepts learnt in new situations and examples. Students need to di exercises related to the concepts learnt in each sub unit. Lastly, Before and after session i created to enable the students to reflect upon the extent to which their ideas have changed [The students need to answer certain activity questions again and compare their prio answers to the new answers. Testing yourself and Challenge yourself sessions contain multiple choice questions, structured questions and essay questions to let the students evaluat themselves on the concepts learnt.
Fig. 6. Students need to drag the answers to the correct spaces
Fig. 6. Students need to drag the answers to the correct spaces
Interactive Multimedia Module with Pedagogical Agent in Electrochemistry EC Lab. As a result, the third, fourth and fifth meetings were conducted at the chemistry laboratory. Students need to carry out the experiments to investigate the three factors that determine the ions to be discharged at the electrodes. Students learnt sub unit four in the sixth meeting. There was an experiment regarding simple voltaic cell in the unit. Hence, students conducted the experiment in the chemistry laboratory. For the seventh meeting, students explored sub unit five in EC Lab about types of voltaic cell. Students in the control group were given notes and lecture for the same sub unit.
Interactive Multimedia Module with Pedagogical Agent in Electrochemistry EC Lab. As a result, the third, fourth and fifth meetings were conducted at the chemistry laboratory. Students need to carry out the experiments to investigate the three factors that determine the ions to be discharged at the electrodes. Students learnt sub unit four in the sixth meeting. There was an experiment regarding simple voltaic cell in the unit. Hence, students conducted the experiment in the chemistry laboratory. For the seventh meeting, students explored sub unit five in EC Lab about types of voltaic cell. Students in the control group were given notes and lecture for the same sub unit.
Table 2. t-test table for specific entry test between control group and treatment group Table 2 shows the t-test table to compare the specific entry test result for treatment anc control group. There was no significant difference in scores for treatment (M = 61.25, SD = 23.40) and control group [M = 58.57, SD = 23.82; t(125) = 0.64, p = 0.05]. The magnitude o: the differences in the means was very small (eta squared = 0.003). These show that student: in both groups have similar level of prior knowledge before they learn Electrochemistry The students’ results ranged from 10% to 100%. The results showed that the students were still weak in the concept of proton number (Item 7, only 39.4% of the students answerec correctly). Students who scored less than 50% (n = 39) of the specific entry test were giver remedial help before they started with the treatment sessions. They were given some revision notes for Chapter Two: The Structure of Atom and Chapter Three: Chemica Formulae. Students were reminded to do revisions before they started the lesson fot Electrochemistry.
Table 2. t-test table for specific entry test between control group and treatment group Table 2 shows the t-test table to compare the specific entry test result for treatment anc control group. There was no significant difference in scores for treatment (M = 61.25, SD = 23.40) and control group [M = 58.57, SD = 23.82; t(125) = 0.64, p = 0.05]. The magnitude o: the differences in the means was very small (eta squared = 0.003). These show that student: in both groups have similar level of prior knowledge before they learn Electrochemistry The students’ results ranged from 10% to 100%. The results showed that the students were still weak in the concept of proton number (Item 7, only 39.4% of the students answerec correctly). Students who scored less than 50% (n = 39) of the specific entry test were giver remedial help before they started with the treatment sessions. They were given some revision notes for Chapter Two: The Structure of Atom and Chapter Three: Chemica Formulae. Students were reminded to do revisions before they started the lesson fot Electrochemistry.
Table 3. t-test table for post-test between control group and treatment group in test scores from pretest (M = 4.46, SD = 3.52) to post-test [M = 35.9, SD = 18.44, t(62) = 12.57, p < 0.05] for the control group. On the other hand, students’ achievement in the treatment group also showed statistically significant increase in test scores from pretest (M = 7.11, SD = 4.00) to post-test [M = 46.01, SD = 29.94, t(63) = 11.03, p < 0.05]. The eta squared statistic (.50) for both groups indicated a large effect size. Although the overall results for the post-test were better compared to the pretest for both groups, 52.4% and 43.75% of them from control group and treatment group respectively still failed the post-test. The results for the post-test ranged from 4.26% to 61.70% for the control group and 2.13% to 93.62% for the treatment group. Both groups showed significant increase results for the achievement test from pretest to post-test indicating that students’ knowledge in Electrochemistry was increased after they gone through the interventions. However, the results for the post-test for treatment group were higher than control group. Table 3 shows the mean scores for post- test between control group and treatment group. An independent-samples t-test was conducted to compare the post-test result for treatment and control group. There was a statistically significant difference in scores for treatment (M = 46.01, SD = 29.94) and control group [M = 35.90, SD = 18.44; t(105.06) = 2.30, p = 0.05]. The magnitude of the differences in the means was small (eta squared = 0.04). Questions in the test consisted of items testing Electrochemistry concepts in macroscopic, microscopic and symbolic levels. Students need to understand the whole process happens during the electrolysis at both electrodes. Comparatively, students from treatment groups were more able to give reasons and explanations to their answers especially in microscopic level. For instance, when copper (II) nitrate solution was changed to concentrated copper (II) chloride solution in electrolytic cell (Item 11), some students from both groups were able to give the correct observations at both electrodes at macroscopic level. However, students from treatment group (e.g. ET 22 & ET33) can explain the reasons to the observations in microscopic level describing the movements of ions and processes happened at both electrodes. On the other hand, students from control group (e.g. KL29) tended to give conclusion as reason for the observation given.
Table 3. t-test table for post-test between control group and treatment group in test scores from pretest (M = 4.46, SD = 3.52) to post-test [M = 35.9, SD = 18.44, t(62) = 12.57, p < 0.05] for the control group. On the other hand, students’ achievement in the treatment group also showed statistically significant increase in test scores from pretest (M = 7.11, SD = 4.00) to post-test [M = 46.01, SD = 29.94, t(63) = 11.03, p < 0.05]. The eta squared statistic (.50) for both groups indicated a large effect size. Although the overall results for the post-test were better compared to the pretest for both groups, 52.4% and 43.75% of them from control group and treatment group respectively still failed the post-test. The results for the post-test ranged from 4.26% to 61.70% for the control group and 2.13% to 93.62% for the treatment group. Both groups showed significant increase results for the achievement test from pretest to post-test indicating that students’ knowledge in Electrochemistry was increased after they gone through the interventions. However, the results for the post-test for treatment group were higher than control group. Table 3 shows the mean scores for post- test between control group and treatment group. An independent-samples t-test was conducted to compare the post-test result for treatment and control group. There was a statistically significant difference in scores for treatment (M = 46.01, SD = 29.94) and control group [M = 35.90, SD = 18.44; t(105.06) = 2.30, p = 0.05]. The magnitude of the differences in the means was small (eta squared = 0.04). Questions in the test consisted of items testing Electrochemistry concepts in macroscopic, microscopic and symbolic levels. Students need to understand the whole process happens during the electrolysis at both electrodes. Comparatively, students from treatment groups were more able to give reasons and explanations to their answers especially in microscopic level. For instance, when copper (II) nitrate solution was changed to concentrated copper (II) chloride solution in electrolytic cell (Item 11), some students from both groups were able to give the correct observations at both electrodes at macroscopic level. However, students from treatment group (e.g. ET 22 & ET33) can explain the reasons to the observations in microscopic level describing the movements of ions and processes happened at both electrodes. On the other hand, students from control group (e.g. KL29) tended to give conclusion as reason for the observation given.
The motivation questionnaire was used to assess the students’ goals and value beliefs for chemistry (especially Electrochemistry), their beliefs about their ability to succeed in the subject and their anxiety toward the test and examination on Electrochemistry. Both groups answer the similar motivation questionnaire before and after the interventions. The paired sample t-test was unable to show any significant difference between pre-motivation and post-motivation mean scores for the both groups. Mean score for control group decreased from pre motivation (M = 3.64, SD = 0.48) to post motivation [M = 3.59, SD = 0.38; t(62) = - 0.98, p < 0.05]. On the other hand, mean score for treatment group increased from pre- questionnaire (M = 3.64, SD = 0.39) to post-questionnaire [M = 3.68, SD = 0.39, t(63) = 1.10, p < 0.05]. The eta squared statistic (.50) indicated a small effect size for the both groups. Although the post-motivation mean score for treatment group is slightly higher than control group, but the t-test was unable to show significant difference between the groups. ndependent samples t-test result in Table 4 above show that there was no significant lifference in post- motivation mean scores for treatment (M = 3.68, SD = 0.39) and control roup [M = 3.59, SD = 0.38; t(125) = 1.43, p = 0.05]. The magnitude of the differences in the neans was small (eta squared = 0.02). Items in the motivation questionnaire consists of six ubscales, namely intrinsic goal orientation, extrinsic goal orientation, task value, control of earning belief, self-efficacy for learning and performance and test anxiety. Students showed he highest mean score in extrinsic goal orientation (treatment group: M = 4.16, SD = 0.61; ontrol group: M = 4.04, SD = 0.63) among all the subscales, indicating that they were trying o show to others that they can perform well in Chemistry. They expected to get reward and raise from the parents and teachers if they got good grades in the subject. The examination riented education system in the country (Anthony, 2006; Keeman, 2007) causes the students o learn to get good grades in the examinations. For instance item six ‘Getting a good grade n this class is the most satisfying thing for me right now’ had a high mean score of 4.31 (SD - 0.72) with majority of them agree (39.4%) and strongly agree (45.7%) with the statement. esults also showed that students were more extrinsically motivated than intrinsically notivated (Chang, 2005). Students were more concerned with the rewards and grades rather han enjoy and learn the Electrochemistry. ae eS ON ly Le Ny 2 ee ee Ck ee: (ann a ene a a ce f Ee f RN: ey Me Me ee See
The motivation questionnaire was used to assess the students’ goals and value beliefs for chemistry (especially Electrochemistry), their beliefs about their ability to succeed in the subject and their anxiety toward the test and examination on Electrochemistry. Both groups answer the similar motivation questionnaire before and after the interventions. The paired sample t-test was unable to show any significant difference between pre-motivation and post-motivation mean scores for the both groups. Mean score for control group decreased from pre motivation (M = 3.64, SD = 0.48) to post motivation [M = 3.59, SD = 0.38; t(62) = - 0.98, p < 0.05]. On the other hand, mean score for treatment group increased from pre- questionnaire (M = 3.64, SD = 0.39) to post-questionnaire [M = 3.68, SD = 0.39, t(63) = 1.10, p < 0.05]. The eta squared statistic (.50) indicated a small effect size for the both groups. Although the post-motivation mean score for treatment group is slightly higher than control group, but the t-test was unable to show significant difference between the groups. ndependent samples t-test result in Table 4 above show that there was no significant lifference in post- motivation mean scores for treatment (M = 3.68, SD = 0.39) and control roup [M = 3.59, SD = 0.38; t(125) = 1.43, p = 0.05]. The magnitude of the differences in the neans was small (eta squared = 0.02). Items in the motivation questionnaire consists of six ubscales, namely intrinsic goal orientation, extrinsic goal orientation, task value, control of earning belief, self-efficacy for learning and performance and test anxiety. Students showed he highest mean score in extrinsic goal orientation (treatment group: M = 4.16, SD = 0.61; ontrol group: M = 4.04, SD = 0.63) among all the subscales, indicating that they were trying o show to others that they can perform well in Chemistry. They expected to get reward and raise from the parents and teachers if they got good grades in the subject. The examination riented education system in the country (Anthony, 2006; Keeman, 2007) causes the students o learn to get good grades in the examinations. For instance item six ‘Getting a good grade n this class is the most satisfying thing for me right now’ had a high mean score of 4.31 (SD - 0.72) with majority of them agree (39.4%) and strongly agree (45.7%) with the statement. esults also showed that students were more extrinsically motivated than intrinsically notivated (Chang, 2005). Students were more concerned with the rewards and grades rather han enjoy and learn the Electrochemistry. ae eS ON ly Le Ny 2 ee ee Ck ee: (ann a ene a a ce f Ee f RN: ey Me Me ee See
An online questionnaire was used for the data collection. After the participants had watched the videotaped classroom session, they were asked to make a diagnosis of the degree of comprehension of different students and to explain what information or knowledge led to this diagnosis. The written responses of the participants regarding the source of their information and knowledge were the textual basis of a qualitative content analysis. The setting is presented in Fig. 1.
An online questionnaire was used for the data collection. After the participants had watched the videotaped classroom session, they were asked to make a diagnosis of the degree of comprehension of different students and to explain what information or knowledge led to this diagnosis. The written responses of the participants regarding the source of their information and knowledge were the textual basis of a qualitative content analysis. The setting is presented in Fig. 1.
Fig. 3. Situation-dependent information - non-observable 3 In each scene presented (altogether four scenes), the participants were requested to make a diagnosis of at least one student’s comprehension after three minutes (a maximum of three diagnostic assessments were possible). Afterwards, the participants were given the opportunity to watch the whole scene again and make another diagnostic assessment of one to three students. The total duration of the different scenes ranged from 4.02 to 10.30 minutes. For reasons of comparability, the results presented in this chapter refer exclusively to the first 3 minutes of each scene.
Fig. 3. Situation-dependent information - non-observable 3 In each scene presented (altogether four scenes), the participants were requested to make a diagnosis of at least one student’s comprehension after three minutes (a maximum of three diagnostic assessments were possible). Afterwards, the participants were given the opportunity to watch the whole scene again and make another diagnostic assessment of one to three students. The total duration of the different scenes ranged from 4.02 to 10.30 minutes. For reasons of comparability, the results presented in this chapter refer exclusively to the first 3 minutes of each scene.
Fig. 2. Situation-dependent information - observable
Fig. 2. Situation-dependent information - observable
Table 2. Descriptive statistics (N = 36) To determine if the participants used some of the categories of information and knowledge sources (situation-dependent information - observable, situation-dependent information - non-observable, class-specific information, professional and experiential knowledge) more frequently than others, the data was aggregated. Therefore, the diagnoses of the participants for all four scenes were put together in a way that summarizes the explanatory statements. If a person mentioned the category “professional and experiential knowledge” in three of the four scenes presented, the value 3 was assigned to this person. If a person mentioned this category in only one scene, the value 1 was assigned to this person. In this manner, an ordinal variable was created. With this variable, it was possible to use Friedman’s test to examine if a category ranked significantly higher or lower than the others. In fact, the results showed that to be the case (y2 (3, N=36) = 89.45, p=.000).
Table 2. Descriptive statistics (N = 36) To determine if the participants used some of the categories of information and knowledge sources (situation-dependent information - observable, situation-dependent information - non-observable, class-specific information, professional and experiential knowledge) more frequently than others, the data was aggregated. Therefore, the diagnoses of the participants for all four scenes were put together in a way that summarizes the explanatory statements. If a person mentioned the category “professional and experiential knowledge” in three of the four scenes presented, the value 3 was assigned to this person. If a person mentioned this category in only one scene, the value 1 was assigned to this person. In this manner, an ordinal variable was created. With this variable, it was possible to use Friedman’s test to examine if a category ranked significantly higher or lower than the others. In fact, the results showed that to be the case (y2 (3, N=36) = 89.45, p=.000).
Fig. 5. Professional and experiential knowledge
Fig. 5. Professional and experiential knowledge
Thereafter, pairwise comparisons were used to identify which categories were used more often. The Bonferroni adjustment was used to calculate the levels of significance (a = 0.01/6 = 0.002; a = 0.05/6 = 0.008). Statistically, all the differences were highly significant except for the difference between the situation-dependent information - non-observable and professional experiential knowledge categories. The results are presented in tables 2 and 3. According to these results, the category “situation-dependent information - observable” was used most frequently, followed by the category “situation-dependent information - non observable.” The difference between the latter category and the category “professional and experiential knowledge” was not statistically significant. As expected, the category “class- specific information was used the least.
Thereafter, pairwise comparisons were used to identify which categories were used more often. The Bonferroni adjustment was used to calculate the levels of significance (a = 0.01/6 = 0.002; a = 0.05/6 = 0.008). Statistically, all the differences were highly significant except for the difference between the situation-dependent information - non-observable and professional experiential knowledge categories. The results are presented in tables 2 and 3. According to these results, the category “situation-dependent information - observable” was used most frequently, followed by the category “situation-dependent information - non observable.” The difference between the latter category and the category “professional and experiential knowledge” was not statistically significant. As expected, the category “class- specific information was used the least.
Chi? tests: Information about the class given vs. not given (N = 36)
Chi? tests: Information about the class given vs. not given (N = 36)
Fostering the Diagnostic Competence of Teachers with Multimedia Training — A Promising Approach
Fostering the Diagnostic Competence of Teachers with Multimedia Training — A Promising Approach
Wilcoxon tests: School subjects
Wilcoxon tests: School subjects
Wilcoxon tests: Situations
Wilcoxon tests: Situations
Perhaps more importantly, incentives provide an important impetus for students to actively use certain technologies, and our experience with blogging clearly shows the impact o: incentives. For both Purdue courses and the Spring 2011 UVa course in Table 3, a small portior of the final course grade (3%) was tied to blog participation and asynchronous collaboratior via commenting in threaded discussions on the course blog. The Spring 2009 UVa course usec no incentives for blog participation. We discuss the role of incentives more in Section 6.2.2, bu both our usage data, as well as student interview data detailed in Section 6.1.3, suggest that < course-grade-related incentive is an important motivator of early and active blog participatior by students. When students are sufficiently motivated, fruitful discussions take place on the blog (Figure 1).
Perhaps more importantly, incentives provide an important impetus for students to actively use certain technologies, and our experience with blogging clearly shows the impact o: incentives. For both Purdue courses and the Spring 2011 UVa course in Table 3, a small portior of the final course grade (3%) was tied to blog participation and asynchronous collaboratior via commenting in threaded discussions on the course blog. The Spring 2009 UVa course usec no incentives for blog participation. We discuss the role of incentives more in Section 6.2.2, bu both our usage data, as well as student interview data detailed in Section 6.1.3, suggest that < course-grade-related incentive is an important motivator of early and active blog participatior by students. When students are sufficiently motivated, fruitful discussions take place on the blog (Figure 1).
Table 4. Summary of HigherEd 2.0 approaches within the diffusion of innovation “5 characteristics” framework (Rogers, 2003). [f=high value, |=low value, — = neutral]
Table 4. Summary of HigherEd 2.0 approaches within the diffusion of innovation “5 characteristics” framework (Rogers, 2003). [f=high value, |=low value, — = neutral]
The next step is to divide the interval [0, a] of the Ox axis in n equal parts of length a/n, where n is a natural number, which should be shown to the students by animation (step-by- step), as shown in Figure 2a. Within each of these intervals, we construct two rectangles: If we mark the zone under the parabola - limited by sides AB and BC of the rectangle and the arc AC - with S, and the other one with P, our next task is to prove the following equation:
The next step is to divide the interval [0, a] of the Ox axis in n equal parts of length a/n, where n is a natural number, which should be shown to the students by animation (step-by- step), as shown in Figure 2a. Within each of these intervals, we construct two rectangles: If we mark the zone under the parabola - limited by sides AB and BC of the rectangle and the arc AC - with S, and the other one with P, our next task is to prove the following equation:
It is obvious that: This was the solution of the given problem via numerical method, but we can offer much more by using the multimedia lessons. In animation shown on Figure 3, students can clearly see that with increasing of n, i.e. the number of circumscribed and inscribed rectangles, these areas will get closer and closer, until they, according to our intuition and visual perception, both become equal to the area S.
It is obvious that: This was the solution of the given problem via numerical method, but we can offer much more by using the multimedia lessons. In animation shown on Figure 3, students can clearly see that with increasing of n, i.e. the number of circumscribed and inscribed rectangles, these areas will get closer and closer, until they, according to our intuition and visual perception, both become equal to the area S.
Fig. 4. The first part of solution of the quadrature of the parabola problem taken from the multimedia lesson (step-by-step)
Fig. 4. The first part of solution of the quadrature of the parabola problem taken from the multimedia lesson (step-by-step)
Fig. 5. The second part of solution of the quadrature of the parabola problem taken from the multimedia lesson (step-by-step) If the value of d is small enough natural number, it means that sub-intervals are smaller and the division I is ‘fine’. If we introduce new breaking points, d gets smaller and smaller so the division gets finer. Consequently - and according to our intuition and multimedia presentation - figure S will get more and more similar to the curvilinear trapezium, so we can conclude that following definition of the area of the curvilinear trapezium F is valid:
Fig. 5. The second part of solution of the quadrature of the parabola problem taken from the multimedia lesson (step-by-step) If the value of d is small enough natural number, it means that sub-intervals are smaller and the division I is ‘fine’. If we introduce new breaking points, d gets smaller and smaller so the division gets finer. Consequently - and according to our intuition and multimedia presentation - figure S will get more and more similar to the curvilinear trapezium, so we can conclude that following definition of the area of the curvilinear trapezium F is valid:
Fig. 7. Definition 2: Area of the curvilinear trapezium F - visualization method
Fig. 7. Definition 2: Area of the curvilinear trapezium F - visualization method
Fig. 8. Animation parts which represents the graphs of the given task and solution in determining the area of a plane figure In a similar way as shown for determining the area of a given figure, we used multimedia animations to explain application of the determined integrals for calculus of volumes of solids, as well as volumes of solids of revolution obtained by revolving a plane figure around Ox or Oy axis.
Fig. 8. Animation parts which represents the graphs of the given task and solution in determining the area of a plane figure In a similar way as shown for determining the area of a given figure, we used multimedia animations to explain application of the determined integrals for calculus of volumes of solids, as well as volumes of solids of revolution obtained by revolving a plane figure around Ox or Oy axis.
Fig. 14. Task: How many axes of symmetry have these flags? One of the examples is presented in Figure 14. Its purpose was to use an interesting example from the everyday life in order to enable students to resume their knowledge. Chapter ‘Problems’ include variety of different interesting tasks, ranged from easier to more complex ones. All tasks are solved, majority with complete solutions and some with instruction how to solve them. The main idea was to enable student to get to the right solution individually, before he or she see it on the screen. Animations do not offer complete solution instantly, but gradually, step-by-step. Some of tasks are typical ones, as in traditional classes, but there are also non-standard tasks taken from the mathematics competitions.
Fig. 14. Task: How many axes of symmetry have these flags? One of the examples is presented in Figure 14. Its purpose was to use an interesting example from the everyday life in order to enable students to resume their knowledge. Chapter ‘Problems’ include variety of different interesting tasks, ranged from easier to more complex ones. All tasks are solved, majority with complete solutions and some with instruction how to solve them. The main idea was to enable student to get to the right solution individually, before he or she see it on the screen. Animations do not offer complete solution instantly, but gradually, step-by-step. Some of tasks are typical ones, as in traditional classes, but there are also non-standard tasks taken from the mathematics competitions.
Therefore, ball in point A should be hit in such a way that would send it through points P, M, N and Q, and it will finally hit the ball in point B.
Therefore, ball in point A should be hit in such a way that would send it through points P, M, N and Q, and it will finally hit the ball in point B.
(Multimedia presentation shows drawing of every line and their intersections, ie. above mentioned points)
(Multimedia presentation shows drawing of every line and their intersections, ie. above mentioned points)
If we mark the intersection of lines A"B" and XY as M, the intersection of lines A''B" and UY as N, the intersection of lines A'M and XV as P, and the intersection of lines B'N and UV as Q, it can be noticed that the following angles are equal: APV=A'PV=XPM, PMX=XMA"=NMY, MNY=B"NU=UNQ, and NQU=B'QV=VOB.
If we mark the intersection of lines A"B" and XY as M, the intersection of lines A''B" and UY as N, the intersection of lines A'M and XV as P, and the intersection of lines B'N and UV as Q, it can be noticed that the following angles are equal: APV=A'PV=XPM, PMX=XMA"=NMY, MNY=B"NU=UNQ, and NQU=B'QV=VOB.
Fig. 16. Examples of symmetry in everyday life: (a) This photograph of the Taj Mahal has two axes of symmetry; Beside the vertical one, there is also a horizontal one, along the water line; (b) Famous Leonardo da Vinci's drawing The Vitruvian Man is also called Canon of Proportions or Proportions of Man. It shows the symmetry of the human body
Fig. 16. Examples of symmetry in everyday life: (a) This photograph of the Taj Mahal has two axes of symmetry; Beside the vertical one, there is also a horizontal one, along the water line; (b) Famous Leonardo da Vinci's drawing The Vitruvian Man is also called Canon of Proportions or Proportions of Man. It shows the symmetry of the human body
Fig. 20. Total average test scores for (a) Test 1 (definite integral) and (b) Test 2 (line reflection) for traditional and multimedia groups at the faculty of Architecture At the Faculty of Architecture, average score on Test 1 (definite integrals) in the traditional group was 67.75 with standard deviation 14.51, and in the multimedia group, average score was 83.21 with standard deviation 15.01. Average score on Test 2 (line reflection) in the traditional group was 76.04 with standard deviation 15.25, and in the multimedia group, average score was 87.92 with standard deviation 12.5. (Figure 20)
Fig. 20. Total average test scores for (a) Test 1 (definite integral) and (b) Test 2 (line reflection) for traditional and multimedia groups at the faculty of Architecture At the Faculty of Architecture, average score on Test 1 (definite integrals) in the traditional group was 67.75 with standard deviation 14.51, and in the multimedia group, average score was 83.21 with standard deviation 15.01. Average score on Test 2 (line reflection) in the traditional group was 76.04 with standard deviation 15.25, and in the multimedia group, average score was 87.92 with standard deviation 12.5. (Figure 20)
Test scores by tasks for all groups are given in Figures 22 and 23. When asked whether they prefer classical or multimedia way of learning, 12% (3 students) answered classical and 82% (22 students) answered multimedia at the Faculty of Architecture, while at the Faculty of Civil Construction Management 20% (5 students) answered classical and 80% (20 students) answered multimedia, explaining it with the following reasons:
Test scores by tasks for all groups are given in Figures 22 and 23. When asked whether they prefer classical or multimedia way of learning, 12% (3 students) answered classical and 82% (22 students) answered multimedia at the Faculty of Architecture, while at the Faculty of Civil Construction Management 20% (5 students) answered classical and 80% (20 students) answered multimedia, explaining it with the following reasons:
Multimedia Approach in Teaching Mathematics — Examples of Interactive Lessons from Mathematical Analysis and Geometry
Multimedia Approach in Teaching Mathematics — Examples of Interactive Lessons from Mathematical Analysis and Geometry
Dick, B. (1999) argues: "The methods, data and interpretation are refined in later cycles, continuously based on experience and knowledge achieved in the previous cycle". Several authors consider Kurt Lewin, a social psychologist and educator who developed his work in action research in the United States of America in the 40s, the pioneer in this methodology.
Dick, B. (1999) argues: "The methods, data and interpretation are refined in later cycles, continuously based on experience and knowledge achieved in the previous cycle". Several authors consider Kurt Lewin, a social psychologist and educator who developed his work in action research in the United States of America in the 40s, the pioneer in this methodology.
Fig. 2. Main Page of Community:” estg]_criar_aprender e partilhar”
Fig. 2. Main Page of Community:” estg]_criar_aprender e partilhar”
For the creation of screencasts and vodcasts the analysis was made on three tools: Jing, Camtasia Studio 6.0.3 and Screentoaster. After analysing the characteristics and capabilities of each of the tools Jing was the chosen one for being open source, easy to use, with the ability to capture, edit and share the content.
For the creation of screencasts and vodcasts the analysis was made on three tools: Jing, Camtasia Studio 6.0.3 and Screentoaster. After analysing the characteristics and capabilities of each of the tools Jing was the chosen one for being open source, easy to use, with the ability to capture, edit and share the content.
For the production of audio castings (podcasts) the following tools were analysed and compared: Audacity, Sony Sound Forge and Adobe Audition. The decision went to Audacity. Choosing Audacity was based on its features and advantages over the others, namely: the fact that it is open source, free to use, easy to configure along with the integration of the usual tools of professional software.
For the production of audio castings (podcasts) the following tools were analysed and compared: Audacity, Sony Sound Forge and Adobe Audition. The decision went to Audacity. Choosing Audacity was based on its features and advantages over the others, namely: the fact that it is open source, free to use, easy to configure along with the integration of the usual tools of professional software.
To select a platform for aggregating multimedia content we also produced a table for a comparative analysis which included: Ning, Elgg, Moodle, Facebook and Grouply. The choice was toward the last one: Grouply. The Grouply platform has advantages such as the fact that it is intuitive to use, very functional and allows to set different graphic templates. On the other hand, Ning platform is not free and Moodle is difficult to configure, and has limited support for Web 2.0 features.
To select a platform for aggregating multimedia content we also produced a table for a comparative analysis which included: Ning, Elgg, Moodle, Facebook and Grouply. The choice was toward the last one: Grouply. The Grouply platform has advantages such as the fact that it is intuitive to use, very functional and allows to set different graphic templates. On the other hand, Ning platform is not free and Moodle is difficult to configure, and has limited support for Web 2.0 features.
Graph. 1. Part II - Question E: Using the casting technologies improved my ICT?
Graph. 1. Part II - Question E: Using the casting technologies improved my ICT?
Finally considering the perceived impact of using multimedia/casting technologies in the classed all teachers referred that it had a positive effect on student’s motivation in the classes (Graphic 7).
Finally considering the perceived impact of using multimedia/casting technologies in the classed all teachers referred that it had a positive effect on student’s motivation in the classes (Graphic 7).
One of the weak points of Agamemnon is the need of installing a full application on the visitor’s mobile phone, with all that implies in terms of compatibility, software maintenance and user assistance. For this reason, we implemented and successfully tested an extension to Agamemnon that only requires a MMS enabled camera phone to support users in their visit to a city. More specifically, the user can take a photo at a monument and submit it via a MMS message to a predefined server application, which recognizes the monument and sends back some information about it, as shown in Figure 1. In some sense, the camera is used as a “virtual eye” of the system that tracks what the user is interested in or, in other words, his/her focus of attention. We note that we developed this prototype in 2007, when smartphones were not equipped with any sophisticated sensor. In our prototype, the camera itself is used as a sensor to understand both the exact location and the orientation of the user at the same time; in other words, the camera is used as a GPS receiver and a magnetometer, which was the main novel aspect of our prototype. During a cultural visit to a city people are constantly on the move, going in and out of buildings and stopping to admire the main landmarks. While moving people may see something that captures their attention and want to know immediately what it is. Our prototype Agamemnon in City aims at satisfying this need and in a sense it can be considered as the natural evolution of a traditional audioguide.
One of the weak points of Agamemnon is the need of installing a full application on the visitor’s mobile phone, with all that implies in terms of compatibility, software maintenance and user assistance. For this reason, we implemented and successfully tested an extension to Agamemnon that only requires a MMS enabled camera phone to support users in their visit to a city. More specifically, the user can take a photo at a monument and submit it via a MMS message to a predefined server application, which recognizes the monument and sends back some information about it, as shown in Figure 1. In some sense, the camera is used as a “virtual eye” of the system that tracks what the user is interested in or, in other words, his/her focus of attention. We note that we developed this prototype in 2007, when smartphones were not equipped with any sophisticated sensor. In our prototype, the camera itself is used as a sensor to understand both the exact location and the orientation of the user at the same time; in other words, the camera is used as a GPS receiver and a magnetometer, which was the main novel aspect of our prototype. During a cultural visit to a city people are constantly on the move, going in and out of buildings and stopping to admire the main landmarks. While moving people may see something that captures their attention and want to know immediately what it is. Our prototype Agamemnon in City aims at satisfying this need and in a sense it can be considered as the natural evolution of a traditional audioguide.
Commercial handheld devices provide a Qwerty virtual keyboard, as users are usually acquainted with it (Figure 3). However, Qwerty is meant to be used with two hands rather than a stylus. Since one needs to move the stylus over the keyboard in order to select characters, the distance between keys is a critical issue in a virtual keyboard. We remark that Qwerty has a landscape orientation, which means that the distance between two keys is typically large. Moreover, letters that form frequent digraphs (i.e. pairs of letters) in English, such as “at” and “in” are distant, which intuitively results in a slower typing speed. Although Baber (1996) claims that the Qwerty layout is the result of a random choice, it is common belief that the letters forming frequent digraphs in English have been placed on purpose far from each other to prevent jams in old typewriters, caused by back to back pressures of two close keys. However it is, Qwerty was devised well before the advent of handheld devices, thus it is not optimized at all for a pen-based interaction.
Commercial handheld devices provide a Qwerty virtual keyboard, as users are usually acquainted with it (Figure 3). However, Qwerty is meant to be used with two hands rather than a stylus. Since one needs to move the stylus over the keyboard in order to select characters, the distance between keys is a critical issue in a virtual keyboard. We remark that Qwerty has a landscape orientation, which means that the distance between two keys is typically large. Moreover, letters that form frequent digraphs (i.e. pairs of letters) in English, such as “at” and “in” are distant, which intuitively results in a slower typing speed. Although Baber (1996) claims that the Qwerty layout is the result of a random choice, it is common belief that the letters forming frequent digraphs in English have been placed on purpose far from each other to prevent jams in old typewriters, caused by back to back pressures of two close keys. However it is, Qwerty was devised well before the advent of handheld devices, thus it is not optimized at all for a pen-based interaction.
Fig. 5. The Opti keyboard. one letter to another. Moreover, frequent letters, such as t, a, n, e, 0, r, which have a cumulative frequency of 40% according to the tables of Mayzner & Tresselt (1965), are arranged in the middle of the keyboard, while less frequent letters, such as z and x appear at the edges. As a result, the movements of the stylus are limited in a small area around the centre of the keyboard, and only rarely need to expand to the edges. Finally, two large keys (the one: with no label) are dedicated to the space character; this choice is based on the observatior that the space character is much more frequent than any other; in fact, according to the tables of Mayzner & Tresselt (1965), the space has a frequency of 18%, while “e”, the seconc most frequent character, has “only” a frequency of 10%. As of today, Fitaly features also < technology known as Instant Text, that allow the use of abbreviations to insert words, whict further decreases the writing speed.
Fig. 5. The Opti keyboard. one letter to another. Moreover, frequent letters, such as t, a, n, e, 0, r, which have a cumulative frequency of 40% according to the tables of Mayzner & Tresselt (1965), are arranged in the middle of the keyboard, while less frequent letters, such as z and x appear at the edges. As a result, the movements of the stylus are limited in a small area around the centre of the keyboard, and only rarely need to expand to the edges. Finally, two large keys (the one: with no label) are dedicated to the space character; this choice is based on the observatior that the space character is much more frequent than any other; in fact, according to the tables of Mayzner & Tresselt (1965), the space has a frequency of 18%, while “e”, the seconc most frequent character, has “only” a frequency of 10%. As of today, Fitaly features also < technology known as Instant Text, that allow the use of abbreviations to insert words, whict further decreases the writing speed.
collaborate and adapt their handwriting in order to make the recognition task easier. This is the rationale of shorthand writing, pioneered in the pen computing domain by Goldberg & Richardson (1993), who proposed Unistroke (Figure 6). Unistroke defines a set of symbols, each associated to a character of the alphabet, which are easy to be recognized. To avoid the problem of segmentation, the user is asked to insert each symbol with just one stroke, hence the name Unistroke. Compared to a handwriting recognition system using the regular English alphabet, Unistroke proved to be twice as faster, as pointed out by Goldberg & Richardson (1993) themselves. However, besides the fact that people may not like the idea of lifting the
collaborate and adapt their handwriting in order to make the recognition task easier. This is the rationale of shorthand writing, pioneered in the pen computing domain by Goldberg & Richardson (1993), who proposed Unistroke (Figure 6). Unistroke defines a set of symbols, each associated to a character of the alphabet, which are easy to be recognized. To avoid the problem of segmentation, the user is asked to insert each symbol with just one stroke, hence the name Unistroke. Compared to a handwriting recognition system using the regular English alphabet, Unistroke proved to be twice as faster, as pointed out by Goldberg & Richardson (1993) themselves. However, besides the fact that people may not like the idea of lifting the
Developing Attention-Aware and Context-Aware User Interfaces on Handheld Devices
Developing Attention-Aware and Context-Aware User Interfaces on Handheld Devices
The excellent British Library’s package has been developed over a long period to include: historical information, real documents, music, arts and poetry but had a complex interface. Aimed at school pupils, the package assumed knowledge in many of these specific fields in order to be able to select information. During testing of the package with the planned age group this knowledge was not apparent and they spent a lot of time trying to work out how to find the material. Giving the developers an idea of how complex they found the package, and the types of searches or information the users needed, allowed a total redesign of the interface. Keeping the high quality underlying information base meant that the redesign was specifically targeted at the interface. The redesign made the resource easier to use and with far greater hit rates for the relevant searches.
The excellent British Library’s package has been developed over a long period to include: historical information, real documents, music, arts and poetry but had a complex interface. Aimed at school pupils, the package assumed knowledge in many of these specific fields in order to be able to select information. During testing of the package with the planned age group this knowledge was not apparent and they spent a lot of time trying to work out how to find the material. Giving the developers an idea of how complex they found the package, and the types of searches or information the users needed, allowed a total redesign of the interface. Keeping the high quality underlying information base meant that the redesign was specifically targeted at the interface. The redesign made the resource easier to use and with far greater hit rates for the relevant searches.
Fig. 2. Screen shot from the Open University Carbon Cycle The chapter includes the detail of both the initial design and of how the improvements affected the usage, together with tips for future design decisions. It is intended that some visualisations of how users have navigated through specific packages will be included, as well as visual images of different methods of designing the package to allow easier navigation and use. 8.3 The Homer package
Fig. 2. Screen shot from the Open University Carbon Cycle The chapter includes the detail of both the initial design and of how the improvements affected the usage, together with tips for future design decisions. It is intended that some visualisations of how users have navigated through specific packages will be included, as well as visual images of different methods of designing the package to allow easier navigation and use. 8.3 The Homer package
Fig. 4. Screen shot from the Sonoran Desert [his package was designed with a strong route defined basis. The routes were selected from 1 compass point system. The user could choose to start off North, South, West or East. Towever as most users are very organised into a top to bottom and left to right orientation n terms of movement through material, the most popular initial route was North. Giving deople a choice without being aware of their natural preferences is not helpful, as in this nstance introductory information was given for each of the routes. If everyone using the onackage is going to select a first route, i.e. North then this route and this route only, could rave the introductory information. Giving multiple repetitions of the basic information (or even of similar interfaces) is likely to either annoy the user or to totally put them off using he package further.
Fig. 4. Screen shot from the Sonoran Desert [his package was designed with a strong route defined basis. The routes were selected from 1 compass point system. The user could choose to start off North, South, West or East. Towever as most users are very organised into a top to bottom and left to right orientation n terms of movement through material, the most popular initial route was North. Giving deople a choice without being aware of their natural preferences is not helpful, as in this nstance introductory information was given for each of the routes. If everyone using the onackage is going to select a first route, i.e. North then this route and this route only, could rave the introductory information. Giving multiple repetitions of the basic information (or even of similar interfaces) is likely to either annoy the user or to totally put them off using he package further.
We design and build a system that can enlarge details at a location or show related media data without directing viewer's focus to separated screens or windows. The system has a function of magnifier or scope to pin-point locations with a small display on a surface. Moreover, because there has no successful example of curved screens that can fit on a curved body for clicking and display, we can then wrap a surface with this two dimensional sheet for signal transmission. By placing a small high-resolution LCD on the sheet, we will be able to see position specific information and media detailed in resolution in contrast to a global map laid on the surface. In this chapter, we introduce the design and implementation of a platform that can look into a 2D or even 3D space by touching the surface sheet with a small display. Several tasks will also be examined to evaluate the performance of the platform. Figure 1 shows such an idea to magnify details at designated locations of an atlas.
We design and build a system that can enlarge details at a location or show related media data without directing viewer's focus to separated screens or windows. The system has a function of magnifier or scope to pin-point locations with a small display on a surface. Moreover, because there has no successful example of curved screens that can fit on a curved body for clicking and display, we can then wrap a surface with this two dimensional sheet for signal transmission. By placing a small high-resolution LCD on the sheet, we will be able to see position specific information and media detailed in resolution in contrast to a global map laid on the surface. In this chapter, we introduce the design and implementation of a platform that can look into a 2D or even 3D space by touching the surface sheet with a small display. Several tasks will also be examined to evaluate the performance of the platform. Figure 1 shows such an idea to magnify details at designated locations of an atlas.
Besides the communication, the power can also be supplied through the 2D communicatio1 sheet. A device such as PDA can be put anywhere on the sheet to work without extra wire: and batteries, which forms a very flexible system of distributed displays. The sheet can bi manufactured in a very low cost affordable to be put on table top, walls, and poster boards The connector is an antenna by which an electromagnetic wave is extracted from or insertec into the 2D sheet. The connector is manufactured in a small piece to contact the sheet fo receiving and passing signals. Figure 3 shows a set of connector (each piece is 25x70x3mn and weights 2¢) and a small display (LCD of 800x480 pixels) that can be put on any positio1 of the communication sheet to receive data and power signals.
Besides the communication, the power can also be supplied through the 2D communicatio1 sheet. A device such as PDA can be put anywhere on the sheet to work without extra wire: and batteries, which forms a very flexible system of distributed displays. The sheet can bi manufactured in a very low cost affordable to be put on table top, walls, and poster boards The connector is an antenna by which an electromagnetic wave is extracted from or insertec into the 2D sheet. The connector is manufactured in a small piece to contact the sheet fo receiving and passing signals. Figure 3 shows a set of connector (each piece is 25x70x3mn and weights 2¢) and a small display (LCD of 800x480 pixels) that can be put on any positio1 of the communication sheet to receive data and power signals.
Fig. 2. Two dimensional communication sheet. (Top) 2D sheet construction; (Bottom) The electromagnetic wave seeping out from the 2D sheet. Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 2. Two dimensional communication sheet. (Top) 2D sheet construction; (Bottom) The electromagnetic wave seeping out from the 2D sheet. Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 4. Acquiring the relative position by the arrival time difference between the infrared and ultrasound waves. Through the above Trilateration, the location of the transmitter can be determined.
Fig. 4. Acquiring the relative position by the arrival time difference between the infrared and ultrasound waves. Through the above Trilateration, the location of the transmitter can be determined.
Fig. 6. Obtaining location and orientation of a display device on the 2D sheet. orientation of the display. The temporal output of the vector provides a trace of orientation moving on top of the communication sheet as depicted in Fig. 6. The drawbacks of this approach are the necessity of the additional sensor devices to generate and sense the infrared and ultrasound waves, as well as the influence from surrounding objects and human bodies, because the reflection of beams from other objects makes the signal noisy when reaching the receivers.
Fig. 6. Obtaining location and orientation of a display device on the 2D sheet. orientation of the display. The temporal output of the vector provides a trace of orientation moving on top of the communication sheet as depicted in Fig. 6. The drawbacks of this approach are the necessity of the additional sensor devices to generate and sense the infrared and ultrasound waves, as well as the influence from surrounding objects and human bodies, because the reflection of beams from other objects makes the signal noisy when reaching the receivers.
Fig. 5. The signals from two positioning sets are alternatively sent to the receivers by using the wireless Zigbee switch controller. (a) The positioning set 1 is turned on switched by Zigbee controller, (b) The positioning set 2 is turned on. Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 5. The signals from two positioning sets are alternatively sent to the receivers by using the wireless Zigbee switch controller. (a) The positioning set 1 is turned on switched by Zigbee controller, (b) The positioning set 2 is turned on. Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 7. Positioning of target device using phase difference of electrode array on 2D Communication sheet. To avoid the interference to the other radio communications performed outside of the sheet the second approach sends signal within the communication sheet. To aim at a simpl system, instead of the utilization of the extra infrared and ultrasound waves mentioned it subsection 3.1, we directly employ the microwaves applied in the 2D communication systen for position estimation. We employ a small signal emitter mounted on the scope device tha is called target device, and attach the multiple microwave receiving electrodes on the side of the communication sheet, as shown in Fig. 7. As the microwave runs much faster than th ultrasonic wave, even when the time clock is 3.4GHz achieved by the chip-scale atomic clock in one of DARPA projects in 2007 [15], the precision of the measured distance computed by the arrival time of radio beacon is about 10 cm. Instead of directly measuring the arriva time of microwave, we here examine another approach in which we measure the phas difference arriving at the different abovementioned electrodes, by transmitting the pilo signal from the target device.
Fig. 7. Positioning of target device using phase difference of electrode array on 2D Communication sheet. To avoid the interference to the other radio communications performed outside of the sheet the second approach sends signal within the communication sheet. To aim at a simpl system, instead of the utilization of the extra infrared and ultrasound waves mentioned it subsection 3.1, we directly employ the microwaves applied in the 2D communication systen for position estimation. We employ a small signal emitter mounted on the scope device tha is called target device, and attach the multiple microwave receiving electrodes on the side of the communication sheet, as shown in Fig. 7. As the microwave runs much faster than th ultrasonic wave, even when the time clock is 3.4GHz achieved by the chip-scale atomic clock in one of DARPA projects in 2007 [15], the precision of the measured distance computed by the arrival time of radio beacon is about 10 cm. Instead of directly measuring the arriva time of microwave, we here examine another approach in which we measure the phas difference arriving at the different abovementioned electrodes, by transmitting the pilo signal from the target device.
Fig. 8. Difference of received phase and signal strength in the 2D sheet.
Fig. 8. Difference of received phase and signal strength in the 2D sheet.
Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 13. Digital scope system called “World Corridor”. Based on the estimated positioning data of the display with the reconstructed MIMIO system, the location-specific media details can be magnified and displayed as a digital scope to learning materials, catalogs, tourists guide, X-ray image of building, and various other purposes. For example, as shown in Fig.13, we have developed a digital scope system called
Fig. 13. Digital scope system called “World Corridor”. Based on the estimated positioning data of the display with the reconstructed MIMIO system, the location-specific media details can be magnified and displayed as a digital scope to learning materials, catalogs, tourists guide, X-ray image of building, and various other purposes. For example, as shown in Fig.13, we have developed a digital scope system called
Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Digital Scope on 2D Communication Sheet for Location-Specific Multimedia Service
Fig. 14. Digital scope system for children’s education materials on what it will become when it grows up. "World Corridor.” Along the “World Corridor,” we can enjoy an around-the-world trip with family and friends while staying at home. Depending on the position where the display is placed, the scenery image of the place on the map can be gradually magnified by zooming in; details of living creatures in the sightseeing spots can also be visualized. We have also developed another digital scope system for children’s education materials. Figure 14 shows a picture book of various eggs and species on what it will become when it grows up. For example, as shown in Fig. 14 , when the display is placed in the middle of 2D sheet, the motion pictures are displayed on how the baby chicken were hatched from eggs, grow up and finally develop to adult chicken.
Fig. 14. Digital scope system for children’s education materials on what it will become when it grows up. "World Corridor.” Along the “World Corridor,” we can enjoy an around-the-world trip with family and friends while staying at home. Depending on the position where the display is placed, the scenery image of the place on the map can be gradually magnified by zooming in; details of living creatures in the sightseeing spots can also be visualized. We have also developed another digital scope system for children’s education materials. Figure 14 shows a picture book of various eggs and species on what it will become when it grows up. For example, as shown in Fig. 14 , when the display is placed in the middle of 2D sheet, the motion pictures are displayed on how the baby chicken were hatched from eggs, grow up and finally develop to adult chicken.
As shown in the Structured Analysis and Design Technique (SADT) diagram (Ross, 1977) in Figure 1, the process is performed in two main steps: Domain Engineering (DE) and Application Engineering (AE). The process begins in the DE, where the metamodel to support the modeling of the applications’ interfaces is built from the requirements of the Rich Interfaces Domain. The metamodel is built in such a way to allow the reuse of the Rich Interface Domain knowledge on application projects of several areas, and to provide a useful infrastructure to automate most of interfaces’ code generation. Also in the DE, based on the rich interface components represented in the developed metamodel, the M2C transformations and the dynamic content adapters are built to act as support mechanisms in the development of rich interfaces in the AE step.
As shown in the Structured Analysis and Design Technique (SADT) diagram (Ross, 1977) in Figure 1, the process is performed in two main steps: Domain Engineering (DE) and Application Engineering (AE). The process begins in the DE, where the metamodel to support the modeling of the applications’ interfaces is built from the requirements of the Rich Interfaces Domain. The metamodel is built in such a way to allow the reuse of the Rich Interface Domain knowledge on application projects of several areas, and to provide a useful infrastructure to automate most of interfaces’ code generation. Also in the DE, based on the rich interface components represented in the developed metamodel, the M2C transformations and the dynamic content adapters are built to act as support mechanisms in the development of rich interfaces in the AE step.
Figure 2 shows the SADT diagram detailing the activities defined for the DE in the Model Driven RichUbi. The activities on the left-hand side correspond to the construction of the metamodel to support rich interfaces modeling. The activities on the right-hand side refer to the construction of the M2C transformations for partial generation code (upper activity) and the content adapters for dynamic adaptation of the developed interfaces (bottom activity). Since these latter depend on the metamodel’s implementation as input artifact, the activities for constructing the metamodel must precede all other activities of the Domain Engineering.
Figure 2 shows the SADT diagram detailing the activities defined for the DE in the Model Driven RichUbi. The activities on the left-hand side correspond to the construction of the metamodel to support rich interfaces modeling. The activities on the right-hand side refer to the construction of the M2C transformations for partial generation code (upper activity) and the content adapters for dynamic adaptation of the developed interfaces (bottom activity). Since these latter depend on the metamodel’s implementation as input artifact, the activities for constructing the metamodel must precede all other activities of the Domain Engineering.
Figure 3 shows, for example, an excerpt of a class diagram that specifies some interface components identified from this activity, which range from ordinary form controls (Button, TextField, and Select) to advanced rich interfaces widgets (TabbedPanel, AccordionPanel, MessageDialog and DatePicker). During this modeling task, the Domain Engineer is assisted by the Mutiple-View CASE (MVCASE) (Lucrédio et al., 2003), a Computer-Aided Software Engineering (CASE) tool currently available as an Eclipse workbenché plug-in to support UML modeling.
Figure 3 shows, for example, an excerpt of a class diagram that specifies some interface components identified from this activity, which range from ordinary form controls (Button, TextField, and Select) to advanced rich interfaces widgets (TabbedPanel, AccordionPanel, MessageDialog and DatePicker). During this modeling task, the Domain Engineer is assisted by the Mutiple-View CASE (MVCASE) (Lucrédio et al., 2003), a Computer-Aided Software Engineering (CASE) tool currently available as an Eclipse workbenché plug-in to support UML modeling.
metamodeling constructs from the Eclipse Modeling Framework (EMF)’. As illustrated in the excerpt of the metamodel’s specification in Figure 4, the interface components specified in the class diagram of Figure 3 were mapped into homonymous metaclasses. The id, class and label attributes from the FormControl class have been factored, respectively, into the Identifiab In addition, the FormC leComponent, ClassifiableComponent and Label metaclasses. ontrol class’ event-handling methods formed the EventType meta- enumeration and the Event, EventComponent and Script metaclasses. Metaclasses to address the interface’s data input constraints were also included in the metamodel, such as the ValueConstraint, NumberValueConstraint, RequiredFieldConstraint and De me ES hn URE <. ST Mee nm Men ame ee een
metamodeling constructs from the Eclipse Modeling Framework (EMF)’. As illustrated in the excerpt of the metamodel’s specification in Figure 4, the interface components specified in the class diagram of Figure 3 were mapped into homonymous metaclasses. The id, class and label attributes from the FormControl class have been factored, respectively, into the Identifiab In addition, the FormC leComponent, ClassifiableComponent and Label metaclasses. ontrol class’ event-handling methods formed the EventType meta- enumeration and the Event, EventComponent and Script metaclasses. Metaclasses to address the interface’s data input constraints were also included in the metamodel, such as the ValueConstraint, NumberValueConstraint, RequiredFieldConstraint and De me ES hn URE <. ST Mee nm Men ame ee een
Fig. 6. Template-based code generation for the tabbed panel component
Fig. 6. Template-based code generation for the tabbed panel component
manipulating XML documents that follow the World Wide Web Consortium (W3C)’s DOM recommendation, such as XHTML and HTML ones. It provides functionalities for reading and writing these documents in the Web server's memory by structuring them into a tree of DOM objects (e.g. Node, Document, | Element). This capability makes possible to analyze and to modify dynamically the document's interfaces as necessary. In order to work properly the DOM API requires wel l-formed XML files as input. However, since many documents available on the Web do not satisfy this requirement, the TagSoup!’ parser was used to correct features like missing and mismatching tags in order to get the DOM tree of the corrected document before processing the dynamic interface adaptation. To guide the interface adaptation, the ContentAdapter class consumes contextual information on the current access device profile and adjusts the document's interface according to the device’s peculiarities. As presented in the class diagram in Figure 7, this information is provided by the ContextManager class, which obtains the device profiles from the public XML database called Wireless Universal Resource File (WURFL)" using the services provided by the WURFLAdapter class. The WUREL stores the profiles of thousands!5 of devices from different brands and models and is used by software developers to guide the creation of appropriate solutions for specific devices. Because it is a public database, the WUREL receives updates on a daily basis by developers spread around the world interested in contributing to completeness and correctness of the information contained therein. As it receives continuous updates from the community development itself, the WURFL was adopted as the main context source from which the device profiles are acquired in this work.
manipulating XML documents that follow the World Wide Web Consortium (W3C)’s DOM recommendation, such as XHTML and HTML ones. It provides functionalities for reading and writing these documents in the Web server's memory by structuring them into a tree of DOM objects (e.g. Node, Document, | Element). This capability makes possible to analyze and to modify dynamically the document's interfaces as necessary. In order to work properly the DOM API requires wel l-formed XML files as input. However, since many documents available on the Web do not satisfy this requirement, the TagSoup!’ parser was used to correct features like missing and mismatching tags in order to get the DOM tree of the corrected document before processing the dynamic interface adaptation. To guide the interface adaptation, the ContentAdapter class consumes contextual information on the current access device profile and adjusts the document's interface according to the device’s peculiarities. As presented in the class diagram in Figure 7, this information is provided by the ContextManager class, which obtains the device profiles from the public XML database called Wireless Universal Resource File (WURFL)" using the services provided by the WURFLAdapter class. The WUREL stores the profiles of thousands!5 of devices from different brands and models and is used by software developers to guide the creation of appropriate solutions for specific devices. Because it is a public database, the WUREL receives updates on a daily basis by developers spread around the world interested in contributing to completeness and correctness of the information contained therein. As it receives continuous updates from the community development itself, the WURFL was adopted as the main context source from which the device profiles are acquired in this work.
Building Adaptive Rich Interfaces for Interactive Ubiquitous Applications As shown in the SADT diagram in Figure 10, the activities defined for the AE step in the Model Driven RichUbi cover the Analysis, Design, Implementation and Testing disciplines from the software life-cycle. It extends the conventional software development processes by including the MDD and DSM conceptions, and it is focused on developing rich interfaces for interactive ubiquitous applications through the reuse of the artifacts produced in the process’ DE step. The usage of the Rich Interfaces Domain metamodel in this step facilitates the application’s interface modeling. Besides, the M2C transformations enable automating most of the interface’s codification, which makes faster the Application Engineer’s tasks. In addition, the content adapters provide decoupled functionalities for dynamic interface adaptation, allowing keeping focus on the development of features related to other avplication’s functional and nonfunctional requirements.
Building Adaptive Rich Interfaces for Interactive Ubiquitous Applications As shown in the SADT diagram in Figure 10, the activities defined for the AE step in the Model Driven RichUbi cover the Analysis, Design, Implementation and Testing disciplines from the software life-cycle. It extends the conventional software development processes by including the MDD and DSM conceptions, and it is focused on developing rich interfaces for interactive ubiquitous applications through the reuse of the artifacts produced in the process’ DE step. The usage of the Rich Interfaces Domain metamodel in this step facilitates the application’s interface modeling. Besides, the M2C transformations enable automating most of the interface’s codification, which makes faster the Application Engineer’s tasks. In addition, the content adapters provide decoupled functionalities for dynamic interface adaptation, allowing keeping focus on the development of features related to other avplication’s functional and nonfunctional requirements.
In this activity the application is specified according to its requirements. In the process, this specification can be accomplished by using UML techniques, such as class and use cases diagrams. For instance, Figure 11(a) shows a stretch of the use cases diagram developed by the Application Engineer through the MVCASE tool to specify the WebRES’ requirements. These requirements include the nonfunctional ones, such as user authentication, and also the functional ones, like recovering the patients’ blood pressure records. Figure 11(b) shows a class diagram that specifies some of the domain entities associated with the WebRES. 3.2.2 Design
In this activity the application is specified according to its requirements. In the process, this specification can be accomplished by using UML techniques, such as class and use cases diagrams. For instance, Figure 11(a) shows a stretch of the use cases diagram developed by the Application Engineer through the MVCASE tool to specify the WebRES’ requirements. These requirements include the nonfunctional ones, such as user authentication, and also the functional ones, like recovering the patients’ blood pressure records. Figure 11(b) shows a class diagram that specifies some of the domain entities associated with the WebRES. 3.2.2 Design
Fig. 12. WebRES’ interfaces model instantiated from the Rich Interfaces Domain metamodel For example, Figure 12 shows, on the left-hand side, the WebRES’ interfaces model built through the model editor plug-in integrated into the MVCASE tool. On the right-hand side there is the components diagram illustrating the corresponding instantiation of the metamodel. In the editor the interface models are created in the Eclipse’s default EMF tree view mode. The icons are shown in the model due to a light-weight mechanism for concrete syntax, which enabled associating one representative icon with each interface component defined in the underlying metamodel. As shown in the figure, the AuthenticateUser and RecoverBloodPressureRecords WebRES’s use cases were mapped, respectively, into an authentication form inside a login page, and a search form inside a tabbed panel in a page designed for searching blood pressure records.
Fig. 12. WebRES’ interfaces model instantiated from the Rich Interfaces Domain metamodel For example, Figure 12 shows, on the left-hand side, the WebRES’ interfaces model built through the model editor plug-in integrated into the MVCASE tool. On the right-hand side there is the components diagram illustrating the corresponding instantiation of the metamodel. In the editor the interface models are created in the Eclipse’s default EMF tree view mode. The icons are shown in the model due to a light-weight mechanism for concrete syntax, which enabled associating one representative icon with each interface component defined in the underlying metamodel. As shown in the figure, the AuthenticateUser and RecoverBloodPressureRecords WebRES’s use cases were mapped, respectively, into an authentication form inside a login page, and a search form inside a tabbed panel in a page designed for searching blood pressure records.
Fig. 14. Tests with the WebRES’ interfaces performed on an iPhone and a desktop been delivered. Finally, the tests of the interfaces provide feedback for the previous AE step’s activities. Figure 14 shows the results of the tests with the WebRES’ interfaces performed on an iPhone and a desktop. Figure 14(a) shows the execution of the page for searching blood pressure records on the iPhone emulator, to which the interface version for smartphones adapted in compliance to the iPhone’s characteristics (e.g. screen width) was delivered. Figure 14(b) shows the same page viewed on a personal computer, to which the desktop version has been delivered.
Fig. 14. Tests with the WebRES’ interfaces performed on an iPhone and a desktop been delivered. Finally, the tests of the interfaces provide feedback for the previous AE step’s activities. Figure 14 shows the results of the tests with the WebRES’ interfaces performed on an iPhone and a desktop. Figure 14(a) shows the execution of the page for searching blood pressure records on the iPhone emulator, to which the interface version for smartphones adapted in compliance to the iPhone’s characteristics (e.g. screen width) was delivered. Figure 14(b) shows the same page viewed on a personal computer, to which the desktop version has been delivered.
Table 1. Experiment’s data Building Adaptive Rich Interfaces for Interactive Ubiquitous Applications
Table 1. Experiment’s data Building Adaptive Rich Interfaces for Interactive Ubiquitous Applications
The main characteristic of a QR Code compared with a traditional bar code is that it contains information in both the vertical and horizontal directions, while a bar code contains data ir one direction only. For this reason QR code holds a considerably greater volume o information. In addition it can encode several types of data including symbols, control codes binary data, and multimedia data. The typical barcode holds a maximum of 20 digits, while the maximum data capacities of a QR code are 7,089 characters for numeric data, 4,29¢ characters for alphanumeric data, 2,953 bytes for binary data, and 1,817 characters for Japanese Kanji and Kana data. Fig. 2 shows a QR Code compared with a traditional bar code.
The main characteristic of a QR Code compared with a traditional bar code is that it contains information in both the vertical and horizontal directions, while a bar code contains data ir one direction only. For this reason QR code holds a considerably greater volume o information. In addition it can encode several types of data including symbols, control codes binary data, and multimedia data. The typical barcode holds a maximum of 20 digits, while the maximum data capacities of a QR code are 7,089 characters for numeric data, 4,29¢ characters for alphanumeric data, 2,953 bytes for binary data, and 1,817 characters for Japanese Kanji and Kana data. Fig. 2 shows a QR Code compared with a traditional bar code.
Table 2. Comparison of QR Code readers
Table 2. Comparison of QR Code readers
Figure 5 shows the structure of the Contactless Communication API.
Figure 5 shows the structure of the Contactless Communication API.
As explained before both technologies, RFID/NFC and QR Code, can be considered as AIDC techniques and used with mobile phones. In Table 3 we summarized the characteristics of both technologies and compared them.
As explained before both technologies, RFID/NFC and QR Code, can be considered as AIDC techniques and used with mobile phones. In Table 3 we summarized the characteristics of both technologies and compared them.
Fig. 2. The BluTV Architecture and its relations with Ginga Architecture Fig. 3. Layers of the BluTV Architecture
Fig. 2. The BluTV Architecture and its relations with Ginga Architecture Fig. 3. Layers of the BluTV Architecture
Fig. 5. Screenshot of the Main Application (Player) on Access Terminal (Set-top Box) The Figure 5 presents the BluTVPlayer application implemented with BluT Viewer widgets, which, besides allowing interaction via remote control, also provides accessibility features for people with special needs (Santos Junior et al, 2009; CENELEC, 2003).
Fig. 5. Screenshot of the Main Application (Player) on Access Terminal (Set-top Box) The Figure 5 presents the BluTVPlayer application implemented with BluT Viewer widgets, which, besides allowing interaction via remote control, also provides accessibility features for people with special needs (Santos Junior et al, 2009; CENELEC, 2003).
Fig. 6. Screenshot of the Repository of News Application Running on Access Terminal
Fig. 6. Screenshot of the Repository of News Application Running on Access Terminal
From its inception, multimedia's documents have experienced many developments. These changes are often the result of technological advances, but economic pressures have also helped to change the form of documents produced and to change the methods of work to product the documents. In the euphoria of the 1990s many multimedia start-ups were created. The lack of good work practices and of risk assessment has led many of them to bankrupt (Viéville, 2003). This experience resulted in a restrictive streamlining of production. Today, most of multimedia documents are only a hierarchical and indexed form of an information website: the risks of financial excesses are limited by reproducing the same documents structures and by applying specific ergonomic rules. So there is a standardization of documents structures and of methods used to produce them (Cartier, 2003). Despite a specification of methods and tools for multimedia design documents - AUTHOR methodology (Huart, 2000) (see figure 1), web design in terms of effective
From its inception, multimedia's documents have experienced many developments. These changes are often the result of technological advances, but economic pressures have also helped to change the form of documents produced and to change the methods of work to product the documents. In the euphoria of the 1990s many multimedia start-ups were created. The lack of good work practices and of risk assessment has led many of them to bankrupt (Viéville, 2003). This experience resulted in a restrictive streamlining of production. Today, most of multimedia documents are only a hierarchical and indexed form of an information website: the risks of financial excesses are limited by reproducing the same documents structures and by applying specific ergonomic rules. So there is a standardization of documents structures and of methods used to produce them (Cartier, 2003). Despite a specification of methods and tools for multimedia design documents - AUTHOR methodology (Huart, 2000) (see figure 1), web design in terms of effective
Fig. 3. Graphical representation of a diegetic molecule Elements of Perceptible Descriptions describe the characteristics of the entity actually visible. This description may contain a set of atoms or molecules of sensible nature, that is to say related to events that will physiological affect our senses of perception (if the corresponding entities are shown). These can be compared to the description of expressions of a message in semiotics (Deely, 2005).
Fig. 3. Graphical representation of a diegetic molecule Elements of Perceptible Descriptions describe the characteristics of the entity actually visible. This description may contain a set of atoms or molecules of sensible nature, that is to say related to events that will physiological affect our senses of perception (if the corresponding entities are shown). These can be compared to the description of expressions of a message in semiotics (Deely, 2005).
The communication's functions of the entity are also described in the semantic descriptions
The communication's functions of the entity are also described in the semantic descriptions
Fig. 6. Example of switching factor of a molecule in a tabulo-graphic representation These interlocking factors are major players in the autonomy of the represented entities So the entities themselves contain the conditions of the necessary information (but not necessarily sufficient) for putting them into function (display, arrival in the area of a perception of the reader...).
Fig. 6. Example of switching factor of a molecule in a tabulo-graphic representation These interlocking factors are major players in the autonomy of the represented entities So the entities themselves contain the conditions of the necessary information (but not necessarily sufficient) for putting them into function (display, arrival in the area of a perception of the reader...).
Fig. 5. Example of semantic descriptions related to the functions of the document design, this fear is not directly observable. This is however not necessary. This description will be converted by visible facts (beads of sweat, eyes bulging, stillness, nervous tremors, screams, etc... falling within the perceptible description) much later, when one approaches the “scenic” and the manufacture of the document - the “scenic” is the process to translate the text into a concrete reality: it is the result of aesthetic choices, practical constraints or financial... (Leleu-Merviel, 2005) .
Fig. 5. Example of semantic descriptions related to the functions of the document design, this fear is not directly observable. This is however not necessary. This description will be converted by visible facts (beads of sweat, eyes bulging, stillness, nervous tremors, screams, etc... falling within the perceptible description) much later, when one approaches the “scenic” and the manufacture of the document - the “scenic” is the process to translate the text into a concrete reality: it is the result of aesthetic choices, practical constraints or financial... (Leleu-Merviel, 2005) .
These modes of differentiation atoms / molecules are mainly features of links to some software used for general text editing or graphics. Indeed, when reading the version of the document, the selection of a molecule located in another molecule provides access to the detailed representation of the molecule in question.
These modes of differentiation atoms / molecules are mainly features of links to some software used for general text editing or graphics. Indeed, when reading the version of the document, the selection of a molecule located in another molecule provides access to the detailed representation of the molecule in question.
Fig. 16. Differentiation atoms / molecules in a graphical
Fig. 16. Differentiation atoms / molecules in a graphical
The combination is to group multiple molecules or atoms to form a new molecule.
The combination is to group multiple molecules or atoms to form a new molecule.
Fig. 22. Example of representation of validations “nested" If an entity is operating, it validates itself, and recursively, the entities that constitute it (provided that their interlocking factors are checked too): if the representation of the latter entities or events has no engagement, they are then validated by default. Thus, for the entity “documentary” to be enabled (see figure 22) the interlocking factor “session beginning” must be checked. The diegetic molecule “documentary” consists of two molecules covering “Life of tarantulas in South America” and “Bears in Europa”. The first theme is treated with a documentary film “film 1”. No constraint is specified for engagement “Life of tarantulas in South America”. Therefore, this entity “film 1” is put into service as soon as the entity “documentary” is operated. This process could be described as “auto-switch”. On the other hand regarding the entity “Bears Poldavia” it will be validated only when the documentary “film 1” is completed.
Fig. 22. Example of representation of validations “nested" If an entity is operating, it validates itself, and recursively, the entities that constitute it (provided that their interlocking factors are checked too): if the representation of the latter entities or events has no engagement, they are then validated by default. Thus, for the entity “documentary” to be enabled (see figure 22) the interlocking factor “session beginning” must be checked. The diegetic molecule “documentary” consists of two molecules covering “Life of tarantulas in South America” and “Bears in Europa”. The first theme is treated with a documentary film “film 1”. No constraint is specified for engagement “Life of tarantulas in South America”. Therefore, this entity “film 1” is put into service as soon as the entity “documentary” is operated. This process could be described as “auto-switch”. On the other hand regarding the entity “Bears Poldavia” it will be validated only when the documentary “film 1” is completed.
events “Play Music” and “show pictures” will be synchronous with the logical conjonction “equals” (which requires special attention when performing). On the other hand, the molecule “broadcast slide show 2”, due to the use of the logical conjunction “starts” indicates that “view photos” and “play music” are operated simultaneously, but their perfect synchronisation is not required. The logical conjunction of actions “meets” of the molecule of action “photographs show” indicates that the three photos are displayed one after the other. Fig. 23. Examples of parallel and sequential actions synchronous or not
events “Play Music” and “show pictures” will be synchronous with the logical conjonction “equals” (which requires special attention when performing). On the other hand, the molecule “broadcast slide show 2”, due to the use of the logical conjunction “starts” indicates that “view photos” and “play music” are operated simultaneously, but their perfect synchronisation is not required. The logical conjunction of actions “meets” of the molecule of action “photographs show” indicates that the three photos are displayed one after the other. Fig. 23. Examples of parallel and sequential actions synchronous or not
The author can define the temporal position of the entities using interlocking factors of diegetic molecules. Thus, when two molecules have identical interlocking factors, the entities they represent are enabled simultaneously. Also - in the example in figure 24 - “shot” and “Smoke at the tail gun” entities are switched at the same time, or when “the enemy shoots”, or when “the player shoots” (the latter screenplay occurs if the “player pulls the trigger” and the “Gun loaded up”).
The author can define the temporal position of the entities using interlocking factors of diegetic molecules. Thus, when two molecules have identical interlocking factors, the entities they represent are enabled simultaneously. Also - in the example in figure 24 - “shot” and “Smoke at the tail gun” entities are switched at the same time, or when “the enemy shoots”, or when “the player shoots” (the latter screenplay occurs if the “player pulls the trigger” and the “Gun loaded up”).
Moreover, the representation of dynamic entities, combined with the definition of a hypothetical sense produced by their display, enables an evolution by closing semantic. A meaning is defined by the creator from assumptions which are products of choices, possibly artistic ones and which often prove arbitrary. In the example in figure 25, if an entity means “meanings i” was displaying, the document will display an entity will mean “meanings i+1” or “meanings i-1”. The document will be able to follow a coherent evolution towards the objectives of the creative elements. The document successively validates semantically close or at the opposite far away elements: some situations may indeed lead to strong semantic ruptures.
Moreover, the representation of dynamic entities, combined with the definition of a hypothetical sense produced by their display, enables an evolution by closing semantic. A meaning is defined by the creator from assumptions which are products of choices, possibly artistic ones and which often prove arbitrary. In the example in figure 25, if an entity means “meanings i” was displaying, the document will display an entity will mean “meanings i+1” or “meanings i-1”. The document will be able to follow a coherent evolution towards the objectives of the creative elements. The document successively validates semantically close or at the opposite far away elements: some situations may indeed lead to strong semantic ruptures.
The autonomy of the entities enables the representation of a model of the reader. It is not to reify the reader but to propose an “avatar” (an avatar is a representation - necessarily schematic - and approximate of the user within the system). These define a psychological state of the player according to their actions. This will often be in a totally arbitrary way, regardless of any psychological model scientifically established, as is the case in the examples in figures 26 and 27. The model serves only to establish a strategy of evolution. A novelist, a screenwriter or film-maker will write on the basis of the potential reactions of an imaginary reader. In the same vein, a model of the reader will enable the author to establish a strategy of interaction between the entities of the document and the reader. Today, it is possible to determine few cognitive states and emotional states of an user (Labour & Kolski, 2010), but it is impossible to determine it directly, in real time and without complex interfaces.. This means, if you want to adapt the system to the reader, that one must go through models of representation of it. In fact, a model of the player is an entity that seeks to characterize the reader from facts which are objective and measurable by the document and its technical environment. The character deduced can also influence the evolution of the document. The author may delegate to specialists in artificial intelligence the design of a specific system of diagnosis of a cognitive, psychological state of the reader during the implementation of the document.
The autonomy of the entities enables the representation of a model of the reader. It is not to reify the reader but to propose an “avatar” (an avatar is a representation - necessarily schematic - and approximate of the user within the system). These define a psychological state of the player according to their actions. This will often be in a totally arbitrary way, regardless of any psychological model scientifically established, as is the case in the examples in figures 26 and 27. The model serves only to establish a strategy of evolution. A novelist, a screenwriter or film-maker will write on the basis of the potential reactions of an imaginary reader. In the same vein, a model of the reader will enable the author to establish a strategy of interaction between the entities of the document and the reader. Today, it is possible to determine few cognitive states and emotional states of an user (Labour & Kolski, 2010), but it is impossible to determine it directly, in real time and without complex interfaces.. This means, if you want to adapt the system to the reader, that one must go through models of representation of it. In fact, a model of the player is an entity that seeks to characterize the reader from facts which are objective and measurable by the document and its technical environment. The character deduced can also influence the evolution of the document. The author may delegate to specialists in artificial intelligence the design of a specific system of diagnosis of a cognitive, psychological state of the reader during the implementation of the document.
This type of model, from simple rule, has enabled student groups to script documents establishing strategies of evolution fwith the player behaviour (or his avatar). They were, for example, documents determining the presence (or absence) of visual impairment in the reader and in order to offer navigation features adapted to their handicap without special interface. To sum up, the molecular model of the reader is not there to “manufacture” a reader, but to contain - in an arbitrary, possibly with the vision of an “artist” - a hypothetical mental state according to his reactions. This model is based on observable facts in the document and its technical environment.
This type of model, from simple rule, has enabled student groups to script documents establishing strategies of evolution fwith the player behaviour (or his avatar). They were, for example, documents determining the presence (or absence) of visual impairment in the reader and in order to offer navigation features adapted to their handicap without special interface. To sum up, the molecular model of the reader is not there to “manufacture” a reader, but to contain - in an arbitrary, possibly with the vision of an “artist” - a hypothetical mental state according to his reactions. This model is based on observable facts in the document and its technical environment.
version for 2 gates. Two cameras are used capturing video streams and implementations of the following algorithms: decoder - unpacking encoded video frames, background remover - the algorithm detecting moving objects in the video stream and removing the background, human detector - the algorithm detecting a human silhouette in the incoming images, event counter - comparing the number of messages with detected events describing incoming people and signaling large imbalance between them.
version for 2 gates. Two cameras are used capturing video streams and implementations of the following algorithms: decoder - unpacking encoded video frames, background remover - the algorithm detecting moving objects in the video stream and removing the background, human detector - the algorithm detecting a human silhouette in the incoming images, event counter - comparing the number of messages with detected events describing incoming people and signaling large imbalance between them.
Fig. 4. A typical flow of activities of complex service creation Fig. 4 presents the typical flow of activities leading to building of a complex service scenario. We start with an idea of multimedia stream processing, for instance identification of well- known objects or events. Then we can focus on the algorithm described in pseudo-code or other high level language, eg. flowchart. When that transition appears, we translate them to the real source code. It is compiled and linked into a computation task executable code. Having the executable algorithm program we enrich it with the quality and parameters metadata to obtain a simple service. The platform provides automatic generation of the WSLD description of this service, which in turn can be used to build more complex structures. The special language MSP-ML was designed to express the service composition keeping characteristic stream-processing constraints.
Fig. 4. A typical flow of activities of complex service creation Fig. 4 presents the typical flow of activities leading to building of a complex service scenario. We start with an idea of multimedia stream processing, for instance identification of well- known objects or events. Then we can focus on the algorithm described in pseudo-code or other high level language, eg. flowchart. When that transition appears, we translate them to the real source code. It is compiled and linked into a computation task executable code. Having the executable algorithm program we enrich it with the quality and parameters metadata to obtain a simple service. The platform provides automatic generation of the WSLD description of this service, which in turn can be used to build more complex structures. The special language MSP-ML was designed to express the service composition keeping characteristic stream-processing constraints.
The KASKADA platform provides a dedicated framework supporting implementation of the analysis algorithms. Every algorithm, embedded in the KASKADA framework, can use a special API providing the platform functionality: multimedia encoding and decoding, intermediate results interchange, signaling the events, processing the input parameters, and stream synchronization. The algorithms are implemented in C++ and together with the framework create computation tasks (see Fig. 5) under the Linux operating system in the cluster environment. Multimedia streams captured by the cameras or microphones are received under the RTSP protocol, the framework decodes the flowing video frames or audio probes and forwards them as input data to the task. Additionally, it can use the metadata related to the received stream, eg. the currently processed frame timestamp, resolution or video frame rate, or the audio probing frequency.
The KASKADA platform provides a dedicated framework supporting implementation of the analysis algorithms. Every algorithm, embedded in the KASKADA framework, can use a special API providing the platform functionality: multimedia encoding and decoding, intermediate results interchange, signaling the events, processing the input parameters, and stream synchronization. The algorithms are implemented in C++ and together with the framework create computation tasks (see Fig. 5) under the Linux operating system in the cluster environment. Multimedia streams captured by the cameras or microphones are received under the RTSP protocol, the framework decodes the flowing video frames or audio probes and forwards them as input data to the task. Additionally, it can use the metadata related to the received stream, eg. the currently processed frame timestamp, resolution or video frame rate, or the audio probing frequency.
From the point of view of output data distribution, the algorithm execution and tests can be performed using archiving or streaming modes (see Fig. 6 d-e). In the first case the processed output stream is stored into the archive file, and in the latter the results are transmitted by a streaming server to the client. There is also the possibility to use a hybrid approach where the data are simultaneously stored in the archive and transmitted after some transformations to the users (see Fig. 6 f).
From the point of view of output data distribution, the algorithm execution and tests can be performed using archiving or streaming modes (see Fig. 6 d-e). In the first case the processed output stream is stored into the archive file, and in the latter the results are transmitted by a streaming server to the client. There is also the possibility to use a hybrid approach where the data are simultaneously stored in the archive and transmitted after some transformations to the users (see Fig. 6 f).
Fig. 8. An example of multiple relations between algorithms' implementation and services
Fig. 8. An example of multiple relations between algorithms' implementation and services
Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 10. Steps required to add and execute a simple service: (a) service definition, (b) service selection from the list, (c) entering input parameters Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 10. Steps required to add and execute a simple service: (a) service definition, (b) service selection from the list, (c) entering input parameters Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 11. From complex service scenario definition (a) trough XML document (b) to GUI service definition (c)
Fig. 11. From complex service scenario definition (a) trough XML document (b) to GUI service definition (c)
Fig. 12. Data and user interoperability of the KASKADA needs to be received, validated, and archived, due to additional offline analysis and legal concerns. The above operations are performed by the tasks controlled by the management server. The server can also cooperate directly with the stream sources. Preprocessing is used to unpack the stream data from its native protocol, usually RTSP, and forward it to the proper analysis tasks located on the computational nodes using an Infiniband network. The higher platform efficiency can be achieved by allocation of tasks with streams to more cluster nodes. A special set of the cluster’s computational nodes is assigned to perform such tasks using the high performance network file system LUSTRE. Physically, the data is stored on a specific data server with high performance 500TB hard drives. The various algorithms exposed as services can consume incoming streams and produce messages (as events created during the analysis) as well as new output streams, which should be delivered to the user. See the next section for more details about the flow of events.
Fig. 12. Data and user interoperability of the KASKADA needs to be received, validated, and archived, due to additional offline analysis and legal concerns. The above operations are performed by the tasks controlled by the management server. The server can also cooperate directly with the stream sources. Preprocessing is used to unpack the stream data from its native protocol, usually RTSP, and forward it to the proper analysis tasks located on the computational nodes using an Infiniband network. The higher platform efficiency can be achieved by allocation of tasks with streams to more cluster nodes. A special set of the cluster’s computational nodes is assigned to perform such tasks using the high performance network file system LUSTRE. Physically, the data is stored on a specific data server with high performance 500TB hard drives. The various algorithms exposed as services can consume incoming streams and produce messages (as events created during the analysis) as well as new output streams, which should be delivered to the user. See the next section for more details about the flow of events.
Fig. 13. Event processing idea in the KASKADA platform Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 13. Event processing idea in the KASKADA platform Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 14. Example of the cluster fragmentation preventing a new task assignment Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 14. Example of the cluster fragmentation preventing a new task assignment Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 16. Domain class diagram of the KASKADA platform Both service types, i.e. simple and complex ones, are going to be deployed on the same JEE application server, we consider using a Tomcat web container for this purpose. They will utilize SOAP technologies over HTTP(S) protocol, in case of synchronous remote calls, and a queue system, i.e. ActiveMQ for asynchronous communication within JMS interface. The result return will be performed in separated objects (and components): Event Handler for messages and Dispatcher for multimedia streams.
Fig. 16. Domain class diagram of the KASKADA platform Both service types, i.e. simple and complex ones, are going to be deployed on the same JEE application server, we consider using a Tomcat web container for this purpose. They will utilize SOAP technologies over HTTP(S) protocol, in case of synchronous remote calls, and a queue system, i.e. ActiveMQ for asynchronous communication within JMS interface. The result return will be performed in separated objects (and components): Event Handler for messages and Dispatcher for multimedia streams.
Fig. 17. A sequence diagram of the complex service execution within the domain model (see Fig. 16) Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Fig. 17. A sequence diagram of the complex service execution within the domain model (see Fig. 16) Real-time Multimedia Stream Data Processing in a Supercomputer Environment
Computational tasks - the executable code of the multimedia stream analysis algorithms embedded in the framework accomplish the appropriate computations. They receive the multimedia streams generated by a camera, microphone, or other device (e.g. medical equipment), make the required analysis and transformation and send an output data stream including discovered events to the proper components, mainly to Event Handler and Dispatcher, forwarding them through the service layers to the clients - a users or an external applications (see Fig. 19).
Computational tasks - the executable code of the multimedia stream analysis algorithms embedded in the framework accomplish the appropriate computations. They receive the multimedia streams generated by a camera, microphone, or other device (e.g. medical equipment), make the required analysis and transformation and send an output data stream including discovered events to the proper components, mainly to Event Handler and Dispatcher, forwarding them through the service layers to the clients - a users or an external applications (see Fig. 19).
Fig. 20. Deployment diagram of KASKADA platform To execute computation tasks all software components should be deployed on the computer cluster. Fig. 20 presents the deployment diagram including hardware nodes with the assigned software components. The core of the platform is the cluster which consists of 672 two-processor (Intel) nodes connected by the fast Infiniband network, each processor has 8 cores, which gives in total 5376 cores.
Fig. 20. Deployment diagram of KASKADA platform To execute computation tasks all software components should be deployed on the computer cluster. Fig. 20 presents the deployment diagram including hardware nodes with the assigned software components. The core of the platform is the cluster which consists of 672 two-processor (Intel) nodes connected by the fast Infiniband network, each processor has 8 cores, which gives in total 5376 cores.
Fig. 19. Component diagram of the KASKADA platform Almost all the above domain classes presented in Fig. 16 can be straightforwardly converted into the software components of the proposed platform. The only exception is the User Console component which aggregates Scheduler class as well as manages the other platform components including operations on the multimedia and other data streams (especially in off-line mode - using recorded data), security and service configuration and deployment (a service repository with the WSDL and UDDI support).
Fig. 19. Component diagram of the KASKADA platform Almost all the above domain classes presented in Fig. 16 can be straightforwardly converted into the software components of the proposed platform. The only exception is the User Console component which aggregates Scheduler class as well as manages the other platform components including operations on the multimedia and other data streams (especially in off-line mode - using recorded data), security and service configuration and deployment (a service repository with the WSDL and UDDI support).
Table 1. The main KASKADA platform functional features available for the applications The KASKADA platform was heavily tested for two pilot applications, developed as a proof of concept of the proposed multimedia processing model: identification of dangerous objects and unusual situations (DOUS) occurring in multimedia streams coming from cameras located in different places, and medical recommender for endoscopy examinations (MREE) showing some disease changes in film taken during gastroscopy track. The former is used for automation of typical video and audio surveillance tasks, like dangerous person identification or left luggage detection. The latter helps medical staff to quickly find the possible lesions in the video recorded during endoscopy examinations. Table 1 presents the usage of the platform features by the applications.
Table 1. The main KASKADA platform functional features available for the applications The KASKADA platform was heavily tested for two pilot applications, developed as a proof of concept of the proposed multimedia processing model: identification of dangerous objects and unusual situations (DOUS) occurring in multimedia streams coming from cameras located in different places, and medical recommender for endoscopy examinations (MREE) showing some disease changes in film taken during gastroscopy track. The former is used for automation of typical video and audio surveillance tasks, like dangerous person identification or left luggage detection. The latter helps medical staff to quickly find the possible lesions in the video recorded during endoscopy examinations. Table 1 presents the usage of the platform features by the applications.
Quality characteristics are one of the most important features evaluating the whole platform behavior. During the platform analysis and design, we found out we need to focus on four key factors: performance, scalability, dependability and security. Performance is crucial for real-time processing, both processor speed and network bandwidth need to be examined, for the platform to work in the real environment. The scalability is another important factor, especially when large numbers of the input streams need to be archived, processed, and transmitted back to the customer. Moreover, even the high performance solutions can't provide the satisfactory results until they are dependable (the designed algorithms are correct and acceptable for various conditions), and finally the character of the processed streams, including surveillance and medical data, requires high security protecting access from unauthorized persons. The above applications, especially DOUS, can achieve the appropriate public security level using strategy illustrated in Fig. 21.
Quality characteristics are one of the most important features evaluating the whole platform behavior. During the platform analysis and design, we found out we need to focus on four key factors: performance, scalability, dependability and security. Performance is crucial for real-time processing, both processor speed and network bandwidth need to be examined, for the platform to work in the real environment. The scalability is another important factor, especially when large numbers of the input streams need to be archived, processed, and transmitted back to the customer. Moreover, even the high performance solutions can't provide the satisfactory results until they are dependable (the designed algorithms are correct and acceptable for various conditions), and finally the character of the processed streams, including surveillance and medical data, requires high security protecting access from unauthorized persons. The above applications, especially DOUS, can achieve the appropriate public security level using strategy illustrated in Fig. 21.
Related topics:
Interactive Multimedia ApplicationsInteractive MultimediaExperimental MultimediaInteractive Multimedia Case Studies
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved