Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Design of Items
Discussion
Future Developments
References
All Topics
Computer Science

Evaluation of ICT Competencies

Profile image of Agustin TristanAgustin Tristan

2013, Handbook of Research on Educational Communications and Technology

https://doi.org/10.1007/978-1-4614-3185-5_26
visibility

…

description

25 pages

link

1 file

Abstract

Information and Communication Technology (ICT) competencies comprise a subset of ­digital literacy, one of the various areas of technological competencies and skills necessary for real-life functioning in all kind of professional activities and levels of education from preschool to graduate levels. Assessment of ICT competencies involves the definition of specific target activities appropriate to the environment in which they are required. These environments range from the home to the workplace and involve such disparate activities as information-retrieval in healthcare settings or libraries; use of clerical, business, or investment applications; and interactions with government or other public services. The range of ICT competencies also includes the specific abilities needed by professionals responsible for the development of software or communication products and services. This chapter discusses several ICT assessment projects, addresses the primary technical specifications required for evaluation, and explores solutions to problems of test administration. These solutions range from electronic quizzes (similar to paper-and-pencil tests) to more “authentic” forms of assessment using e-portfolios or simulations of real software applications. A detailed analysis of the primary approaches for assessment of social, academic, commercial, or economic environments reveals that these approaches primarily focus on a basic core of skills consisting of Web navigation and the use of e-mail and office tools (text processor, spreadsheet, presentation, and database management). In the future, ICT evaluation will involve automatic scoring of natural language responses in documents, solutions of mathematical problems, and graphical or health applications, among many other real-life endeavors.

Related papers

Electronic Rubrics to Assess Competences in ICT Subjects
Manuel Cebrian de la Serna

European Educational Research Journal, 2014

Helping students to acquire specific competences is nowadays one of the basic pillars of university teaching; therefore its evaluation and accreditation is of key importance. As of late, rubrics and in particular electronic rubrics (e-rubrics) have become an important resource to assess competences and guide students in their learning processes. In this contribution, the authors present a quasi-experimental study that was conducted to explore and evaluate the use of the e-rubrics in subjects related to information and communication technology (ICT). Data were collected on how students viewed their use of e-rubrics in self-evaluation and peer evaluation. They show that the experimental groups using e-rubrics were better evaluated by their professors than the control groups who did not engage in self-evaluation using e-rubrics. The students see e-rubrics as a positive resource because they feel e-rubrics allow for a truly objective evaluation. Additionally, e-rubrics are considered to...

View PDFchevron_right
Assessment of 21st century ICT skills in Chile: Test design and results from high school level students
Susana Valenzuela

This paper describes a study that evaluates fifteen-year-old Chilean students Information and Communication Technology (ICT) skills. The paper presents an operational definition of ICT skills, an instrument measuring these skills as well as the students' results in the test. The definition of ICT skills used considers Chile's curricular framework, functional and cognitive skills. Specifically, ICT skills were defined as the capacity to solve problems of information, communication and knowledge in digital environments. A performance-based assessment was designed in a virtual environment to measure these skills. The analysis of the results showed that the majority of students were able to solve tasks related to the use of information as consumers, i.e., approximately three quarters of the students were able to search for information and half of them were also able to organize and manage digital information. Additionally, they show that very few students were able to succeed in tasks related to the use of information as producers, i.e., only one third of the students were able to develop their own ideas in a digital environment and less than one fifth were able to refine digital information and create a representation in a digital environment. Socioeconomic group, access, daily use and confidence in doing ICT-related activities were all positively associated with higher scores, showing the need to implement strategies to compensate this inequality, possibly by explicitly defining these aims in the national curriculum.

View PDFchevron_right
Assessing information and communications technology literacy for higher education
Irvin Katz, P. Cooper, Heather Nadelman, Diego Zapata-Rivera

2004

Whereas few would argue about the growing importance of information and communication technology (ICT) skills as they affect educational attainment, workforce readiness, or lifelong learning, there is less agreement as to what these skills and knowledge are and how best to measure them. This paper describes an effort to define ICT Literacy and to design and develop an assessment that measures ICT Literacy for the higher education community. We begin with a summary of efforts to understand ICT Literacy as it pertains to educational achievement in a variety of higher educational settings. We use this foundation to describe how Evidence Centered Design (ECD) methodology is being used to design and develop a simulations-based assessment that serves both individuals and higher education institutions. We follow with a presentation of the challenges and tensions that exist when implementing such an assessment design in the form of operational simulation tasks. We conclude with a discussion of the strategies and challenges of automatically scoring such an assessment using Bayesian networks.

View PDFchevron_right
Computing competencies
Gloria Gonzales-Dholakia

2003

View PDFchevron_right
Ict Competency Standards for Teachers
Ola Erstad

2007

Foreword To live, learn, and work successfully in an increasingly complex, information-rich and knowledge-based society, students and teachers must utilize technology effectively. Within a sound educational setting, technology can enable students to become: • Capable information technology users • Information seekers, analyzers, and evaluators • Problem solvers and decision makers • Creative and effective users of productivity tools Through the ongoing and effective use of technology in the schooling process, students have the opportunity to acquire important technology capabilities. The key individual in helping students develop those capabilities is the classroom teacher. The teacher is responsible for establishing the classroom environment and preparing the learning opportunities that facilitate students' use of technology to learn, and communicate. Consequently, it is critical that all classroom teachers are prepared to provide their students with these opportunities. Both p...

View PDFchevron_right
Design and Validation of a Digital Competency Questionnaire for Research
Carlos Arturo Torres Gastelu

UA Journals, 2019

Currently, citizens find themselves in a technological environment that demands a series of digital skills that must be developed to perform adequately, either in society or in any type of profession. It is of our interest that these competences allow students to perform satisfactorily in the academic field. Thus, the need to establish the level of digital skills that students require in order to carry out scientific research in a satisfactory manner. For this reason, an instrument of self-perception was adapted that gives a diagnosis of digital competences for research conducted by new university students. In order to carry out the analysis of reliability and validity of the instrument, a pilot test was implemented with a representative sample of 71 new students, belonging to a military engineering university in Veracruz, Mexico. As a result of the analysis, the instrument obtained a remarkable reliability, yielding a Cronbach's Alpha of 0.890 and its construct validity was average, obtaining a global Kaiser Meyer Olkin index of 0.718. Regarding the level of competence detected, a better perception of development was obtained in the students coming from private schools and technological baccalaureates, while the lowest levels were found in those coming from CONALEP, COBAEV and tele-baccalaureates. In light of the results, it is important to visualize the factors that can affect the academic development of students in order to allow them to enter universities with a better expectation of success.

View PDFchevron_right
Assessing ICT Competencies among Postgraduate Students Based on the 21st Century ICT Competency Model
Mazalah Ahmad

Asian Social Science, 2013

Assessing the ICT competency among Postgraduate students in the Higher Learning Institutions (HLI) is crucial. Therefore this paper presents in detail the process of developing an instrument to assess the competency in ICT which is called ICTC-Test (Information Communication Technology Competency Test). The details are on the utilization of the seven domains in the 21st Century ICT Competency Model to determine the competency level among students. The development of the items in ICTC-Test is based on the application of the performance-based assessment and the task-based assessment. The assessment of ICT competency using ICTC-Test is different compared to existing instruments where ICTC-Test changes the focus in assessing the competency in ICT from the ability to use and maintain the hardware, software and network to competency in ICT that includes technical ability and cognitive skills. Moreover, ICTC-Test also integrates the five generic skills highlighted in MQF which are critical thinking skill, problem solving skill, information technology and communication skill, communication and management skill.

View PDFchevron_right
Evaluation of ICT Literacy Skills
Samuel A B I O D U N Olaleye

EVALUATION OF ICT LITERACY SKILLS AMONG STUDENTS OF FEDERAL POLYTECHNIC OFFA, KWARA STATE, NIGERIA: A REVIEW OF RELATED LITERATURES, 2016

The purpose of the present paper has been undertaken with a view to evaluate the ICT skills of students amongst the College Library Users of Federal Polytechnic, Offa, Kwara State. So far, economic research has failed to provide a clear consensus on the level of ICT literacy skills of student’s in Polytechnics and how it affects their academic achievement. Our paper aims to summarize the main findings of the literature and to give complementary explanations. The first explanation focuses on the concept and revolution of ICT in the 21st century on standard explanatory factors. The emergence of polytechnics in Nigeria was discussed in relation to the establishment of Federal Polytechnic Offa, Kwara State. The second phase of the review advocates on the importance of ICT literacy skills as well as the purpose of use, frequency and the effect of the skills on academic activities of polytechnic students. The challenges facing the ICT skills of polytechnic students were evaluated in which several recommendations were made. Keywords: ICT, ICT Literacy, ICT Literacy Skills, Internet, Polytechnic Students, Polytechnic Library

View PDFchevron_right
Competency Standards Modules ICT COMPETENCY STANDARDS FOR TEACHERS
Boris Popov

To live, learn, and work successfully in an increasingly complex, information-rich and knowledgebased society, students and teachers must utilize technology effectively. Within a sound educational setting, technology can enable students to become:

View PDFchevron_right

References (44)

  1. American Educational Research Association, American Psychological Association, & National Council of Measurement in Education (1999). Standards for Educational and Psychological Testing. Washington, DC: Author.
  2. Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model. Mahwah, NJ: Lawrence Erlbaum Associates, Pub.
  3. Ceneval (2010). ENLACE Media Superior. Manual para docentes y directivos [ENLACE for high school. A guide for teachers and directors].
  4. Mexico, DF: Author.
  5. Cohen, R. J., & Swerdlik, M. E. (2002). Psychological testing and assessment; an introduction to tests and measurement. Fifth ed. Boston, MA: McGraw Hill.
  6. Costa, A. L., Garmston, R. J., & Lambert, L. (1988). Evaluation of teaching: The cognitive development view. In Sarah J. Stanley & James Popham (Eds.), Teacher evaluation: six prescriptions for success. New York, NY: Association for Supervision and Curriculum Development.
  7. Draba, R. E. (1977). The identification and interpretation of item bias. Memorandum 25. Educational Statistics Laboratory. Chicago, IL: University of Chicago.
  8. Educational Testing Service (2005). Beyond Technical Competence: Literacy in Information and Communication Technology. An issue paper from ETS, ICT Literacy Assessment. Princeton, NJ: Author.
  9. Finch, F. L. (1988). A taxonomy of competency testing programs. In Richard M. Jaeger & Carol Kehr Tittle (Eds.). Minimum Competency Achievement Testing (Chap. 24). Berkeley, CA: McCutchan. Pub.
  10. Corp.
  11. Gago, H. A., & Mercado, C. R. (1995). La evaluación en la educación superior en México [Evaluation in high-level education in Mexico].
  12. Revista de la Educación Superior, N. 96, pp. 61-86. Mexico: ANUIES.
  13. Glassick, C. E., Taylor, M., & Maeroff, G. I. (1997). Scholarship assessed. Evaluation of the Professoriate. The Carnegie Foundation for the Advancement of Teaching. San Francisco, CA: Jossey-Bass Inc. Pub.
  14. Haladyna, T. M., & Downing, S. M. (1989a). A typology of multiple choice item writing rules. Applied Measurement in Education, 2, 37-50
  15. Haladyna, T. M., & Downing, S. M. (1989b). Validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2, 51-78.
  16. Haladyna, T. M. (1992). The effectiveness of several multiple-choice formats. Applied Measurement in Education, 5(1), 73-88.
  17. Instituto Colombiano para el fomento de la Educación Superior (2004). Evaluación por competencias ICFES [ICFES Competencies evaluation]. Editorial Magisterio. Bogotá, Colombia: Author.
  18. Instituto de Evaluación e Ingeniería Avanzada (1998). Kalt Criterial® User's manual. San Luis Potosi, Mexico: Author. Available through: www. ieia.com.mx
  19. Johnson, E. G. (1994). The NAEP 1992 technical report. Educational Testing Service, National Center for Educational Statistics, July 1994, Report No.23-TR20.1031. Washington, USA: US Dept. of Education.
  20. Kilpatrick, J. (1979). Methods and results of evaluation with respect to mathematics education. In B. Christiansen & H. G. Steiner (Eds.), New Trends in Mathematics Teaching (4) (pp.162-179). Paris, France: UNESCO.
  21. Kubiszyn, T., & Borich, G. (2007). Educational Testing and Measurement. Eighth Ed. New York, NY: John Wiley & Sons, Inc. Ed.
  22. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575
  23. Linacre, J. M. (2010). Winsteps® User's Manual. Australia: Winsteps.com.
  24. Louis, R., Jutras, F., & Hensler, H. (1996). Des objectifs aux competences: implications pour l'évaluation de la formation initiale des maîtres. Revue Canadienne de l'Éducation, 21(4), 414-432
  25. Magno, C. (2009). Taxonomy of aptitude test items: A guide for item writers. The International Journal of Educational and Psychological Assessment, 2(1), 39-53.
  26. Manatt, R. P. (1985). Competent evaluators of teaching: their knowledge, skills and attitudes. In William R. Duckett (Ed.), The Competent Evaluator of Teaching. Bloomington, IN: Phi Delta Kappa.
  27. McNally, H. J. (1977). Performance-based teacher evaluation. NASSP Bulletin, 61, 104-105.
  28. OECD (2010a). PISA 2009 Results: Learning to Learn -Student Engagement, Strategies and Practices (Volume III). Retrieved from http://dx.doi.org/10.1787/9789264083943-en OECD (2010b). Strong performers and successful reformers in education. Lessons from PISA for Mexico. Retrieved from http://www.pisa.oecd.org/dataoecd/55/0/46638969.pdf
  29. Pacheco, M. T., & Diaz, B. A. (2000). Evaluación académica. [Academic evaluation] Mexico: CESU-UNAM.
  30. Peterson, K. (2004). Research on school teacher evaluation. NASSP Bulletin, 88, 60-79
  31. Reyes, G. (1997). NAFTA and the future of Mexican High-school education. The ANNALS of the American Academy of Political and Social Science, 550(1), 96-104
  32. Rueda, M., & Landesmann, M. (1999). ¿Hacia una nueva cultura de la evaluación de los académicos? [Towards a new academic evaluation culture?] Pensamiento Universitario 88. Mexico: UNAM.
  33. Simpson, R. H. (1976). L'educateur et l'autoévaluation. [Teacher Self- evaluation], Coll. SUP. Collin MacMillan, International, Inc. Paris, France: Presses Universitaires de France.
  34. Tristán, L. A. (2004). Aporte a la metodología para diseño -a la medida‖ de reactivos de opción múltiple [Methodological contribution to -tailor-made‖ multiple-choice items].
  35. Foro de Evaluación Educativa VI. Mexico: Ceneval.
  36. Tristán, L. A., & Salazar, M. (2001). Examen de Certificación para profesores de educación media superior (ECPEMS) [Certification exam for teacher in High-school education]. Foro Regional de Evaluación Educativa (pp. 71-77). Campeche, Mexico: Ceneval.
  37. Tristán, L. A., & Vidal, U. R. (2007a). Standards for objective tests. AERA meeting, session: -New Developments in Measurement Thinking‖, Chicago. Available through ERIC document: ED496126.
  38. Tristán, L. A., & Vidal, U. R. (2007b). Linear model to assess the scale's validity of a test. AERA meeting, session: -New Developments in Measurement Thinking‖, Chicago. Available through ERIC document: ED501232.
  39. White, M. (2011). Predicting success in teacher certification testing: The role of academic help seeking. The International Journal of Educational and Psychological Assessment, 7(1), 24-44.
  40. Wilkerson, J. R., & Lang, W. S. (2004). A Standards-driven, Task-based Assessment Approach for Teacher Licensure or Certification with Potential for College Accreditation. Practical Assessment, Research, and Evaluation, 9(12), [on-line journal], Retrieved at http://pareonline.net/getvn.asp?v=9&n=12
  41. Wilkerson, J. R., & Lang, W. S. (2005). CAATS -Comprehensive Assessments Aligned with Teacher Standards -A Five Step Design Model for Assessing Teachers Validly and Reliably. Annual Meeting of the American Association of Colleges of Teacher Education. Available through ERIC document: ED502867.
  42. Wright, B. D., & Masters, G. (1982). Rating Scale Analysis. Chicago, IL: Mesa Press.
  43. Wright, B. D., & Stone, M. H. (1979). Best Test Design. Chicago, IL: Mesa Press.
  44. Zirkel, P. A. (1996). The Law of Teacher Evaluation. A Self-assessment Handbook. Bloomington, IN: Phi Delta Kappa Educational Foundation.

Related papers

Assessment Tests for Digital Skills: A Tool for Learning Outcomes and University Accreditation
Zeal Press

Journal of Modern Mechanical Engineering and Technology, 2015

An important question asked by academicians is whether students use Internet languages constructively to further themselves in their chosen field of study while in college, or merely use them for social networking and entertainment. The iSkills Test is a possible tool for assessment of students' skills in using digital technology on the Internet. This paper seeks to investigate the test's usefulness in helping to answer the question stated above and in the process, add positively to the reaffirmation of accredited universities where these students are enrolled. An original method has been shown to allow instructors/coaches and institutions to assess the effectiveness of their programs through the use of this test, even though the grading scheme has been changed between the initial and final assessment tests (in one term). The grading code of the iCritical Thinking Certification test (now updated as the iSkills test) has been found, at least for the present. After two years of testing data at the University of Miami, the University has been provided with three major strengths and three major weaknesses of the students. The undergraduate strengths are (1) shopping, (2) following directions, (3) using information ethically. The students' weaknesses are (1) selecting resources, (2) researching, (3) knowing or understanding what they find. It is apparent that the areas of weaknesses are areas of opportunity for the University to improve their students' digital skills. Some useful suggestions have been developed to help students improve their critical thinking abilities and thus perform better in the certification test. The iSkills tests involved organization of information, development of a search strategy, creating a slide, summarize researched information, creating a visual representation and constructing an advanced search. These tasks involved a variety of skill sets and they mimic real world tasks. This test is recommended for the assessment by universities of their students' improvement in digital skills development while on the Internet, across the disciplines. The tests are used to help convince university accreditation bodies like Southern Association of Colleges and Schools in the U.S.A. International acceptance of the test is recommended since the use of the Internet is global, and digital skills development is an undeniably important asset.

View PDFchevron_right
Information and Communication Technology (ICT) Literacy: Integration and Assessment in Higher Education
Rhaegee Tamaña

Despite coming of age with the Internet and other technology, many college students lack the information and communication technology (ICT) literacy skills-locating, evaluating, and communicating information-necessary to navigate and use the overabundance of information available today. This paper presents a study of the validity of a simulations-based assessment of ICT literacy skills. Our overall goals for the assessment are to support ICT literacy instructional initiatives at colleges and universities.

View PDFchevron_right
Frameworks for Digital Competence Assessment: Proposals, Instruments, and Evaluation
Antonio Cartelli

… of Informing Science & IT Education …, 2010

After a short introduction on the need for a new literacy, coming from the work of many organizations and institutions during last twenty years, the approach adopted by an Italian research group for the planning and development of a framework for digital competence assessment is discussed. This model, the first built until now, has been used to create a questionnaire experimented with High School students and has been discussed with colleagues in conferences and meetings. The criticisms and amendments which have been proposed led to a new framework, which has the main feature of being centred on the subject while focussing the analysis of the digital competence, more than adopting a discipline (IT/ICT) centred approach for its assessment, like in the first framework.

View PDFchevron_right
Web-based assessment tool for Teachers Digital Competency
Andrea Kulakov

PROCEEDINGS OF THE 8th INTERNATIONAL CONFERENCE ON APPLIED INTERNET AND INFORMATION TECHNOLOGIES, 2018

Even though there are several different frameworks and standards for proving the digital competencies expected from teachers, there is a need for a Web-based assessment tools that allow real, dependable and valid assessment of these competencies. This paper addresses the design challenges related to a software solution for selfand peer-assessment of teachers' digital competencies. In this system the competencies are assessed by a teacher his/her-self or by her/his peers using the performance indicators that are based on the competency model for Teachers.

View PDFchevron_right
Bebras Contest and Digital Competence Assessment
Valentina Dagienė

International Journal of Digital Literacy and Digital Competence, 2010

The article is made of two parts. The first part discusses the importance of informal education environments supported by IT/ICT in students’ learning, followed by reports of some international competitions and the role they have in improving students’ interest and use of Informatics and related disciplines. At the end of the section, it describes the Bebras contest, an international competition supporting students’ Information and Communication Technology competences with emphasis on cross discipline competences, which are useful to solve real life problems. In the second part of the article, the outcomes of a research study on the features of a framework for digital competence assessment are reported. Based on this, some criticisms emerging from the analysis of the answers that students gave to a questionnaire built on the guidelines of the mentioned framework are analysed. They are integrated by the comments that teachers, colleagues and researchers made on the structure of the h...

View PDFchevron_right

Related topics

  • Computer Science

    • Cited by

    Digital Literacy of EFL Students: An Empirical Study in Vietnamese Universities
    Anita Habók

    Libri

    The current research investigates the digital literacy levels of 1661 English as a foreign language (EFL) learner at Vietnamese universities. We used an adapted questionnaire to assess students’ digital knowledge and their perceived skills, their attitudes toward the use of digital technologies, and the frequency of use of technology applications in English learning. The findings reveal that most Vietnamese students can access digital technologies at home and in their institutions. Furthermore, students achieve an adequate level of knowledge regarding digital literacy, and their technological skills range from low to average. Students’ attitudes toward technologies are positive, but they do not use technologies extensively when learning English. Comparisons show that males have better digital knowledge and skills than their female peers. Although female students are more aware of the digital integration benefits of learning than their male peers, males tend to use technologies more ...

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved