Teamwork in educational settings for learning and assessment has a long tradition. The reasons, g... more Teamwork in educational settings for learning and assessment has a long tradition. The reasons, goals and methods for introducing teamwork in courses may vary substantially. However, in the end, teamwork must be assessed at the group level as well as on the student level. The lecturer must be able to give students credit points or formal grades for their joint output (pro-duct) as well as for their cooperation in the team (process). Schemes for such multicriteria quantitative assessments appear difficult to define in a plausible way. Over the last five decades, numerous proposals for assessing teamwork processes and products on team and student level have been given using diverse scoring schemes. There is a broad field of empirical research and practical advice about how team-based educational assessment might be set up, implemented , improved, and accepted by staff and students. However, the underlying methodological problems with respect to the merging of several independent measurements has been severely underestimated. Here, we offer an entirely new paradigm and taxonomy of teamwork-based assessment following a rigorous fuzzy-algebraic approach based on two core notions: quasi-arithmetic means, and split-join-invariance. We will show how our novel approach solves the problem of team-peer-assessment by means of appropriate software tools.
Conventional approaches to group and peer assessment are invariably based on highly questionable ... more Conventional approaches to group and peer assessment are invariably based on highly questionable assumptions about educational measurement and how to weight and aggregate multiple scores into an overall judgement of outcome and performance. In this paper, we propose an alternative approach based on a sound theory of educational scoring and rating. Our approach is particularly relevant for software engineering education, where group projects with multiple types of outcomes are used to assess individual students. We will also demonstrate our novel approach by means of two software tools and compare them with other tools based on conventional approaches. Throughout, we will focus on explaining the reasons for and benefits of adopting our paradigm changing approach to group-peer marking.
Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful e... more Quasi-Arithmetic Scoring Theory grew out of a desire to justify peer assessment as a meaningful educational measurement technique. Peer assessment is used to evaluate group work and other collabo-rative learning activities. Students evaluate each other's participation to joint (project) assignments. The evaluations are used to split an overall team score into individual student scores to inform student grading. So far, a formal treatment of peer assessment methodology has been lacking. Traditionally, peer assessment uses bounded scales (equivalent to the standard percentage scale) together with a single n-ary operation: the weighted arithmetic mean. This practice is rather weak and questionable. Quasi-Arithmetic Scoring Theory puts peer assessment and judgmental scales in general on a sound and strong mathematical foundation. The bounded scales will be equipped with two basic operations (different from their arithmetic counterparts) so that they get the structure of a module. Peer assessment is then modelled by bivariate module theory of one bounded scale of peer ratings acting on another bounded scale of student scores. A bounded scale is an interval [ , ] of scores such that ≤ ≤ together with quasi-arithmetic operations of addition and scalar multiplication of scores, where scores may be multidimen-sional composites, i.e. consisting of subscores (items or criteria). Such bounded scales are mathematical structures called modules. Modules have the right structure for educational metrics and related statistics. The percentage scale [0,1] plays the role of standard bounded scale. All properties valid for the standard scale hold for all bounded scales, e.g. the popular Likert scales. To model peer assessment, Quasi-Arithmetic Scoring Theory needs to be extended to bivariate modules , i.e., a bounded scale of peer ratings acting upon another bounded scale of student scores. The assessor sets a group score as the default student score. Also, he specifies a scoring rule to define the action of peer ratings on student scores. The action of peer ratings on scores can be constrained to a subscale around the group score so that unrealistic deviations from the group score will be avoided. Moreover, the impact of peer ratings on scores can be made weaker or stronger There exist three distinct types of scoring rules for peer assessment. Besides the traditional arithmetic type of scoring rule, there are two types of scoring rules using modules with addition, scalar multiplication and quasi-arithmetic mean as main operations. The traditional approach is deceptively simple requiring nothing more than knowledge of arithmetic and its related concept of arithmetic average-but it is weak and inadequate, resting on questionable assumptions about the calculus of scores. The modern approach based on modules. distinguishes between quasi-additive and quasi-multiplicative scoring rules, using either one of the two module operations. All three scoring rules come with several extensions which allow fine-tuning of the underlying scoring models to tutor preferences, course spe-cifics, or university regulations.
Uploads
Papers by Suraj Ajit