Gesture Language

The objective of this research is twofold. Firstly, we argue that gaze and gesture play an essential part in interactive explanation and that it is thus a multimodal phenomenon. Two corpora are analyzed: (1) a group of teacher novices and experts and (2) a student teacher dyad, both of whom construct explanations of students' reasoning after viewing videos of student dyads who are solving physics problems. We illustrate roles of gaze in explanations constructed within a group and roles of gesture in explanation constructed within a dyad. Secondly, we show how the analysis of such knowledge-rich empirical data pinpoints particular difficulties in designing human-computer interfaces that can support explanation between humans, or a fortiori, that can support explanation between a human and a computer.

The quantity of educational digital content available for the hearing impaired community is very scarce. However, due to extraordinary progress of the new technologies, remarkable opportunities to bring a better quality of life to the public in general arose. Making these opportunities available to those who endure handicap and disabilities is a core concern in today's society and a must to promote equity and inclusion. The target community addressed by our work, the hearing impaired community, has its own language, known as Sign language. The work presented in this paper consists in the development of a game to make the process of learning sign language enjoyable and interactive. In order to do this, a game was created in which the player controls a character and interacts with various objects and non-player characters with the aim of collecting several gestures from the Portuguese Sign Language. These gestures can then be represented by the character. This allows the user to visualize and learn or train the various existing gestures. To raise the interactivity and to make the game more interesting and motivating, several checkpoints were placed along the level. This will provide the players a chance to test the knowledge they have acquired so far on the checkpoints by using Kinect. A High Scores system was also created as well as a history to ensure that the game is a continuous motivating process as well as a learning process.

We have chosen to dedicate this volume to gestural expression used primarily by hearing speakers. The book illustrates how research-conducted with a range of different methods, on people of different ages, who use one or multiple languages, in a variety of situations, and within different theoretical frameworks-adds richness to the field of gesture studies. Within the limits of what can be done in a single volume, we have covered as many aspects of the dynamic relation between gesture and language across the lifespan as we could. But we did not cover them all. For example, we did not focus on sign languages or on the rich (dis)continuities between gesture and sign. Whether the relation between gesture and sign is presented as a "cataclysmic break" (Singleton et al., 1995; see also Brentari & Goldin-Meadow, 2017) or as naturally continuous (Müller, 2018) depends, in part, on the time span under consideration. Grammatical elements of sign languages can be traced to gestural origins over his

The quantity of educational digital content available for the hearing impaired community is very scarce. However, due to extraordinary progress of the new technologies, remarkable opportunities to bring a better quality of life to the public in general arose. Making these opportunities available to those who endure handicap and disabilities is a core concern in today's society and a must to promote equity and inclusion. The target community addressed by our work, the hearing impaired community, has its own language, known as Sign language. The work presented in this paper consists in the development of a game to make the process of learning sign language enjoyable and interactive. In order to do this, a game was created in which the player controls a character and interacts with various objects and non-player characters with the aim of collecting several gestures from the Portuguese Sign Language. These gestures can then be represented by the character. This allows the user to visualize and learn or train the various existing gestures. To raise the interactivity and to make the game more interesting and motivating, several checkpoints were placed along the level. This will provide the players a chance to test the knowledge they have acquired so far on the checkpoints by using Kinect. A High Scores system was also created as well as a history to ensure that the game is a continuous motivating process as well as a learning process.

For Wittgenstein our word language reflects human way of thinking, but a word-language is not only one possible, he was writing a lot about pictorial- or image-language, and also mentioned gesture- and sound-languages.

According to Wittgenstein, language is a vehicle of thought, however, some thoughts are inexpressible in the word-language. Through Wittgenstein’s texts, it comes out that human beings in ordinary life use pictorial language actively and conceptually. Wittgenstein considered language as ‘languages’ which are themselves systems.

‘122 “Language” is only languages, plus things I invent by analogy with existing languages. Languages are systems.’ (PG §122, 170) Hence, language is a ‘concept defined by certain systems we call “languages” and such systems as are constructed by analogy with them’ (PG, §139, 191).

There is a multifariousness of ‘languages’ (PG, §129, 179):
• word-language,
• picture-language,
• gesture-language, 
• sound-language.

In PG (93) Wittgenstein explained: ‘Words are not essential to what we call “language”, and neither are samples. Word-language is only one of many possible kinds of language, and there are transitions between one kind and another.’

In the 1934-1935 Cambridge lectures (AWL 2001, 112, Lecture XIII) Wittgenstein wrote, ‘We have three systems:
(1) the system of verbal expressions,
(2) the system of pictures,
(3) the system of actions.’ 

All these systems have the same multiplicity (ibid.). Between these systems exist projective relations ‘leading from the words to the pictures and from the pictures to the actions’ (ibid.). However, ‘Thinking is comparable in every way to drawing pictures’ (BT 2005, 67e), but ‘what the picture tells me is itself.’ It means, ‘That is, it’s telling me something consists in its own structure, in its own forms and colours’ (PG, §121, 169; italics in the original). Therefore, picture ‘narrates something to me, it is a story’ — ‘a story tells me itself’ (PG, §121, 169). When a picture narrates it means that it tells one its own story (like a description of a genre picture), so we do not compare it with the reality outside the picture. It narrates its story using words; hence, we compare the story not with the outside reality but with ‘a combination of linguistic forms’ (ibid., 170). For a long time, Wittgenstein’s later philosophy was regarded to be an example of imageless thinking, however, after the publication of all his Nachlass which contains more than 1300 pages of his own drawings and after a more detailed analysis of his texts, the situation becomes reversed. In BT (2005, 153e) Wittgenstein wrote that language has a pictorial nature and it is connected with grammar.

However, verbal language (or word-language) is the primary one. ‘I want to say over and again that primarily it is the apparatus of our language, our word-language, that we call language; and only then do we call other things language, depending on their analogy or comparability to word-language’ (BT 2005, 155e).

Wittgenstein emphasised that ‘“Talking” (whether out loud or silently) and “thinking” are not concepts of a similar kind, even though they are in closest connection’ (PI II 2009, 228e).

Wittgenstein noted an important connection between the use of verbal language, the process of thinking and pictorial images. In BT (2005, 18e) he gave an example, when we think that now it is probably three o’clock, we ‘have an image as in a toolbox of language, from which we know that we can take out the tools any time we need them.’ So, we have images of things, processes, etc., these images are linked with our language. If I think about a cat, an image of a cat may immediately appear in my mind. Such images are a part of a ‘toolbox’ of our language. However, images do not necessary accompany thinking with words. In BT (2005, 19e) Wittgenstein continued, ‘“I want to paint this spot red”, having a mental image of the colour, and “knowing” how this mental image is to be translated into reality.’

Wittgenstein distinguished not only between what could be said and shown, but also what can be written down and what not. He writes, ‘In reality one can only write down – that is, without doing something stupid & inappropriate – what arises to us in the form of writing.’ He did not clarify what this form of writing is. We may only suggest that it is a coherent, clear and concise thought. ‘Everything else seems comical & as it were like dirt.’ That is, something that needs to be wiped off.’ Then Wittgenstein added, ‘Vischer said ‘speaking is not writing’ and thinking is even less SO!’ (PPO 2003, 35). Therefore, in addition to the distinction between sayable and showable, here appear thinkable, speakable and writable.

In this tribute to Dominique Boutet and the kinesiological approach he founded, the authors have tried to make their memories of the scientific collective projects they worked on together resonate with the written work of this extraordinary scientific partner, gathered in his published articles and his habilitation document (Boutet 2018). His approach based on an intimate knowledge of the biomechanics of the human body is centered on the structuring role of the body in gestures (and signed languages). It is form-based: the form of gestures shapes their meaning or function. Gestural units are described on the basis of their formal characteristics and physiological constraints rather than their imagistic iconicity. The article presents the foundations of the approach, a synoptic description and some examples of its application. The originality of the kinesiological approach lies in the double revolution that it allows us to operate: on the one hand, gesture is not simply an appendix o...

Through constant exposure to adult input, in dialogue, children's language gradually develops into rich linguistic constructions that contain multiple cross-modal elements subtly used together for coherent communicative functions. In this chapter, we retrace children's pathways into multimodal language acquisition in a scaffolding interactional environment. We begin with the first multimodal buds children produce that contain both gestural and vocal elements and how adults' input, including reformulations and recasts, provide children with embedded model utterances they can internalize. We then show how these buds blossom into more complex constructions, focusing on the importance of creative non standard forms. Children's productions finally bloom into full multimodal intricate productions. In our last part, we focus on argument structure, Tense, Mood and Aspect and the complexification of co-verbal gestures as they are coordinated with speech.

In this tribute to Dominique Boutet and the kinesiological approach he founded, the authors have tried to make their memories of the scientific collective projects they worked on together resonate with the written work of this extraordinary scientific partner, gathered in his published articles and his habilitation document (Boutet 2018). His approach based on an intimate knowledge of the biomechanics of the human body is centered on the structuring role of the body in gestures (and signed languages). It is form-based: the form of gestures shapes their meaning or function. Gestural units are described on the basis of their formal characteristics and physiological constraints rather than their imagistic iconicity. The article presents the foundations of the approach, a synoptic description and some examples of its application. The originality of the kinesiological approach lies in the double revolution that it allows us to operate: on the one hand, gesture is not simply an appendix o...

From the perspective that language and language usage consist of both speech and gesture (McNeill 1992, 2005, 2012) a number of studies have been conducted to examine gesture's role in second language (L2) acquisition over the past thirty years. These studies have demonstrated the important role that gesture plays in both second and foreign language learning and teaching 1 (for reviews,

Although the Nâtyashâstra is acknowledged as the Indian authority in
theatrical fields, different Indian regions have produced a good number
of works about dramatic art in general or about one of its aspects, for
example the gesture language… This article presents a short text from
Kerala, the Hastalakshanadîpikâ, a treatise in Sanskrit about gestures.

We have chosen to dedicate this volume to gestural expression used primarily by hearing speakers. The book illustrates how research-conducted with a range of different methods, on people of different ages, who use one or multiple languages, in a variety of situations, and within different theoretical frameworks-adds richness to the field of gesture studies. Within the limits of what can be done in a single volume, we have covered as many aspects of the dynamic relation between gesture and language across the lifespan as we could. But we did not cover them all. For example, we did not focus on sign languages or on the rich (dis)continuities between gesture and sign. Whether the relation between gesture and sign is presented as a "cataclysmic break" see also Brentari & Goldin-Meadow, 2017) or as naturally continuous (Müller, 2018) depends, in part, on the time span under consideration. Grammatical elements of sign languages can be traced to gestural origins over historical time Wilcox, 2007, among others). But over ontogenetic time, sign and gesture have been found to be distinct vehicles (e.g., gesture-sign mismatches in signers learning math problems predict success on the problems, just as gesture-speech mismatches do; . As Cook explains in Chapter 10 of this volume, gesture and speech are generally placed in different categories because they are produced in different modalities. Gesture and sign are produced in the same modality, which often makes it difficult to separate the two. But gesture and speech also differ in the representational formats they use to convey information , and so do gesture and sign (Goldin-Meadow & Brentari, 2017), which for certain categories of gesture could be considered as imagistic and continuous forms versus conventional and categorical forms in sign, other categories of gestures (e.g., emblems), and speech. There is consequently good reason to separate the two vehicles but also good reason to pay attention to how gesture and speech or sign work together; for example, to understand the role of the visual and kinesic modalities in language or to capture the conventionalization processes from idiosyncratic to socially transmitted grammaticalized forms. The chapters in this book have taken different perspectives and used different methods and, as a result, have highlighted multiple issues. The relation between gesture and language depends on the definition of language a researcher uses (Morgenstern & Goldin-Meadow, Chapter 1). Whether gesture is considered part of language or partner to language also depends on the timespans, groups, and types of gestures studied. But all of the chapters consider gesture to be a crucial communicative and cognitive resource that serves multiple functions.

Old master paintings speak their own language, which is not conveyed by vocal words but mainly by meaningful hand gestures. Painters used them to transmit theological concepts, popular customs, scientific ideas, political values, emotional states and social identities. Despite the importance of the pictorial codified gestures in art history, noteworthy but only partial studies have been dedicated to this subject so far. The aim of this book is to propose an extensive and unifying study of the genealogy and the particularities of the language of hands in art, as well as to establish clear definitions of each one of the main pictorial chirograms (graphical representation of a hand sign). Thanks to their flexibility, the hands are able to form signs of multiples shapes and to cover a vast range of conceptions. Nevertheless, the pictorial chirograms are fundamentally different from real life gestures – they are frozen signs. Painters took advantage of this problem and created a unique well-structured and long-term communicative system. It is based on the combination of complementary signs (forming chirographic accords), the clarity of gestures shared by numerous cultures (such as the deictics), as well as the popularity of the iconographic models. The meanings of the chirograms were deciphered here using a vast repertoire of sources, mainly from social and artistic contexts of the 17th and 18th century, such as art theory, everyday life customs, liturgical rituals, theater, dance, oratory, anatomical studies, physiognomony, monastic sign language, sign language of the Deaf, etc. Around 160 gestures from different fields are addressed in the book and listed in an index. The study of chirology during the Early Modern period does not only reveal the multiple conceptions related to the hands, but also the rich iconography of drawings and engravings, illustrating this knowledge. In painting, some gestures are commonly used and widespread in Europe, while others are extremely rare. The most frequent pictorial chirograms are studied in separate chapters, including: pointed forefinger; joined palms; intertwined fingers of both hands; thumb, forefinger and middle finger raised (benedictio); raised palms; finger on the mouth (signum harpocraticum); hands crossed on the chest; extended open hand; head leaning on the hand (melancholy sign); finger of one hand touching or holding a finger of the other hand (comput); thumb inserted between the forefinger and middle finger bent (fig sign); forefinger and little finger extended (horns); fist; little finger raised; hand shake (dextrarum junctio) and self-holding hands. In addition, some rare pictorial chirograms are also presented in separate chapters, including: joint thumb and forefinger (ring gesture); thumb up or pointing thumb; hand underneath the chin; hand in the elbow (manichetto) and tense extended arms. The study of hands language in art reveals how those frozen signs relate to living cultures and contributes to the conservation of the European gesture heritage.