Word_and_image_in_contemporary_art.pdf

2.5 Word and image in contemporary art

These psychological studies of word and image processing in the anterior temporal lobes (ATLs) do not usually deal with visual artworks that juxtapose images and words. What neuroaesthetic and cognitive psychological approaches ignore consistently is a major aspect of many kinds of art: the interplay of word and image. This is not only something that has a long history in many cultural traditions (see Camille (1992) for illustrated medieval European manuscripts, Clunas (1997) for Chinese scrolls, and the journal Word and Image for many other examples), but it continues to be a major aspect of contemporary art practice.

Joseph Kosuth’s Art as Idea as Idea (Art), 1967, is a photograph ( $48 \times 48$ in) mounted on board. The artwork is visible and yet, if understood as a gallery label or dictionary entry, it is referring to art as an absent, invisible or abstract concept. It piques interest by inviting a perceptual examination of the shapes of the letters as ‘art’, yet also invites us to override this response, encouraging us to reflect on the nature of vision and art. There are many works of art that use puns and word games to present the viewer with visual paradoxes. One of the earliest examples is Magritte’s The Treachery of Images (Ceci n’est pas une pipe), 1928-1929, which, according to Zeki, ‘goes against everything the brain has seen, learnt and stored in its memory’ (1999, pp. 46-58). Yet this ‘going against’ is pleasurable and conceptually interesting. It also acts as an important and influential schema for many subsequent artworks. In Kosuth’s image, we alternate reading a text with seeing an image. We also read the words ‘human skill’ and ‘execution’ in the text presented, concepts which have been extended by Kosuth to include ‘quoting’ and recontextualising words, an act of quoting that he wants us to consider is also indicative of art. The artwork adopts a mutually reinforcing strategy: it suggests a reading between the lines and its visual equivalent, an understanding beyond optical sensation.

One could say that in Magritte’s famous painting of The Treachery of Images a relationship is suggested between the two perceptual cues, pipe and text. However, if I were to understand these perceptual cues as

Figure 15. Reconstruction of Joseph Kosuth’s Art as Idea as Idea (Art), 1967 (original, photographic enlargement on compressed styrofoam board, $451 / 2^{\prime \prime} \times 451 / 2^{\prime \prime} ; 115 \times 115 \mathrm{~cm}$ ), present location unknown.

IMAGE and WORD, I could also assume that the painting depicts the abstract concept of REPRESENTATION. Here, the conceptual blend suggests that Magritte’s painting is a representation of REPRESENTATION. The subject of the painting is a highly abstract concept. Usually, representation is used to denote something that is absent, but the painting’s text denies that it is representational in this way by stating the truth: the painting of the pipe is not a pipe. In being a representation of REPRESENTATION, it is ‘being itself’, so to speak, and is not absent. Meanwhile, the words themselves that lead us to become aware of the problem are also representational; they represent a truth that the pipe is not real, and yet the words are part of the painting. One part of the painting (the ‘text’) seems to be denying the veracity of another part (the ‘pipe’), while revealing a truth (that the pipe is a fiction). The painting thus manages to be both factual and fictional, allowing the first of these terms to state the nature of the second, but the second element also serves to give the first something to refer to. One cannot help but think that to describe or experience the painting solely in sensory terms would be to miss out on many of these twists and turns, which are

premised on a number of propositional and analogical relationships, as well as transitive inferences. Understood primarily as a complex logical problem concerning representation, Kosuth’s work would require amodal kinds of processing along with the sensorimotor action of reading (an action that, itself, could become thematic and treated conceptually).

Recent studies have also shown that artificially suppressing activity in the ATLs (using a technique known as repetitive transcranial stimulation (rTMS)) selectively disrupts semantic judgements for both words and pictures (Lambon-Ralph et al., 2007; Pobric et al., 2007). This is interpreted to mean that the anterior temporal lobe is responsible for the underlying psychological process of meaning creation common to looking at pictures and reading words. If meaning creation were restricted to the visual cortex while looking at pictures, then performance would not be affected by damage to other brain areas outside of the visual cortex. In Kosuth’s work, which relies heavily on words and images, reading becomes the subject of visual art. We read about concepts that point to our perception of the letters and words in the artwork when normally these are invisible to us. The only other times we are conscious of the shapes of the letters as a theme is with extraordinarily calligraphic feats, when the font is unusual and, even then, we might suppress this perceptual quality in order to read the script. In Kosuth’s works, the very meaning of the words points to their appearance; conception and perception (or ‘ception’, as Talmy puts it) seems difficult to separate. Martin (2007, p. 34) remarks that one of the vital areas at which research in cognitive psychology should be directed is how object conceptual and lexical representations are linked. It could do worse than beginning with the interplay of lexical and object categories brought into play by Kosuth and the scores of other contemporary artists who exploit word and image responses.

The processing of spoken words (hearing) and images (seeing) can cross modalities in reading, and this synthesis of higher-level meaning is itself not modal (but amodal), according to Visser et al.: ‘Although modality specific cortical areas play an important role in semantic processing, information needs to be combined across modalities to allow the correct conceptual relationships to be discerned independent of surface similarities. This is because semantically related objects, such as a banana and a kiwi, may be dissimilar in several key modalities (e.g., shape, color, texture)’ (2010, p. 1092). This appears to bring together and explain the kind of modal and amodal processes that would cooperate in some of the meaning creation involved in viewing Kosuth’s conceptual works, but not in all of them.

Kosuth named his works ‘analytical propositions’ where, following the early Wittgenstein, the artwork or language is supposed to enclose itself in propositional self-dependency, isolated from any external contingency making a claim to being a self-sufficient statement, visually and

conceptually self-evident. Kosuth’s Clear Square Glass Leaning (19651967) consists of four square pieces of glass upon which are placed the words of the title of the artwork. As with many of Kosuth’s works, this employs word labels to reference the material substrate upon which the words rest, in a kind of self-reflexive loop, trying to bracket out external references. However, the work also asks us to consider the phenomenal qualities of reading and how the perceptual features of the letters, their typeface and size, and the squareness and shiny aspect of the glass, its perceptual and material features, seem to become transparent when we focus on the conceptual meaning or look through the glass to the wall underneath. Four Colors Four Words is a neon artwork in four colours and although the letters do not bear any resemblance to the things to which they refer, the syntactical arrangement of four words and four colours appears to be iconic. In addition, the neon light tends to beam out a message of ‘illumination’. The difficulty that the artwork creates for us is epitomised by our hesitation in answering the question: ‘what does Four Colors Four Words refer to?’

This is similar to Jasper Johns’ various paintings of coloured words on patches of different colour, creating an interesting dissonant effect. This is interesting because there are various psychological tests that are designed to achieve a similar incongruent effect, which reveal that activation in the anterior cingulate cortex (ACC) area of the brain is associated with error detection and conflict resolution. Interestingly, the ACC is also associated with the reward system, so feelings of pleasure result from solving a problem and so help motivate subjects to seek solutions. A typical conflict-inducing experiment is the Stroop task (Pardo et al., 1990), which involves congruence: the word BLUE (written in blue) or incongruence (BLUE written in yellow). This is exactly what is involved in paintings by Jasper Johns (see his False Start, 1959). Activity in the ACC is also indicative of suppressing mind wandering; hence, Magritte, Kosuth, Nauman and Johns, among others, intuitively grasped that certain wordimage juxtapositions would hold the attention of viewers and help to structure their attention in order to create conceptual complexity as well as visual interest. This, undoubtedly, helps to explain some of the fascination that word-image art holds for us, the mutually reinforcing action of perceptual stimulation and conceptual elaboration generating complications. In many of these works, however, it is clear that perceptual experience is deliberately reduced in favour of the theme of the relevance of perception.

At higher levels of abstraction and relational knowledge, the congruence or incongruence of word and image running through many of Kosuth’s works is premised on Wittgenstein’s early logical positivism,

on the one hand, which valorised the self-sufficiency of propositions, where Kosuth attempts to suggest that art is its own reference with no hidden messages, transparently public. On the other hand, the work seems to undermine this pure conceptual logic, with the conundrum of word games, a polysemy that has more in common with the later Wittgenstein of the Philosophical Investigations (1953) where the philosopher suggested how language use and contextual situations make concepts relative. Interestingly, it also plays off a naïve understanding of direct perception (the irrefutable facticity and self-evidence of the words) and conceptual sophistication. This word-image interplay shows that Kosuth’s work involves several levels of cognitive and perceptual processing. It builds upon a system of relational knowledge that groups together what we know of his works over several years, as well as wider connections to other systems of thought involving Wittgenstein, Magritte and other artists’ works, and public signage and neon.

In Kosuth’s work and more recently in the neon works of Tracey Emin, Cerith Wyn Evans and Jason Rhoades, semantic processing occurs both at the level of the text and the image and, interestingly, at the higher level of cognition, which considers how the interplay between these different modalities functions. The left lateral temporal cortex is supposed to house a verbal language-based semantic system, while the right lateral temporal cortex represents non-verbal spatial or otherwise imageable concrete concepts. This is consistent with the dual coding system of Paivio (1986), who suggested that concrete abstracts rely more on imagery, whereas both abstract and concrete concepts can be processed in a language-like code. More recently, an extensive meta-analysis of nineteen neuroimaging studies concluded: ‘Results clearly suggest a greater engagement of the verbal system for processing of abstract concepts, and a greater engagement of the perceptual system for processing of concrete concepts’ (Wang et al., 2010, p. 1463).

I have mentioned the key role that the ATLs play in generating and organising meaning for both words and pictures. Yet, it is likely that this is supported by the prefrontal cortex and other brain areas to encode semantic networks containing abstract concepts such as REPRESENTATION. The system of relational knowledge that enhances and varies the production of meaning is made more complex by the fact that it can extend over different art forms, and the modality associated with each art form, but this seems to have been anticipated by Kosuth’s word image strategies. With his Art as Idea as Idea (Art), and in Bruce Nauman’s neons, the image-text relation can be manipulated at the level of perceptual, phenomenological and sensorimotor exploration, especially when the image part, the mounted photograph, is attended to. It is

Figure 16. Bruce Nauman, Having Fun/Good Life, Symptoms, 1985 (fabricated neon, H: $175 \times$ W: $334 \times$ D: 34 cm ). Carnegie Museum of Art, Pittsburgh. Purchase: gift of the Partners of Reed Smith Shaw and McClay and Carnegie International Acquisition Fund. © ARS, NY and DACS, London 2012.
also processed linguistically when the meanings of the words are focused on. Our categorising is skewed because we are encouraged to read about art as well as to look at it as an image. The work also seems to suggest an explanation, a speech utterance. This complex cross-modal experience situates a system of relational knowledge consisting of similar artworks, not only among different artists but within the series of works Kosuth titled First Investigations (after Wittgenstein’s Philosophical Investigations), in which Art as Idea (Art) is situated.

Conceptual production is an objective of many of Bruce Nauman’s works, although such works may also happen to be visually attractive at a superficial level. Bruce Nauman’s Having Fun/Good Life, Symptoms, 1985, consists of two sentences of coloured neon words in spirals that simulate a joint movement, the doubling of the spirals suggesting eyes that create a feeling of vertigo. Thus, the work is not body neutral; it has an effect on the body reminding us of our embodiment engaged in standing vertical while trying to read words that are upside down and while referencing words to do with affect, location and action. It also makes us conscious of how long it takes to read the work, for its phrases seem to unfurl slowly, considerably complicating the psychological process of mental rotation, word recognition and schemata involved in anticipating meaning. This phenomenal experience of reading is further complicated

by a feeling of ever-tightening sentences that seem to escape capture and the anticipation of meaning. ${ }^{1}$ The way words unexpectedly light up in groups in rapid succession induces a sense of urgency to the reading.

The artist’s choice of words represents abstract concepts grouped together to produce a good mood by using positive-sounding words on the right and negative ones on the left. Thus, abstract concepts are combined with the stimulation of affective and embodied responses, along with important spatial effects. I have discussed how the latest research in psychology shows that concepts can be distributed in both modal and amodal areas. They are sustained by coordinated operations of organising functions in frontal areas, linguistic processing areas (ATL), along with parietal areas that move around or notate modal demonstrations. Barsalou considers such a possibility when he states: ‘certain physical artefacts exhibit a kind of “perceptual productivity”, using combinatorial and recursive procedures on schematic diagrams to construct limitless complex diagrams. In architecture, notations exist for combining primitive schematic diagrams combinatorially and recursively’ (1999, pp. 592-593).

Similarly, a deceptively simple artwork such as Nauman’s appeals to at least four major systems of processing: rational and semantic, sensorimotor, spatial and emotional. The work seems to draw them out and relate them to each other in unexpected ways. However, we should not assume that visual experience here is coded along purely rational lines. On the one hand, artworks, traditionally and with modernism, have been examined for their syntactic, schematic and spatial organisation of perceptual data, which often encourages formalist art theorists such as Greenberg to value spatial and perceptual processing as a way of organising conceptual thought above all other kinds of art. On the other hand, Dada, conceptual art and contemporary art challenged the dominance of an aesthetic system based on visual formal structures as ideological rather than natural, and have attempted several strategies to resist these kinds of aesthetic judgements based on traditions of visual order. Thus, Nauman’s work carefully balances syntax with semantics as a way to encapsulate this conflict. He creates a word and image interplay that is not simply an orchestration of sensory effects. By suggesting the subpersonal mechanical processes of grammar and syntax in sentence construction, he parodies the ‘genius’ of formal design principles valorised by modernism, in purely visual

^[1]

terms: the two spirals that intersect reinstate the primacy of visual and spatial order over semantics. This means that the visual and spatial, by their very configuration, impose a meaning onto the words, which also exert their own presence and meaning by the collocation in sentences and contrasting pairs.

The colours of the neon, flashing on and off at different times, present another ordering system superimposed on the work. As in a puzzle, the work offers a structured means by which to dissect a complex experience that commands an impressively diverse range of our psychological resources spread over brain, body and situation. Given that the work can be returned to and re-examined, it helps us to think about several semiotic systems (chronological, chromatic, semantic, syntactical, spatial) and how they intersect, complications that cannot be kept in the mind easily at any one time.

The result is an iterated process of metacognition where we read the codes, examine how this affects our mental states and discover more about how the codes interact with each other. With relatively simple means, Nauman has managed to reinstate the visual into contemporary art, yet not in ways that would necessarily reinstate the traditional aesthetic values of visual pleasure and formalism involved in the contemplation of the work’s physical properties (the work seems to stimulate vertigo, if anything). Instead, we might consider that it is rewarding to elaborate aspects of various conceptual systems sparked by the artwork, based on processes of metacognition and the systematicity of language, the complexities of which are anchored in the dynamism of the artwork.

As well as the more obvious reading of the text in a linear sense going into the spiral and in the reverse direction coming out, the words can be read as they are stacked vertically above each other, cutting across the spiral. The multidirectionality of the work in its visual configuration is increased by the fact that there is an impulse to read backwards, as well as randomly, flitting from one spiral to another, picking out words and creating one’s own associations. The work demonstrates the longstanding belief in conceptual art that the viewer helps to construct the artwork. It promises to be continually manipulable and variable, not least because there are also suggestions of feelings, beliefs, parodies of advertising jargon, Pop Art references and aphorisms. This is where the structures of relational knowledge begin to emerge within the psychological response.

Nauman designed the work to light up and switch off particular phrases on each side, introducing motion and dynamism to the work, and to create a series of temporal stutters and associations that interfere with the ideal of ceding control of the word order to the viewer. Instead, as in a

conversation, we need to adapt expectations constantly, and this again shows how Nauman is intent on stressing contingencies, suggesting intersubjectivity, instead of the logical positivism of a definite message. On the right, the words suggest the cardinal directions, prepositions: up, down, in, out, which interplay with the potential mental rotations required in reading the texts. The text on the left is worrying, not only suggesting illness (fever and chills, dizziness, dryness and sweating) but also seeming to add to the difficulty of the puzzle, whereas on the right, the text is a series of self-congratulating, banal statements associated with the feelgood factor that many advertising campaigns aim to achieve (such as ‘the good life’, ‘I’m having fun’), which makes the artwork seem more lighthearted and ‘fun’; a playful puzzle. The artist thus is able to manipulate embodied, emotional, intellectual, art historical (referencing Pop Art, Op Art, Kosuth), ironic (advertising messages, hypochondria) and aesthetic responses, along with philosophical references (the parody of logical positivism, Wittgenstein and semiotics). In short, Nauman has communicated a theory of mind effectively with visual and lexical means.

In terms of organising knowledge, the work shows global, ‘betweensystems’ connections (Goldstone and Rogosky, 2002) to other artists such as Joseph Kosuth, Dan Flavin and many others who have produced neon works, as well as within-system relations where Nauman references his own previous neon works (in particular, another famous neon spiral piece with the looping sentence, The Artist Helps the World by Revealing Mystic Truths, constructed in 1967). There is also a within-work system of relational knowledge, in terms of words and concepts that can reference each other. In literary terms, the work is intertextual, an interdigitation of features of other works within new contexts and assemblages, working against the isolation of the artwork. These across-system communications also stress the social and linguistic aspects of conceptual art.

Each artwork will draw out sensorimotor processes, but the ways in which between-system considerations are staged are more likely to be amodal, coding groups and relations and types of relations rather than committing to action in any isolated, sensorimotor manner. Instead, these ways of connecting situations in art are formed by mental processes such as propositional logic, analogical modification, abstract concept combination and inference. These amodal connections allow us to shortcut or bypass the processing of sensory actions and situations in order to engage with a number of possible ways of organising relational knowledge. However, we are always at liberty to turn our attention to the perceptible qualities of the work. As with many contemporary artists, the sensual and intellectual combine but, importantly, not in order to create an indiscriminate phenomenal state. Instead, there is a structured

passage between strong and weak sensuous and emotional engagements, which we often step outside of during rational problem solving and metacognition.

Many psychologists valorise the faster processing of concepts with the aid of concrete or modal situations as the normative way that we use concepts. Yet, the difficulties involved in problem solving and organising concepts that do not employ obvious sensory input or concrete situations to grasp them are just as ‘natural’. Abstract concepts are more difficult to process than concrete concepts or concepts without situational or background representations. Nevertheless, such difficulty is pleasurable in games and puzzles, emotionally rewarding and profound in art, and is a crucial process routinely activated in study, employment, and scientific and philosophical invention and discovery. It is the rapid connectivity between tokens and exemplars in a formal and amodal sense that allows for efficient conceptual abstraction and elaboration, and the ability to experiment with possible options and to be flexible with such procedures; these are the kinds of operation Nauman’s work requires. Encounters with art and science frequently present us with opportunities to apply these rapidly connected networks of concepts against the ordinary, automatic or ‘stock’ associations between concepts that require little creativity in everyday experience.

Some of this is conceded by Barsalou and Wiemer-Hastings (2005). Quoting from lesion and neuroimaging studies that locate the processing of abstract concepts in left frontal areas, they write: ‘In these studies, participants usually receive isolated words for abstract concepts not linked to particular situations. Thus, retrieving situations should be difficult, and word associations could fill the conceptual void. Consistent with this account, left-frontal areas tend to be implicated with word generation processes’ (2005, p. 134). These areas of the brain are presumably active in the following processes, described by Wiemer-Hastings and Xu: ‘Abstract concepts are relational concepts, which are likely linked to an extensive number of other concepts…The rich connections between abstract concepts are reflected by the fact that many of their “features” are often abstract concepts themselves’ (2005, p. 733).

In a sense, the semantic theory that ‘the meaning of a concept is given by its role within its containing system’ (Goldstone et al., 2005, p. 286) suggests that the system provides meaning, and not the individual lexical units, concepts or metaphors on which Barsalou, Lakoff, Johnson and others focus in order to show perceptual grounding for each unit. As I have mentioned, Nauman’s work requires the cooperation of several containing systems that are manipulable. We can view it from withinsystem and between-systems perspectives, yet we can also understand it in

terms of syntactical (spatial) and semantic (linguistic) codes. The perceptual grounding for a sentence or conversation is much harder to justify using arguments that ignore conceptual webs and amodal organising principles. By extension, this is true for art, as we have seen in Nauman’s neon, which provides a ‘containing system’ for individual units of meaning, either for different objects depicted in a work of art or between groups of artworks, enhancing the meaning of these individual units. This would mean that both context and lexical units, as well as categories and the relational aspects of concepts, are processed rapidly and efficiently, especially if we can refer back to the artwork so as to lighten the load on working memory. In Nauman’s work, each visual code will activate different but complementary brain areas and provide a system of complex relations where words will represent abstract concepts that will be given extra meaning by their positioning within the artwork. This will stimulate spatial (parietal) and ATL activation, as well as memory areas and the prefrontal cortex that will have to integrate different stimuli.

Visser et al. (2010) suggest that the overarching meaning of a sentence, its rapid expression in speech and comprehension, or rapidly presented visual stimuli, and even their fine-grained meaning, are processed not directly by the sensorimotor system but by the ATLs. Vigliocco et al. (2009) suggest that abstract concepts are learnt implicitly or explicitly through language learning, dictionaries, definitions in speech and conversation. Sensorimotor processes need not be employed for these tasks all of the time. This suggests some sort of internal representation of word meaning of an amodal sort. Firth (1957) suggested that ‘[y]ou shall know a word by the company it keeps’, and that we learn at least part of the meaning of a word from ‘its habitual collocation’ with other words. This is an important way of learning abstract concepts based on systematic relations in domains such as sport, tools, technology and art. Art, however, can present a combination of such domains in one artwork, as we have seen in Nauman’s work, where our absorption is premised on several interconnected systems of cognition. Making explicit how these kinds of artworks produce such immersive effects on the viewer requires that we integrate studies in spatial, visual and linguistic cognition, as I have tried to do with my analysis of Kosuth’s and Nauman’s works.