Search

Successful communication – whether relayed verbally, visually, or in any other mode or mode-combination – crucially depends on cooperation between sender and recipient. Relevance theory assumes that, ceteris paribus, humans are naturally inclined to help each other and therefore attempt to optimize the chance that their fellow creatures understand them. Given the folk wisdom that “a picture tells more than a thousand words,” we may be forgiven to think that visual communication, when possible, is always preferable to its verbal variety. We should not underestimate, however, how much background knowledge is presupposed in communication via pictures or other visuals. A visual message may thus misfire because its sender misjudges the background knowledge and values of the envisaged audience. A further complicating factor is that visual (and all other) messages come with varying degrees of commitment to the meaning conveyed, this meaning ranging from being fully explicit, via being strongly or weakly suggested, to being unintentionally transmitted. Unsurprisingly, visual communication is even more challenging when it straddles different cultures. After presenting a bare-bones introduction to relevance theory, I discuss a number of exclusively or partially visual messages that involve, in one way or another, intercultural communication.

We analysed 30 English awarded and non-awarded picture books for children depicting families and gender roles, combining an intra- and intertextual corpus-driven approach. Focusing on the pictures, we systematically attempted to answer the following questions: What can be said about the child protagonist in terms of sex, skin colour, and gender preferences? Which persons in and outside of the family are depicted, and what can be said about them in terms of these variables and roles? Do we see instances of positive and negative physical contact? Where do activities take place? Do “arts” and “sports” play a role in the stories? Is the child’s imagination stimulated? In the final part we briefly discuss other themes that deserve systematic research in this genre.

While it is crystal clear that communication can draw on many semiotic resources, research in the humanities has hitherto strongly focused on its verbal manifestations. “Multimodality” labels a variety of approaches and theories trying to remedy this bias by investigating how for instance visuals, music, and sound contribute to meaning-making. The contours of what is developing into a new discipline begin to be discernible. This chapter provides a brief survey of various perspectives on multimodality, addresses the thorny issue of what should count as a mode, and makes suggestions for further development of the fledgling discipline.

Film viewers make sense of films first of all at a pre-cognitive level, triggered by their bodily responses. The key notion here is movement – the movements of screen-characters; the movements simulated by the viewers who watch and hear these characters; and the camera movements that mediate between the two. This essay review evaluates two monographs: Maarten Coëgnarts’ Embodied Cinema (2019), which expands conceptual metaphor theory to account for film’s unique affordances for communicating embodied meaning; and Vittorio Gallese and Michele Guerra’s The Empathic Screen (2019), which buttresses embodied simulation by film viewers experimentally by demonstrating the workings of “mirror neurons.” The review ends with proposals for how the findings in the two books under review tie in with other developments in charting gene-culture co-evolution.

It is often claimed that a picture tells us more than a thousand words, but studying pictorial metaphors reveals how much background knowledge is needed to understand and evaluate visuals. Commercial print advertising and billboards make for good case studies, because their goal is unambiguous: to sell consumer products and services. In this chapter some of the pitfalls in analysing visual and multimodal metaphors are discussed, Consideration of a number of examples suggests how metaphors involving visuals may misfire when they are interpreted by members from another culture than the one for which they were designed. In the conclusion some ideas are put forward to make these insights productive in educational contexts.

In order to understand the cartoons, a short characterization of Geert Wilders is called for.1 In the late 1990s, after doing several jobs within the party, Wilders became an MP for the VVD. In 2004, he left the VVD (“Volkspartij voor Vrede en Democratie”/“People’s Party for Peace and Democracy”) to found his own “Vereniging Groep Wilders”, a forerunner of the PVV (“Partij voor de Vrijheid”/“Party for Freedom”),, attracting the attention of prospective voters by voicing his opinions in contributions to national newspapers. Compared to other political leaders, Wilders appears to be represented very frequently in Dutch political cartoons, just as right-wing party leader Rita Verdonk was before him. Taking seriously Lakoff and Johnson’s claim that “metaphor is primarily a matter of thought and action, and only derivatively a matter of language”, Forceville proposed a model for visual metaphor that was based on Max Black’s “interaction theory”, but adopted the generally accepted terminology introduced by Conceptual Metaphor Theory (CMT).

This chapter discusses and reflects on the contributions in the “metaphor analysis in multimodal discourse” part of the current book, and sketches developments and challenges in CMMT (Conceptual Metaphor & Metonymy Theory). Specifically, aspiring multimodality scholars are exhorted to invest time and energy in learning about other modes and media than the one they have been trained to analyze; to familiarize themselves with other theoretical paradigms; to investigate other tropes besides metaphor and metonymy; and to be aware of the crucial importance of genre.

To help visual studies (and multimodal studies with a visual component) mature into a serious humanities discipline, it is crucial to be able to unveil patterns in the way visuals can communicate. Finding patterns requires first of all that it should be possible to identify recurring “building blocks” in visuals. Only if any recurring elements are found, it is sensible to ask whether any “rules” or “conventions” exist that prescribe how these elements can interact to create meaning – and how they cannot. In this chapter it is argued that since traffic signs constitute coded information, it is possible to use and adapt traffic signs' templates as a kind of visual "speech acts" to convey novel meanings, whose interpretation is steered and constrained by this speech act character.

In this chapter we argue that certain salient contrasts that Frederick Wiseman presents non-verbally and multimodally in his Direct Cinema documentaries can be understood as antitheses that play an argumentative role. In this type of documentary, which renounces the use of voice-over narration and music, a filmmaker has to rely on cinematography and editing in combination with participants’ spoken language to guide viewers’ interpretations. We argue that the ways in which certain sequences and shots are filmed and edited as well as the way dialogue, image, and sound combine within the shot create multimodal meanings that can be accounted for in terms of antithesis and can thereby contribute to the argument developed by the filmmaker. We focus on instances of visual and multimodal antithesis in five of Wiseman’s early films (Titicut Follies, High School, Hospital, Juvenile Court, and Primate) and one later film (Zoo).

A central claim of Lakoff and Johnson’s Conceptual Metaphor Theory is that human beings systematically understand abstract and complex phenomena in terms of concrete phenomena, the latter being phenomena that pertain to sensory perception and bodily behaviour. For this reason CMT is also known as the embodied cognition theory of metaphor. One such deeply embodied metaphor is LIFE IS A JOURNEY or, more technically, LONG-TERM PURPOSIVE ACTIVITY IS SELF-PROPELLED MOTION TOWARD A DESTINATION. Human beings often understand achieving goals in life in terms of undertaking a journey. Since metaphor is "not a figure of speech, but a mode of thought" (Lakoff 1993, 210), this metaphor should appear in non-verbal forms no less than in language. In earlier work I analysed the JOURNEY metaphor in a number of animation films. Expanding on these studies, I here focus on two of its specific dimensions: the FORCE and the BALANCE schemas, demonstrating that the metaphorical potential of these schemas is highly pertinent in journey films of the animated kind. Arguably, they moreover retain some of their impact in the medium of dance. Keywords: conceptual metaphor; visual & multimodal metaphor; LIFE IS A JOURNEY; embodied cognition; FORCE schema; BALANCE schema; animation.

Lakoff and Johnson’s (1980) pioneering work claimed that human beings think metaphorically, thereby initiating a remarkable revival of the scholarly interest in this queen of tropes. This interest reigned at first primarily among linguists. Since then, however, metaphor research has considerably broadened beyond the investigation of its verbal manifestations. Varieties involving visuals, specifically, are attracting much attention. This chapter sketches dimensions of metaphor pertaining to the medium of film. After a short historical positioning of cinematic metaphor within the wider field, key terms (target, source, mapping) are defined and explained. Central issues pertaining to metaphor in film (mode, creativity, embodiment, genre, diegetic versus nondiegetic source domain) are addressed, providing film scholars with tools for analysis. The discussions are enlightened by some examples.

"Mixed metaphors" in language use two or more different source domains to predicate something about the same target domain in a short stretch of discourse. This often leads to unintendedly humorous results and is usually considered bad style. Given that metaphors may be expressed pictorially or multimodally as well as verbally, one may ask whether non-verbal modalities can also give rise to metaphors of the "mixed" kind. If so, would such instances be considered odd, humorous, or stylistically awkward? And what, if anything, would make such "mixed metaphors" different from metaphoric blends with three input spaces (one target and two sources)? The provisional conclusion is: we should, for the time being, not adopt "mixed pictorial/multimodal metaphor" as a technical term; but the discussion provides leads for further research from which both metaphor theory and multimodal discourse analysis will benefit.

Over the past few decades, metaphor has come to be seen as a trope that governs thought, not just language. A consequence of accepting this view is that its manifestations should be examined in other media than language alone. Together with metaphor’s role in gesturing, its function in visuals has hitherto been the richest area of non-verbal research. This chapter provides an overview of issues that deserve attention in the investigation of pictorial (or: visual) metaphor and multimodal metaphor involving visuals. These issues include: monomodal versus multimodal metaphor; identifying non-verbal metaphor; creative versus structural metaphor; diegetic versus extradiegetic source domains; metaphor in static versus dynamic discourses; metaphor and other tropes.

In this paper, we explore the argumentative role of visual metaphor and visual antithesis in the so-called 'fly-on-the-wall' documentary. In this subtype of documentary, which emphatically renounces voice-over narration, the filmmakers guide their viewers into reaching certain conclusions by making choices regarding the editing as well as the cinematography. We analyse a number of scenes from two films by one major representative of the Direct Cinema or 'fly-on-the-wall' documentary, Frederick Wiseman.

The quickly growing discipline of multimodality has hitherto primarily found its inspirational models in semiotics and in Systemic Functional Linguistics. However, Cognitive Linguistics, and specifically its Conceptual Metaphor Theory branch, has over the past years proved a store of knowledge and methods of analysis that can benefit the further advance of the young discipline. In this paper the metaphor searching for one’s identity is looking for a home in animation films is examined. It is shown that (a) analysing this metaphor presupposes understanding “home” as a symbol; (b) animation has medium-specific affordances to implement the metaphor; (c) the metaphor combines embodied and cultural dimensions

Elaborating on my earlier work (Forceville 1996: chapter 5, 2005, 2009; see also Yus 2008), I will here sketch how discussions of visual and multimodal discourse can be embedded in a more general theory of communication and cognition: Sperber and Wilson’s Relevance Theory/RT (Sperber and Wilson 1986, 1995; Wilson and Sperber 2004, 2012). The focus of attention will be the visual mode, sometimes accompanied by the written verbal mode, but the idea is that the reasoning developed here is generalizable to other (combinations of) modes. Such a project should benefit both multimodality theory, which urgently needs more rigorous analytic models than have hitherto been proposed for it, and RT, which while claiming to hold for all forms of communication has been mainly applied to its spoken verbal varieties.

Metaphors present one kind of thing (a "target") in terms of another (a "source"), and are therefore ideal instruments for advertisers to make claims about products (the metaphors’ targets) efficiently and creatively. Since the intended interpretation of metaphors is often not spelled out, advertisers often get away with suggesting meanings without taking responsibility for them by making skillful use of visuals as part of metaphors. This chapter explains how visual and multimodal metaphors in advertising work, and discusses some cases to show how metaphor analysis can be a critical tool in the evaluation of advertising.

The Conceptual Metaphor Theory (CMT) fathered by George Lakoff and Mark Johnson claims that human beings do not just use metaphor in language, but actually think metaphorically. An important tenet in the theory is that ABSTRACT concepts are systematically understood in terms of CONCRETE phenomena. These latter are characterized by being accessed via sensory perception and bodily movement. For this reason, CMT is also known as a theory of "embodied cognition." In order to further investigate the ramifications of the theory and its consequences, it is crucial that conceptual metaphors are examined not just in purely verbal discourses, but also in multimodal ones, such as films. In this paper, we argue that the genre of werewolf films, a highly "embodied" film genre, draws systematically on a conceptual metaphor that can be verbalized as DEVIANT IDENTITY IS TRANSFORMED BODY. We discuss five specimens of the genre, showing that each film emphasizes different aspects of the "deviant identity." The findings will benefit both (werewolf) film theory and conceptual metaphor theory.

Comics is rapidly developing into a scholarly discipline in its own right, with a growing output of books, journals, and conferences. Two practitioners have been crucial influences. Eisner (1985) and McCloud (1993, 2000, 2006) have done much to aid the understanding of how comics create meaning. But unsurprisingly their work does not show the rigour and systematicity of academic scholarship, and this is what we aim to provide. This chapter presents a survey of visual stylistic devices specific for the medium of comics to communicate information and generate meaning. The authors discuss categories of devices, such as balloons, pictorial runes, and panel arrangements, thereby building a checklist that can help comics scholars and others interested in visuals to detect stylistic patterns and idiosyncracies. The chapter focuses on visual dimensions of comics, and has little to say about its verbal contributions; and discusses European rather than American or Japanese ones.

The Bilder-Sehen-Denken conference at Chemnitz University of Technology witnessed the foundation of the Gesellschaft für interdisziplinäre Bildwissenschaft (GIB). In this paper, I will briefly sketch where I see good opportunities for humanities scholars to contribute to the fledgling discipline. To begin with, humanities scholars have a crucial role in reflecting on the ontology of the image, on the way images help constitute or destroy power relations between (groups of) people, on the ethics of representation, and on other philosophical issues. Secondly, it is essential that humanities scholars write macro- and micro histories of images and their uses. I will not, however, comment on these two scholarly pursuits; others are far more qualified than I am to do so. My proposals pertain to a third issue which I consider indispensable for a healthy image science: the development of ›tools‹ for analysis. I present my program in the form of six suggestions, or cues, that are based on my own experiences in the field. Clearly, six is an arbitrary number, and the list has no pretense to be exhaustive, while there is also some overlap. Furthermore, my orientation is heavily biased toward English-language scholarship - but where appropriate I will briefly refer to chapters from the state-of-the art volume edited by Sachs-Hombach (2005a) in order to help build further bridges between the two language communities. Hopefully this paper provides starting points for getting to work - or else at least food for thought.

The road movie is a well-established and productive genre within film studies. (Cohan & Rae Hark 1997). The fascination with cinematographically narrating a journey, of course, resides to a considerable extent in the fact that the journey is always undertaken for a reason that goes beyond getting from A to B: there is a problem to be solved, a mission to be fulfilled, penance to be done - in short, a "quest" must be performed. It is because we capture the meaningfulness of our lives in terms of quests of various sorts that the metaphor LIFE IS A JOURNEY is so pervasive in both artistic and everyday discourse. Unsurprisingly, the "journey" is also an important theme in documentaries, particularly autobiographical ones. The journey format invites cinematographic narration, and with its inherent motion lends itself well to the dynamics of a "quest." In fact, three central dimensions of the autobiographical travel documentary - journey, quest, story - share an underlying cognitive schema: the deeply "embodied" (e.g., Gibbs 2006) "Source-Path-Goal" schema, hitherto primarily developed within cognitive linguistics (Johnson 1993, Turner 1996). This isomorphism between the three dimensions moreover allows for rich artistic ambiguity. In Forceville (2006a), the implications of this schema for three autobiographical travel documentaries by Johan van der Keuken, Ross McElwee, and Frank Cole were examined. In the chapter proposed here we will investigate how the possible interpretations of two closely related travel documentaries by Agnès Varda are both constrained and enriched by the S-P-G schema. One of the challenging factors in Varda’s films is that, contrary to those by van der Keuken, McElwee, and Cole, coincidence and happenstance are crucial themes, evoking the question whether a quest guided by chance is not a contradictio in terminis. The broader interest of this chapter is in the claim by Lakoff & Johnson (1980, 1999) that we think metaphorically, and do so systematically, by considering the multimodal discourse of film. After all, if Lakoff and Johnson are right, metaphors and other blends (Fauconnier & Turner 2002) should be demonstrable in non-verbal and multimodal representations and discourses (Forceville 1996, 1999, 2005, 2006b, Forceville & Urios-Aparisi,2009) no less than in the verbal ones that have dominated the Cognitive Metaphor Theory paradigm. Investigating non-verbal and multimodal manifestations of conceptual metaphors is crucial for uncovering aspects of such metaphors that are specific for the pictorial or sonic mode and that do not necessarily surface in language.