Search

This paper presents the findings from a case study which explores how students engage with learning materials mediated through 360-degree panoramic videos. The case study extends findings from a pr...

This paper uses a Systemic Functional Multimodal Discourse Analysis (SF-MDA) approach to analyse tweets from the Twitter accounts of Presidents Barack Obama (@Barack Obama) and Donald Trump (@realDonaldTrump). The tweets were posted during the last nine months of President Obama’s effective presidency and the first nine months of President Trump’s presidency. The tweets are analysed using automated text analysis which is interpreted through an SF-MDA lens, supplemented by manual analysis. The analysis examines ideational and interpersonal emphasis in the tweets with the aim of showing how the composition and content construct a view of how each president and his presidency are presented to the public. The findings suggest marked contrasts in presidential style with President Trump foregrounding the interpersonal while President Obama foregrounds the ideational. Where President Trump presents as self-promoting, autocratic, opinionated and igniting discord in his tweets, President Obama presents as democratic, moderate, restrained and seeking social harmony.

This article investigates the discursive construction of international students on Australian and Chinese university webpages that are directed at international students. The international student theme webpages of three Australian and three Chinese universities were examined in relation to how verbal and visual semiotic resources were co-deployed on the webpages to address international students in the climate of neoliberal thinking in higher education. The tools used for analysis and interpretation were informed by socio-semiotic approaches to multimodal discourse analysis, international education discourses and communication accommodation theories. It was found that the Australian and Chinese university webpages differed in several salient ways to the effect that international students were portrayed as agentive and informed individuals to explore a study abroad experience at the Australian universities versus being explicitly guided through their study abroad at the Chinese universities. These results were compared and interpreted as reflecting each country’s conception of transnational education. The article concludes with a summary of the impacts of cultural and intercultural factors and neoliberal thinking in higher education on multimodal representations of international students on university webpages and directions for further research.

In their article, Ledin and Machin (2018) consider core concepts and models from systemic functional (SF) approaches to multimodality in relation to how suitable they are for analysing multimodal f...

This paper uses a social semiotic perspective to analyze Donald Trump’s domination of media coverage of the US presidential campaign from 16 June 2015, when he announced his candidacy for nomination as the Republican candidate until 8 November 2016, when he was elected as President of the United States. The paper argues that one of the keys to Donald Trump’s domination of media coverage was that, in presenting himself and his agenda, he foregrounded interpersonal meaning by making himself the focus of attention of the campaign through strategies that invaded various semiotic spaces to form a “sub-semiosphere” of Trump dogma. The effects of this were that what he did and what he said captured the majority of media attention at the expense of his opponents, enabling him to win the election, despite his complete lack of background experience as a politician.

This paper reports on a study that is part of a project which aims to develop a multimodal analytical approach for big data analytics, initially in the context of violent extremism. The fin...

This study takes a systemic functional multimodal social semiotic approach to the analysis and discussion of image and text relations in two sets of data. First, patterns of contextualisation of images and text in the online magazines Dabiq and Rumiyah produced by the Islamic extremist organisation which refers to itself as Islamic State (referred to here as ISIS) are examined. The second data set consists of a sample of texts from Western online news and blog sites which include recontextualisations of images found in the first data set. A sample of examples of the use and re-use of images is discussed in order to identify patterns of similarity and difference when images and text are recontextualised. It is argued that the ISIS material tends to foreground the interpersonal metafunction in combination with the textual metafunction (i.e. the stance towards the content and the organisation of the message for this purpose), while the other data set tends to foreground the ideational metafunction (the participants, processes and circumstances of what is being reported). These inferences indicate that further exploration of a larger data set is worth pursuing. Such studies would provide deeper insights helping to distinguish between online material which supports terrorism and that which opposes it, as well as facilitating the further development of multimodal social semiotic approaches to image and text relations.

There is a growing interest in the use of visual thinking techniques for promoting conceptual thinking in problem solving tasks as well as for reducing the complexity of ideas expressed in scientific and technical formats. The products of visual thinking, such as sketchnotes, graphics and diagrams, consist of multimodal complexes that combine language, images, mathematical symbolism and various other semiotic resources. This article adopts a social semiotic perspective, more specifically a Systemic Functional Multimodal Discourse Analysis approach, to study the underlying semiotic mechanisms through which visual thinking makes complex scientific content accessible. To illustrate the approach, the authors analyse the roles of language, images, and mathematical graphs and symbolism in four sketchnotes based on scientific literature in physics. The analysis reveals that through the process of resemiotization, where meanings are transformed from one semiotic system to another, the abstractness of specialized discourses such as physics and mathematics is reduced by multimodal strategies which include reformulating the content in terms of entities which participate in observable (i.e. tangible) processes and enhancing the reader/viewers engagement with the text. Moreover, the compositional arrangement creates clear stages in the development of the ideas and arguments that are presented. In this regard, visual thinking is a form of cultural communication through which abstract ideas are translated and explained using a multimodal outline or summary of essential parts by adapting resources (e.g. linguistic resources and mathematical graphs), using new resources (e.g. stick figures and other simple schematic drawings) and maintaining others from the original text (e.g. mathematical symbolic notation), resulting in a congruent (or concrete) depiction of abstract concepts and ideas for a non-specialist audience. © 2018, The Author(s) 2018.

This paper uses a multimodal mixed methods approach for exploring general recontextualisation patterns of violent extremist images in online media. Specifically, the paper reports on the preliminary findings of a preliminary study which investigates various patterns in the reuse of images which appear in ISIS’s official propaganda magazines Dabiq and Rumiyah by others across various public online media platforms (e.g. news websites, social media news aggregates, blogs). Using a mixed methods approach informed by multimodal discourse analysis, and combined with data mining and information visualisation, the study addresses questions such as which types of images produced and used by ISIS in its propaganda magazines recirculate most frequently in other online media over time, on which types of online media these images reappear, and in which contexts they are used and reused on these websites, that is that is, whether the tone of the message is corporate (formal) or personal (informal). Preliminary findings from the study suggest different recontextualisation patterns for certain types of ISIS-related images of over time. The study also found that the majority of violent extremist images used in the sample analysis appear to circulate most frequently on Western news and politics websites and news aggregate platforms, in predominantly formal contexts.

Two notable features of the current conflict in Syria and Iraq are the number of foreign fighters from western countries fighting for Sunni militant organisations, and the use of the Internet and s...

Multimodality, involving the study of meaning arising from the integration of language with images and other resources in multimodal texts, interactions and events, addresses the fundamental need to understand human communication in the current age of digital technology. However, multimodality is not considered to be a discipline per se at present. By drawing parallels between mathematics and linguistics, it is proposed that if multimodality is to become a discipline, then abstract context-based frameworks for modeling multimodal semiotic resources and methodologies for investigating patterns of human communication are required. An example of how this could be achieved is provided. From here, multimodality has the potential to provide the foundations for a range of multimodal sciences, in much the same way that mathematics and linguistics underpin the mathematical and language sciences respectively. In doing so, it may become possible to track the changes in human communication arising from digital technology and the resultant impact on thought and reality.

In recent years there has been increased academic and professional interest and awareness in approaches to English language teaching (ELT) that take a plurilingual approach. This is often combined with a multimodal stance. The outcome of this combination is an approach to English language teaching that integrates multiple languages and multiple semiotic resources. This paper examines how a plurilingual approach to ELT can be viewed through a multimodal lens by analyzing the construction of a plurilingual talking book created as a student project in an elementary public school. The analysis uses multimodal analysis software to map the interaction of languages and images, in order to determine how these function as meaning-making resources in a multimodal, multiple-language text created by linguistically diverse students with high ELT needs. The findings indicate how combinations of different semiotic resources work together to create meaning, delineates the role of English in meaning-making, and illustrates the children’s multilingual interactions in the creation of their collaboratively composed multimodal talking book.

This study employs a multimodal social semiotic approach to the analysis of text and image relations in material produced by the violent extremist organisation known as Islamic State in Iraq and al-Sham (ISIS). The study focuses on iconisation, where meanings are condensed and interpersonally charged through ‘bonding icons’ which embody the organisation's world view and values. A sample of issues in the online magazine Dabiq produced by ISIS are analysed using Systemic Functional Multimodal Discourse Analysis (SF-MDA). The ISIS world view is shown to be internally cohesive, based on a narrow, fundamentalist interpretation of selected Islamic scripture and violently opposed to any other world view. To represent, synthesise and justify its values and world view, ISIS uses bonding icons constructed from combinations of artefacts, supported by references to selected Islamic scripture. The study is a prelude to more detailed investigation of bonding icons in materials produced by ISIS, the attraction o...

© 2016 Taylor & Francis. This article presents a mixed methods approach for analysing text and image relations in violent extremist discourse. The approach involves integrating multimodal discourse analysis with data mining and information visualisation, resulting in theoretically informed empirical techniques for automated analysis of text and image relations in large datasets. The approach is illustrated by a study which aims to analyse how violent extremist groups use language and images to legitimise their views, incite violence, and influence recruits in online propaganda materials, and how the images from these materials are re-used in different media platforms in ways that support and resist violent extremism. The approach developed in this article contributes to what promises to be one of the key areas of research in the coming decades: namely the interdisciplinary study of big (digital) datasets of human discourse, and the implications of this for terrorism analysis and research. ispartof: TERRORISM AND POLITICAL VIOLENCE vol:31 issue:3 pages:454-474 status: published

This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scene, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic ap...

Content and Language Integrated Learning (CLIL) is a dual-focused pedagogical approach in which a foreign language is used for the learning and teaching of both content and language. CLIL specialists have recommended different types of scaffolding techniques, mainly in relation to language use. However, there is increasing interest in multimodal scaffolding techniques involving language in combination with visual resources. Within this context, visual thinking methodology is considered here as a potentially valuable tool for mediating CLIL. Using a Systemic Functional Multimodal Discourse Analysis (SF-MDA) approach, this study aims to identify several features of visual thinking that could help scaffold CLIL in the science classroom. Some strategies are proposed, both to promote acquisition of scientific language and to facilitate the development of content knowledge. The approach is explored in relation to students’ understanding and communication of complex scientific knowledge in a foreign language in upper secondary education.

Digital technology provides access to resources extending beyond the boundaries of the classroom, while offering opportunities for personalised and collaborative learning and innovation and creativity. Off-campus online learning has become a dominant trend in the higher education sector in response to rapid advances in digital technology and increased pressures of rising costs, reduced government support and fierce competition in the global education market. Universities worldwide face the challenge of reinventing themselves to survive the challenges of a changing educational environment in the digital age. However, if digital technologies, redesigned classrooms and off-campus online learning environments are to offer students the opportunity to reach their potential, we need to attain a thorough understanding of the strengths and challenges of digital technologies as pedagogical tools. This chapter presents the initial findings from an ongoing project that aims to enhance the quality of learning outcomes in online teaching and learning situations in tertiary education, by developing and trialling a web-based application for encouraging and mapping student engagement with online content through 360 degree video annotation and data analytics.

This chapter presents the findings from a pilot study which builds on and expands recontextualisation principles to analyse violent extremist propaganda materials and the multimodal contextualisation and recontextualisation of images from these sources. It discusses the initial results of a pilot project which is the first step towards the development of the mixed-methods approach. The chapter develops and uses an interactive visualisation application to investigate how the images, article types and their combinations in change over time, and to trace recontextualised images imported into Dabiq and recontextualisations of images from Dabiq across different media platforms. It describes the multimodal dataset under consideration, the background of ISIS, and some preliminary findings from the pilot project. The chapter briefly reviews the background of ISIS and their beliefs about Islam. Dabiq is published in a number of languages, including English, and is one place where ISIS makes its agenda accessible to the non-Arabic speaking world.

In 2015, the Australian Council for Educational Research (ACER) was tasked with developing literacy and numeracy skills testing for pre-service teachers. All undergraduate and postgraduate trainee teachers are now required to pass these literacy and numeracy tests at some stage on their journey to becoming a teacher; for commencing students from 2017 in all states (except New South Wales), successfully passing these tests is a requirement for graduation. Following this announcement, ACER released a number of sample test items for both literacy and numeracy. This study used multimodal analysis to study the nature of the language, symbolic notation and visual representations in the sample test items for numeracy, with a focus on the linguistic complexity of the questions. The findings suggest that the types of linguistic constructions found in the assessment questions create significant challenges in terms of reading and comprehension which could impact on how the students engage with the numeracy assessment items. Refereed/Peer-reviewed

By analysing multimodal TV advertisements, this study aims to show how intertextual voices are exploited in advertising discourse to enhance persuasive power. Taking as their point of departure the assumption that all discourses are intertextual recontextualizations of social practice that draw on external voices from both specific discourses and discursive conventions, the authors identify two types of intertextual voice in TV advertisements: character and discursive voice. This article illustrates the multimodal construction of voices and demonstrates that the choice of voices is closely related to the ‘domain’ of the product. It is argued that the intertexual voices contribute to the advertising discourse through multimodal engagement strategies. Character voice endorses the advertised product through such resources as lexico-grammar, intonation, facial expression and staged narrative, while discursive voice endorses the advertised product through contextualization and intertextual discourse structure. It is hoped that the study will shed light on the understanding of the heteroglossic nature of advertisements, the interaction between intertextual voices and the advertised message, and multimodal construction of voices and engagement.

This article uses systemic functional linguistics, particularly ideas about the relationship of language to its context, to examine the selections and functions of different sets of genres found in different curriculum areas in junior secondary education. Contrasting sets of genres taken from current textbooks in what are called technical and humanities subjects are analysed to determine which genres are favoured and to discover why they are favoured. It is concluded that the selections of genres used in different curriculum areas are functional and serve to point students in different directions: either from action to reflection or from reflection to action. It is further argued that explicit knowledge about how different curriculum areas use different genres is likely to facilitate learning and teaching.

In New South Wales secondary schools, physics, chemistry, biology, and chemistry have been taught as an integrated subject, Science, since the mid-sixties. Geography remains a separate subject area, with a focus on physical geography in thejunior secondary school. For the most part its discourse is indistinguishable from that used in science (Martin, 1990), the main difference being the absence of experiments to illustrate the scientific world view constructed in introductory textbooks. The most closely related discipline in American high schools would appear to be Earth Science. Geography as explained to junior high school students is very explicit about its goals. One junior high school textbook (Sale, Wilson, & Friedman, 1980, p. 3) states that the primary task of the geographer is to look for "order and meaning in the world." The procedures used to uncover this order and meaning are, first, to "observe and describe"; then, "to group and classify" and finally "to analyse and explain." "Analyse" is further defined as "to seek an explanation." The three stages of the geographer's task can thus be summarized as observing, ordering, and explaining the experiential world. The New Sottth Wales School Certificate Syllabtts hz Geography (Secondary Schools Board, 1984-1985, p. 6) lists the "important questions" geographers ask. These are