This innovative work introduces the interdisciplinary field of research of kinesemiotics, offering a new adaptable model and means of analysis for understanding forms of movement-based communication, such as dance, that use a codified language shared by a community of users. It begins with a theoretical overview and review of existing literature on the main approaches to movement-based communication, specifically dance, which underpin kinesemiotics as an area of study. It reaffirms previous work which established dance as a form of embodied communication in that it encompasses a wide range of semiotic styles and forms shared by communities of "speakers." In collaboration with the English National Ballet, Maiorani employs the genre of ballet as a means through which to understand and analyse some of the key concepts of kinesemiotics, mainly that of space as a semiotic dimension and "motivated movement," or movement with meaning. Supported by automated movement recognition tools from the fields of bio-robotics engineering and computer science, Maiorani argues for ballet’s capacity, when movements are projected into meaningful space, to extend beyond sequences of physical movements to become a meaning making practice. Kinesemiotics advances interdisciplinary research in the fields of social semiotics, media and communication, multimodality, linguistics, and performance studies and will be of particular interest to students and scholars in these areas.
Multimodality has most recently been described no longer as a research field or discipline on its own, but rather as a “stage of development within a field” (Bateman 2022a, 49). The realization that (1) many different fields and disciplines now enter their own multimodal phase with new interest in multimodal phenomena and that (2) these disciplines all commit to the development of multimodality research with their own theoretical principles and methodological tools, brings with it not only an immense breadth of potential analytical objects, but also many new meta-methodological issues. “We need to find ways of ‘combining’ insights from the variously imported theoretical and methodological backgrounds brought along by previous non-multimodal stages of any contributing disciplines” (Bateman 2022a, 49). At the same time, the search for a meta-methodology for multimodal analyses is pushed further by the recent trend towards more empirical approaches to multimodal phenomena and the development and use of larger multimodal corpora that just as well require theoretical and methodological refinements. “We need to develop ways of strengthening claims with robustly applicable methods which nevertheless remain firmly anchored theoretically” (Bateman 2022b, 64). For a productive handling of these issues, disciplinary triangulation and finding a ‘common language’ or metalanguage (Maton & Chen 2016) for an ‘integrationist interdisciplinarity’ (van Leeuwen 2005) are the greatest challenges in contemporary multimodality research (Bateman 2022a). Also, there is a need for reconceptualizing the practice of analysis by making available large-scale corpora and broader and more complex empirical setups to fully process the ‘move from theory to data,’ and to substantiate long-lasting theoretical and methodological hypotheses (Pflaeging et al. 2021). For this project, we see these challenges productively as “a multimodal task from the ground up,” as John Bateman (2022b, 64) has phrased it in one of his most recent papers. This Research Topic will address this task by convening the most recent theoretical, methodological, practical, and empirical developments within contemporary multimodality research. The aim is to gain new insights in • the metalanguages or external languages that are currently being developed for multimodal analysis in many different research fields and disciplines, e.g., in pedagogy, literary theory, cultural studies, design, argumentation theory, computer science, and (experimental) psychology; • newest results from data collection methods and multimodal corpus analyses that expand the current quantitative work by, e.g., applying existing theories and methods to larger datasets, or exploring the newest communication technologies. We are particularly interested in seeing how works addressing these aspects contribute to finding ways of productive triangulation and integration for and within a meta-methodology for multimodality research. This Research Topic aims to bring together scholars from a variety of disciplines interested in multimodality research to review, explore, and advance the contributions that John Bateman, as one of the key figures in multimodality research, has made to both theory- and method-building as well as to the driving forward of multimodal empirical and corpus analyses. We welcome contributions that, for example, • critically address the theoretical and methodological advancements that John Bateman has made with regard to the notions of semiotic mode, discourse semantics, genre, textuality, etc.; • apply one of the many approaches that John Bateman has developed for the empirical analysis of multimodal artefacts (e.g., the GeM model for page-based documents, his work on multimodal film and audio-visual analysis, and the discourse semantics and/or annotation approach to visual narratives) to larger corpora or currently newly developing communicative situations; • expand on one of the abovementioned aspects with new ideas and insights from disciplines that have not yet been included in multimodality research.
This book is a first attempt to map the broad context of performance studies from a multimodal perspective. It collects original research on traditional performing arts (theatre, dance, opera), live (durational performance) and mediated/recorded performances (films, television shows), as well as performative discursive practices on social media by adopting several theories and methodologies all dealing with the notion of multimodality. As a mostly dynamic and also interactive environment for various text types and genres, the context of performance studies provides many opportunities to produce meaning verbally and non-verbally. All chapters in this book develop frameworks for the analysis of performance-related events and activities and explore empirical case studies in a range of different ages and cultures. A further focus lies on the communicative strategies deployed by different communities of practice, taking into account processes of production, distribution, and consumption of such texts in diverse spatial and temporal contexts.
Looking at both English Medium Instruction (EMI) and multimodality in higher education, this edited volume bridges the gap between the two contexts by offering various new insights into fundamentals in multilingual education, EMI discourse and current teaching practices in internationalised contexts. Current demands in communication, especially in higher-education contexts, require examining EMI from a multimodal perspective with the aim of giving explicit attention to modern discourse practices. The contributors reflect on the principles guiding EMI and multimodality and their application in higher education using both practical examples and data-driven evidences. They discuss EMI multimodal discourse from an empirical perspective to unveil communicative practices in internationalised higher-education contexts; and exemplify classroom applications and ESP and EAP pedagogical practices that promote multimodal competence in higher education. The contributors provide solid theoretical foundations, key principles, research evidence and pedagogical implications that inform current methodologies and practices for EMI, ESP and EAP, as well as multimodality in higher education. This volume on EMI and multimodality in higher education will have broad appeal for researchers worldwide from various fields of expertise within education and applied linguistics.
This collection explores the relationships between theory and evidences in functional linguistics, bringing together perspectives from both established and emerging scholars. The volume begins by establishing theoretical common ground for functional approaches to language, critically discussing empirical inquiry in functional linguistics and the challenges and opportunities of using new technologies in linguistic investigations. Building on this foundation, the second part of the volume explores the challenges involved in using different data sources as evidence for theorizing language and linguistic processes, drawing on work on lexical cohesion in language variation, neuroimaging and neuropathological data, and keystroke logging and eye-tracking. The final section of the volume examines the ways in which evidences from a wide range of data sources can offer new perspectives toward challenging established theoretical claims, employing empirical evidences from corpus linguistic analysis, keystroke logging, and multimodal communication. This pioneering collection synthesizes perspectives and addresses fundamental questions in the investigation of the relationships between theory and evidences in functional linguistics and will be of particular interest to researchers working in the field, as well as linguists working in experimental and interdisciplinary approaches which seek to bridge this gap.
This book looks to expand the definition of translation in line with Susan Bassnett and David Johnston’s notion of the “outward turn”, applying this perspective to contemporary art to broaden the scope of how we understand translation in today’s global multisemiotic world. The book takes as its point of departure the idea that texts are comprised of not only words but other semiotic systems and therefore expanding our notions of both language and translation can better equip us to translate stories told via non-traditional means in novel ways. While the “outward turn” has been analyzed in literature, Vidal directs this spotlight to contemporary art, a field which has already engaged in disciplinary connections with Translation Studies. The volume highlights how the unpacking of such connections between disciplines encourages engagement with contemporary social issues, around identity, power, migration, and globalization, and in turn, new ways of thinking and bringing about wider cultural change. This innovative book will be of interest to scholars in translation studies and contemporary art.
This collection explores the mediation of a wide range of processes, texts, and practices in contemporary digital environments through the lens of a multimodal theory of communication. Bringing together contributions from renowned scholars in the field, the book builds on the notion that any form of digital communication inherently presents a rich combination of different semiotic modes and resources as a jumping-off point from which to critically reflect on digital mediation from three different perspectives. The first section looks at social and semiotic practices and the implications of their mediation on artistic production, cultural heritage, and commerce. The second part of the volume focuses on dynamics of awareness, cognition, and identity formation in participants to digitally-mediated communicative processes. The book’s final section considers the impact of mediation on shaping new and different types of textualities and genres in digital spaces. The book will be of particular interest to scholars, researchers and students in multimodality, digital communication, social semiotics, and media studies.
This book offers an innovative, unified theoretical model for better understanding the processes underpinning naming and framing and the power that words exert over human minds. The volume integrates theoretical paradigms and empirical insights from across a broad array of research disciplines, several of which have not been combined before, and uses this foundation as a point of departure for introducing its four-layered model of distinct but connected levels of analysis. Bringing together insights from cognitive linguistics and psycholinguistics together with multimodal perspectives, Smith establishes new cross-disciplinary links, further integrating work from neighbouring fields such as marketing, health communication, and political communication, that indicate paths for future research and implications for communicative ethics. This book will be of particular interest to students and scholars in multimodality, communication, semiotics, cognitive psychology, and linguistics, as well as those in related disciplines such as marketing, political communication, and health communication.
This collection critically considers the question of how learning and teaching should be conceived, understood, and approached in light of the changing nature of learning scenarios and new pedagogies in this current age of multimodal digital texts, practices, and communities. The book takes the concept of digital artifacts as being composed of multiple meaning-making semiotic resources, such as visuals, music, and design, as its point of departure to explore how diverse communities interact with these tools and develop and explore their understanding of digital practices in learning contexts. The first section of the volume examines different case studies in which involved participants learn to grapple with the introduction of digital tools for learning in children’s early years of schooling. The second section extends the focus to secondary and higher education settings as digital learning tools grow more complex as do students, parents, and teachers’ interactions with them and the subsequent need for new pedagogies to rethink these multimodal artifacts. A final section reflects on the implications of new multimodal tools, technologies, and pedagogies for teachers, such as on teacher training and community building among educators. In its in-depth look at multimodal approaches to learning as meaning-making in a digital world, this book will be of interest to students and scholars in multimodality, English language teaching, digital communication, and education.
