Georgina Barton

Multimodal texts surround us. What are they? How can we use them in our teaching?


 The ways in which we communicate with each other in today’s world are wide ranging. We live in a time where politicians tweet national policy announcements, a YouTuber can have 75 million subscribers from around the world, and pre-teen children communicate using images on Instagram. It seems strange then, that assessment practices in schools largely remain focused on traditional written texts such as essays and reports. These texts often involve only language mode despite there being other modes that can be effectively used to express meaning. By other modes I mean communication including things like images, sounds, signs and gestures. When a text uses two or more modes we call it a multimodal text.

I have been researching how teachers use and teach multimodal texts and I believe Australia needs to update the way we understand multimodality in our schools and how we assess our students across the curriculum.

What are modes and what is multimodality?

A mode is a socially and culturally-shaped resource for meaning-making. Modes include not only language but gesture, image, sound and space. In fact, digital platforms such as Instagram and Snapchat rely on communication through pictures. Instagram is a photo and video sharing app and Snapchat is a messaging app that lets users exchange pictures and videos, called snaps, that are meant to disappear after they are viewed.

Other forms of communication might include ensembles of modes such as movement and sound. How these are orchestrated determines the intent and main message. A film for example contains a character’s voice and gestures as well as the space in which they move, not to mention the film score. Another example could be a sculpture that uses visual and spatial conventions. This is known as multimodality and it is important that teachers in schools teach all of the modes so all students can succeed.

 My research has revealed that teachers understand multimodal texts, defined as a combination of two or more communication modes by ACARA, as texts only involving digital technologies. But they can be much more than this. Cultural theorists understand texts as cultural artefacts—meaning even an ancient ceramic pot can be considered a multimodal text as it shares knowledge and meaning for a viewer through its shape, markings on it as well as the compositional material. Therefore, multimodal texts are more than just iMovie trailers.

How to plan for and teach authentic multimodal texts

 It is important that students can effectively communicate through both oral and written language, not just within the subject of English, but in other subjects as well. The Australian curriculum now expects students to be able to comprehend and compose multimodal texts in curriculum areas such as science, history and the arts.

Planning for teaching that supports students’ understanding and knowledge of literacy demands in texts that use a range of modes is critical for student to be able to compose multimodal texts effectively. These literacy demands include the codes and conventions associated with each mode.

When students learn about the codes and conventions, their meta-language (the way they can talk about the codes and conventions) improves and this transfers to their compositions.

Knowing how to create cohesive multimodal texts includes knowing how the modes link and talk to each other. For example, visual image and text in children’s pictures books work together to share the narrative. Curriculum planning therefore needs to ensure inclusion of content about a range of texts in each curriculum area as well as how different modes are used in these texts.

Effective teaching methods are also needed to teach critical multimodal literacies. These approaches to teaching relate to both the comprehension and composition of multimodal texts. Models of teaching need to be age-appropriate and address students’ personal, social and cultural needs. They also need to support students in being able to read texts critically through inferential comprehension.

How students can be effective readers and producers of multimodal texts

Students need to be able to read multimodal texts. A proficient reader of multimodal texts will more likely be a more effective composer of multimodal texts.

Here is my list of what I believe proficient readers of multimodal texts should be able to do.

  •  Use prior and new knowledge to share and understand meaning through all the modes
  • Make connections between their own experience and others
  • Develop metalanguage related to all of the modes
  • Draw appropriate and diverse conclusions from a range of texts
  • Make predictions through reading, testing and revising
  • Create different texts as a result of interpreting other texts
  • Critique what multimodal texts they read.

Once students are able to analyse and talk about multimodal texts they can then more effectively create their own.

Assessment related to composing multimodal texts

Varying assessment and options for students is empowering and makes them at the centre of learning. Multimodal texts and assessment therefore can be the catalyst in creating authentic and engaging assessment tasks that students write for purpose and with a particular audience in mind.

“Different modes demand different intellectual work from pupils and this work ‘fills up’ the concepts to be learnt in different ways. The range of representational resources made available through visual communication (spatial relations, colour, etc.), for example, enable the expression of kinds of meaning that would be difficult, or perhaps impossible, in language (Jewitt et al., 2000)”(p. 84).

It is therefore critical that educators consider ways in which to vary their assessment to meet the needs of their students. Referred to as ‘multimodal reshaping’ teachers can offer students a range of options in terms of assessment that address the same criteria. It ensures students have a choice and voice and can therefore express their meaning via a range of modes and ensembles of modes.

For example, if students are required to present an argument for or against climate change they could do so through embodiment (dance or drama), an artwork, a sequence of photographs, a newspaper report, a blog or a 3D model such as a diorama.

 What are some examples of multimodal texts across the curriculum?

English Stories that include text and images, newspaper articles, photographs, memes, comic strips, dramatisations
Science Scientific illustrations or animations, 3D models, ‘How To’ guides with pictures, museum exhibits
History Biographical portraits, dioramas, replicas of primary sources, pamphlets, posters, mobiles
The arts Sculptures, performances – dramatic and musical, soundscapes, choreographed works, advertisements

My research with teachers and students into the use and teaching of multimodal texts continues. In the meantime, I hope we can see assessment practices in schools capture more effectively some of the wide-ranging ways we communicate with each other in the world today.

Georgina Barton is Associate Professor (Literacies and Pedagogy) in the School of Teacher Education and Early Childhood at the University of Southern Queensland. She was a teacher in schools for over 20 years with experience as an Acting Principal,lead literacy intervention teacher and Head of Department, the Arts. She also spent time in South India teaching English. She has over 80 publications in the areas of literacy, multiliteracies, multimodalities, the arts and culturally and linguistically diverse contexts including internationalisation. She is currently involved in an Australian Research Council grant led by Professor Mary Ryan that explores effective and reflexive approaches to teaching writing. She is a Fellow of the Australian Literacy Educators’ Association and was the Conference Chair of the Australian Association for Research in Education (AARE)2017-2018. Georgina is on Twitter at @BartonGeorgina


‘Invisible’ literacies are literacies for the future. What are they? Why is teaching them vital?

“We have so much pressure on us to teach literacy in our classrooms. The arts are not valued at all except when it comes to public relations and open day”.(Arts teacher)

Improvement in literacy is high stakes in education today. Educators are constantly told literacy results are not good enough and that more testing will help solve the problem.

Tests can indeed tell us how students are achieving at one point in time in traditional or ‘visible’ literacies such as reading and writing text, spelling, punctuation and other language conventions. These are, of course, important basic literacies. However the constant literacy testing in schools focuses on a narrow conception of literacy and literacy skills. It leads to teaching to the test in many classrooms, and significantly it overlooks vital ‘invisible’ literacies that are used in the world today.

We believe these invisible literacies need to be given more prominence and practice in schools and should be more readily recognised by policy makers, curriculum designers and educational leaders as essential to every day life in the 21stcentury.

Our research interest lies specifically in the range of literacies involved in the arts and we have collaborated in a study to look more closely at what is happening in Australia, France and Canada with arts literacies. Our research findings have wide implications for future classroom practice.

What are ‘invisible’ literacies?

Disciplinespecific literacies (literacies that are specifically needed in a discipline, for example, by an historian or a lawyer, in order for them to work effectively in their fields) have been the focus of much research. Content area specialists build knowledge in their field. They use ‘invisible’ and important literacies that go beyond traditional writing text, spelling, punctuation and other conventional literacies. These include other modes (such as collaboration or demonstration) or semiotics (signs, marks) present in different subject areas.

In orderto become an ‘artist’, a ‘scientist’ or an ‘historian’ in the classroom students need to know how knowledge is built through sophisticated uses of a range of modes in the way that such specialists do. Students need to understand the literacy demands placed upon them when reading historical artefacts, scientific reports or laboratory work and/or artworks such as sculptures or musical scores. They also need to know how such specialists go about their work and make meaning of information through various communicative methods.

Invisible literacies in the arts

In the arts for example, teachers support students to interpret others’ artworks and create their own. Students work towards being an artist by the end of their schooling by critically viewing and discussing artistic-aesthetic elements of art.

Arts-literacies are important because they can enable unique ways of looking at the world that aren’t available in any other subject area. American education philosopher, Maxine Greene, calls this way of knowing ‘being wide awake’. Arts literacies help students develop design-thinking, creativity and critical thinking—all skills said to be important for the future workforce.

Artistic practices intrinsically involve the reception-production of “signs” in a continuous process of “translation” from one ‘language’ to another. Of course, being able to talk, read and write about the arts and arts practice is important to the artistic process, however an artist uses much more than visible traditional literacies to create meaning.

They use arts-specific vocabulary, metaphors, embodiment, and other more demanding ways to express themselves through using their art. Additionally, collaboration and sharing are important aspects of arts classrooms. These literacies however, remain invisible, particularly in schools where high stakes tests are used constantly to check up on traditional visible literacies.

Our curriculums acknowledge the need for invisible literacies

In Australia, Canada and France school curricula acknowledge invisible literacies.

The Australian Curriculum for example highlights in the General Capability: Literacy that all teachers are teachers of literacy. The key concepts in this capability are given as text, grammar, word and visual knowledge. (Other modes such as aural, gestural and spatial that are used when we communicate with each other should be included here.) The arts curriculum includes the notions of responding to and making art.

In Canada, the digital arts have been a strong focus in the curriculum, emphasising how artists, arts professionals, and arts community organisations integrate digital tools as part of their practice. The Canadian curriculum is looking to the future on virtual reality and augmented reality for economic and social growth, as well as continuing their recent teaching and research focus on maker education (problem solving or project based learning)

Art and Cultural Education (ACE) in France, allows students to go beyond explore ‘les enseignements artistiques’ or artistic teachings allowed a common good for ‘classical’ school subjects. Two intertwined educational purposes include education about art,whichis about students’ acquisition of genuine artistic knowledge, and education through art, which allows studentsto develop themselves as cultural citizens that contribute to society.

But our research has revealed that teachers are being pressured to teach and practice traditional literacies at the expense of these important arts-literacies.

Why is it important to pay more attention to these ‘invisible’ literacies?

In our research, we argue that certain approaches to literacy are stifling teachers’ work within their disciplines. Rather than continuing to focus only on traditional literacies such as reading and writing in language/linguistic mode, we believe it is time to make ‘invisible’ literacies visible, that is, acknowledged and valued by schools.

Researchers around the world have spent decades looking at the relationships between the arts, well-being, and the ways we perform literacies on a daily basis. Philosophers of education such as Maxine Greene and Elliot Eisner have defended the need to pay more attention to the arts in both teaching and learning, and research, as it would have positive impact in the schooling of all students.  In classrooms it would mean more focus on important general capabilities such as personal and social capabilities, critical and creative thinking, and ethical and intercultural understanding.

More recently in Canada, work by Brock University Professor Jennifer Rowsell on the relationships between humans, literacy, and the arts as part of the community arts zone project has expanded the way educators understand the teaching of literacy and at the same time revitalised the literacy and arts community in Southern Ontario.

The work of this project can be summed up by one of the participating teachers:

I treat my classroom like it’s a studio, so it’s a place for them to work, so I give them the power…that they are the artists coming into this space, using this as a space, as an opportunity for them to create, and the time to create, and to dialogue with their peers about what they enjoy, and you know, be up to their necks in the creative process without any other distractions and I think that’s really important for them to grow. (CAZ visual arts teacher participant, December, 2013)

As Eisner said, “…the distinctive forms of thinking needed to create artistically crafted work are relevant not only to what students do, they are relevant to virtually all aspects of what we do, from the design of curricula, to the practice of teaching, to the features of the environment in which students and teachers live.”

In other words, communities develop their meaning-making skills by “doing stuff,” and playing with materials, sounds, video, images in order to make sense of the world and engage in contemporary understandings of what reading and writing is in curriculum studies.

What will happen if we don’t make ‘invisible’ literacies ‘visible’?

This necessary question invites a collateral one, which is: what happens when we make invisible literacies visible?  Canadian educators Professor Jennifer Rowsell, and Professor Maureen Kendrick brilliantly argued that the need to make invisible literacies visible depends on teachers understanding and using them in valid ways to articulate students’ motivations, goals, needs, and interests.

To the question what will happen if teachers and researchers do not adopt these types of practices, we answer that there is a high probability that educators will further marginalise young people who struggle with finding their “best mode” or modes of communication so they can articulate their motivations, goals, needs, and interests. As established researchers have found, these modes are often hidden.

For example, sociology of education research has observed how there is a ‘hidden curriculum’ for both students and teachers. That is, there are ways of interacting that implicitly support cultural and social awareness and encourage personal and professional growth, but are not stated in any curriculum. It includes how teachers ‘tune in’ to their particular student or students, and understand and encourage the different ways they can or may want to express themselves and communicate with others.

Maker education (problem based or project based learning) has also been a recent example of how literacies can be hidden. Maker work can include making meaning (using invisible literacies) by playing with sound, building structures, coding and playing with software, making 3D impressions. We must then ask what are the implications for post-millenial students in this day and age? What audiences are they making for, and what kind of citizens do they aspire to be?

Impact on the future

If we continue to focus on narrowed conceptions of literacy we are at risk of creating an educative space that is restrictive for students, particularly those from socio-culturally and linguistically diverse backgrounds. The research has, for some time, suggested that young people need to have the opportunities to choose the best method of expressing meaning for themselves in assessment practices. Without acknowledging the inherent skills of students as well as what ‘could be’ through their engagement with the arts we are indeed providing a deficit approach (an approach that is based on labeling weaknesses or failures, in tests for example) to learning and teaching rather than one that supports the implicit and invisible and critical work needing to be done in the classroom.

 

 

Georgina Barton is Associate Professor (Literacies and Pedagogy) in the School of Teacher Education and Early Childhood at the University of Southern Queensland. She was a teacher in schools for over 20 years with experience as an Acting Principal, lead literacy intervention teacher and Head of Department, the Arts. She also spent time in South India teaching English and learning Carnatic music. She has over 80 publications in the areas of literacy, multiliteracies, multimodalities, the arts and culturally and linguistically diverse contexts including internationalisation. Between 2014-2016 she led an Office for Learning and Teaching innovation and development grant exploring international students’ work place experiences. She is a Fellow of the Australian Literacy Educators’ Association and is Conference Chair of the Australian Association for Research in Education (AARE) 2017-2019. Georgina is on Twitter at @BartonGeorgina

Amélie Lemieux is Assistant Professor of Literacies at Mount Saint Vincent University. Her research interests revolve around visual and written literacies, aesthetics in art and literacy education, and literacy practices in the 21st century. Her original contributions to literacy and aesthetic reception were recognized by the Social Sciences and Humanities Research Council of Canada (SSHRC), especially with the Joseph-Armand Bombardier Scholarships both at the Master’s and PhD levels. She completed a Postdoctoral Research Fellowship at the Centre for Research in Multiliteracies, Brock University. She now works towards shaping debates on the relevance of investigating and valuing subjectivity as a main denominator of meaning-making in multimodal literacies, and intends on further informing pedagogical practices in the literature classroom, specifically in light of fulfilling the competency of “exploring diverse texts of literature”, both at the college and high school levels. Amélie is on Twitter at @ame_lemieux

Jean-Charles Chabanne is Professor of Educational Science at the École Normale Supérieure’s French Institute of Education in Lyon.His academic background is French language and literature with a initial focus on literary and linguistic approaches to humour. He went on to research how language, literature and writing is taught, how language interacts with other disciplines, and how language serves as a form of knowledge, as a tool for thinking and learning, and as a working tool for teachers and mediators.He has directed the LIRDEF research lab (Interdisciplinary Laboratory in Teaching, Education and Training) in Montpellier, and currently leads the scientific programme Alféa(Arts, Language, Training, Teaching, Learning) at the ENS-IFE in Lyon. This programme investigates the form, place and function of language (verbal and other semiotic systems) within learning contexts involving art education or aesthetic experience. His work is situated in the shared space between art/cultural education and the fields of language, linguistics and literature.