digital technology in schools

Using evidence to help build and evaluate good ideas in education technology

As researchers, we care that our educational systems improve, support all learners, and are grounded solidly in research evidence. But how do we work with stakeholders like educational technology startups to support effective use of that evidence? Researchers and practitioners worry about this, because we care about evaluating and scaling good ideas. By ‘scaling’ we mean adjusting and improving good ideas as they are rolled out and used.

Some common ways that people think about how we build evidence and scale innovations include:

  • taking approaches tested in controlled settings and implementing them
  • looking for ‘success stories’ and trying to copy lessons from them and
  • taking a systematic approach to analyse context for places to change and evaluating these changes, the Improvement Method.

Research on how we use evidence in policy and practice (and policy practice) can help inform us when we try to work with startups and other stakeholders on education projects. Professor of Politics and Public Policy at the University of Stirling in the UK, Paul Cairney, compares the three approaches in the table below.

Three approaches to evidence-based policy-making

Emulate Approach

In much work in education, we are looking to implement programs or technologies in contexts using an emulation approach; copying tested interventions. In our teaching that can also result in coming at research from a top down perspective, using key studies and methods but with a disconnect from local needs and context.

But these interventions are critiqued for their simplicity in the education context because they imply that interventions occur in a vacuum rather than in a complex context where we’ve already got lots of interventions going on. We might be evaluating a program that has already been implemented, and often our implementation process doesn’t follow this linear model.  

Storytelling Approach

The push back against the emulation approach is sometimes to instead focus very heavily on local context and storytelling approaches. This approach respects the expertise of professionals – which is important – but can result in key lessons not being distilled and shared, idiosyncratic ‘hit or miss’ practices, and ad hoc improvement cycles that may be driven by particular interests.

In the edtech space, much of the evaluation conducted by providers is based on testimonials. Although these can be useful, they’re typically not going to get at deeper issues of learning or help us evaluate our work. 

Improvement Methods

So, then, Improvement methods have been adopted in education systems, for example explicitly by the Carnegie Foundation, an independent research and policy centre in the US, and arguably in other forms such as Research Practice Partnerships (which are collaborative, long-term relationships between researchers and practitioners, designed to improve problems of practice in education) and other design based research approaches. Because these approaches work closely with practitioners to connect theory and real-world problems, they attempt to avoid ‘transmissive’ communication (one way communication) of research.

Our UCL EDUCATE project

At UCL (University College London) – which Simon recently visited while on sabbatical – the EDUCATE project has been created to help build a stronger evidence base in the EdTech sector. It uses this kind of approach. The approach is visualised through the ‘golden triangle’ connecting EdTech companies, entrepreneurs and start-ups with first-class business trainers, experts and mentors.

https://lh5.googleusercontent.com/rxgBpc9p3QFcI84iFhsNMfVQKihSpezX15eIFHz-v4HXTgpJqY8XBgQiKhoZuPfmePhnAm5w2GiuHkcvYJEwa3hJehhV_ELuRy1cxACayLhSl8IU4JPoe07rwx1T5DLLN6T9G9cF

The Golden Triangle of evidence-informed educational technology

The UCL EDUCATE project worked with 252 small to medium-sized enterprises (max 250 employees, <£5m annual turnover) in 12 cohorts between 2017-19.  The idea was to get EdTech creators, educators, investors and policy makers working together to understand what “works for learners and how to use technology to serve its users effectively.” As the program developed, it shifted from more general introductions to research methods and established research knowledge, to greater recognition that the nature of evidence is both varied and serves different purposes for enterprises at different stages of development.

The EDUCATE programme avoided the issue of transmissive or emulation-based research by building capacity in educational technology enterprises to conduct their own research, using theories of change to generate practical, robust, research. The aim, then, isn’t just to translate research into practice, or implement outcomes from RCTs, but to try and move from storytelling about products, to an improvement mindset. 

UTS Implementing Learning Analytics

In the work we’ve been conducting at the University of Technology Sydney we’ve taken a kind of improvement based approach, by looking at existing teaching practices, and seeking to augment those practices, rather than simply dropping in a new technology without understanding the context, or with a requirement for a particular type of teaching for it to be used.  Our focus is improvement-oriented innovation. This approach is intended to improve adoption and support existing good practices by learning from them and to amplify them through the technology. 

We believe it is important, when we think about the role of new technologies and approaches in education, to consider the way we use evidence. Understanding the different approaches – implementation, storytelling, or improvement – and how they work to achieve impact can be invaluable to all stakeholders.

Simon Knight is a Senior Lecturer at the Faculty of Transdisciplinary Innovation at the University of Technology, Sydney. His research investigates how people find and evaluate evidence, particularly in the context of learning and educator practices. Dr Knight received his Bachelor’s degree in Philosophy and Psychology from the University of Leeds before completing a teacher education program and Philosophy of Education MA at the UCL Institute of Education. Following teaching high school social sciences, Dr Knight completed an MPhil in Educational Research Methods at Cambridge, and PhD in Learning Analytics at the UK Open University. Simon is on Twitter @sjgknight

Anissa Moeini is a doctoral candidate at the UCL Knowledge Lab, Institute of Education, University College London, UK. As a seasoned tech entrepreneur, Anissa identified the need to build research capacity in edtech enterprises that is both agile to their pace of change and also adaptable to the rhythm of SMEs. Through her doctoral research she developed the Evidence-informed Learning Technology Enterprise Framework (ELTE) as a practical tool for edtech companies and other non-academic stakeholders (investors, policymakers and education practitioners)  to both evaluate the efficacy of edtech enterprises (i.e. their products and services) and to build capacity to be evidence-informed.  Anissa completed her MA at Teachers College, Columbia University in NY, USA and her iBBA at the Schulich School of Business in Toronto, Canada. She will be defending her doctoral dissertation in 2020. Anissa is on Twitter @AnissaMoeini

Alison Clark-Wilson is a Principal Research Fellow at UCL Knowledge Lab, UCL Institute of Education, London. Her research spans the EdTech sector with a particular emphasis on the design, implementation and evaluation of technology in real school settings. Dr Clark-Wilson received a Bachelor’s degree in Chemical Engineering prior to becoming a secondary school mathematics teacher in the early 1990s. Her 30-year career has spanned school, university and industry-based education contexts. Dr Clark-Wilson completed a MA at the University of Chichester and a PhD from UCL Institute of Education, both in mathematics education. Alison is on Twitter @Aliclarkwilson

 

EdTech is killing us all: facing up to the environmental consequences of digital education

Digital technology is now a major part of education. Even the smallest schools are stuffed full of digital devices, display screens and projectors. Anything that can be digitized is stored online. Lessons are live-streamed, resources are downloadable, and school communications take place through apps and email. Behind the scenes, schools maintain their own servers, host campus-wide Wi-Fi and run complex management systems and other platforms. All told, schooling today is dependent on substantial amounts of digital technology.

This digital dependency is rarely seen as a problem. Any gripes usually centre on potential risks of digital distraction, cyberbullying, breaches of data privacy and so on. These issues prompt vigorous debates over the ‘appropriate’ and ‘right’ ways in which technology should be implemented. At no point, however, is there serious consideration of the long-term sustainability of digital technology use.

To be blunt, digital technology is damaging the environment. I believe the use of digital technology in education (EdTech) is not sustainable in the ways we have grown accustomed to using it.

First, let us dispel any thoughts that the increased use of digital technology in schools is somehow environmentally beneficial. For sure, there are obvious environmental benefits in reduced paper use, using Skype to attend remote meetings, and installing ‘green tech’ such as smart lighting and smart metering. It might also be argued that online classes reduce the carbon footprint of schools and colleges, not least by reducing travel-related emissions of students coming onto campus.

All these technologies uses offer some recompense, but they in no way offset the hugely detrimental life-cycles of the digital products and processes that education is now reliant on. Instead, the end-to-end environmental consequences of any form of digital technology use quickly eclipse any hopes of digital education somehow being a green option. As such, every use of digital technology contributes to the degradation of our planet in ways that education urgently needs to face up to. This includes:

  • The raw ingredients of digital devices – what Toby Miller terms the “dirty, material origins” of digital technology. Behind the sleek chrome and glass exteriors, every digital device is constructed from dozens of different metals, and numerous ‘rare earth elements’. From lithium batteries through to copper cabling, EdTech inherently involves the earth being depleted of non-renewable resources. In the short-term, this extraction causes considerable environmental contamination and pollution. In the longer-term this extraction is simply non-sustainable. Alongside the rapid loss of scarce minerals, for example, more than half the copper that will ever be extracted from the earth has already gone. In basic geological terms, we cannot continue to produce digital technology in the ways we currently do.
  • The environmentally destructive manufacture and production of digital devices. Regardless of how they are actually used in a classroom, between 70% to 80% of energy expended during the life-time of a digital device occurs during its initial manufacture. As Crawford and Joler’s forensic ‘anatomy’ of Amazon’s Echo device illustrates, the production of any digital technology “requires a vast planetary network” to facilitate the smelting, processing and mixing of raw materials that are shipped halfway around the world to be assembled. Each of these stages involves the accumulation of harmful waste products, hazardous chemicals and toxic waste disposal.
  •  The energy-greedy data infrastructures that lie behind digital transactions. In contrast to the abstract notion of data processing and storage occurring somewhere ‘in the cloud’, is the rather less romantic reality of brown-field, climate controlled data-centres and server-farms. It is estimated that data-centers consume up to 3 percent of all global electricity production and account for about 2 per cent of total greenhouse gas emissions. These figures are fast-rising, and already place the digital data industry roughly equivalent to the airline industry, and mean that educational internet use takes up a significant amount of energy. For example, even a one-off internet search generates a telling amount of CO2. As soon as a student or teacher does anything ‘online’ the impact is felt around the world.
  • The environmental cost of dismantling and disposing digital hardware. As the growing problem of ‘e-waste’ show, microelectronics is an extremely difficult and costly product to recycle. The recycling (often simply the dumping) of devices that are deemed to have outlived their usefulness leads to heightened levels of pollution, contamination and toxic waste in some of the poorest regions of the world. In this sense, the continued imperative to upgrade and keep EdTech ‘up-to-date’ is one of its most destructive qualities.

In light of all these costs and consequences, it is difficult to see how education can continue for much longer with its excessive levels of technology consumption and use. In a near-future of rising sea-levels, climate mass migration and low-carbon restrictions, much of the current hype that surrounds EdTech is likely to quickly seem inappropriate if not obscene. Demands for ‘One Device Per Student’, unlimited data storage, live streaming and the expectation for everyone to be ‘Always-On’ will seem as anachronistic as twentieth century attitudes toward smoking cigarettes and burning fossil fuels.

In a practical sense, then, it now makes sense to prepare for a near-future where there are insufficient natural resources to produce and sustain the educational use of digital technologies at the levels we have come to expect. If you are not fully convinced by these ecological arguments, then there are also good moral reasons for doing this. Indeed, the environmental issues just outlined are underpinned by a litany of associated ethical failings in terms of exploitation of human labour, the illicit trade in rare earth elements, and the deadly money trail associated with so-called conflict minerals such as tin, tantalum, tungsten and gold. As Ingrid Burrington put it, alongside the environmental degradation “there is blood in every piece of your technology”.

These are all controversies that no-one in education should be comfortable being implicated in. Yet as it currently stands, EdTech is exacerbating all of these issues. Everyone in education therefore needs to ask themselves whether they are happy to continue being part of what is clearly a catastrophic drain on the planet and a fundamental threat to the living conditions and life chances of future generations. If not, then we urgently need to start rethinking the sorts of digital technology use that are really needed in education, and how these might be achieved in more sustainable ways.

 

 

Neil Selwyn is a professor in the Faculty of Education, Monash University (Australia). He previously worked in the UCL Institute of Education, and Cardiff School of Social Science (UK). Neil is currently writing a book on the topic of robots, AI and the automation of teaching. Over the next six months he will be posting writing on the topic, hopefully resulting in: Selwyn,  N. (2019) Should Robots Replace Teachers? Cambridge, Polity.

Neil can be found on Twitter @neil_selwyn

Transforming education through technology: Vision vs Reality

Government advisors emphasise the importance of digital technology’s “transformative potential” for learning; top private schools splash advertisements of children with expensive robots across newspapers; and tech companies pitch their latest product to the market “to turn lessons into enriching learning experiences”. The vision is clear: digital technology will transform education.

But look into the average school around Australia and what really is happening?

First order barriers still exist  

Bringing digital technologies into schools in Australia has not been easy.  Some schools have yet to pass the “first order” barriers, such as access to reliable Internet. Many schools are using Bring Your Own Device (BYOD) programs and the different devices and platforms that teachers have to deal with can be a nightmare. Lack of onsite tech support adds more pressure.

All of this can make the use of technologies and teaching Information and Communication Technology (ICT) in schools time consuming and frustrating for many teachers.

Other concerns with digital technologies include dealing with online bullying, students misusing technology in class and addressing parental concerns over excessive screen time.

Increase in teacher workload

On the face of it, digital technologies should create extra time. Online feedback to students can be immediate for some things, saving having to test and process a whole class. And everyone has instant access to expert knowledge: no more trudging off to search shelves in the library. Working with technology that once required a lot of time, expertise and equipment, such as having students work on a video together, can now be assigned for homework.

Online communication and collaboration can be quick and effective, helping develop many 21st Century skills, such as critical thinking and teamwork. The technologies now available should help schools more easily “focus on human interaction, strategic and creative thinking”: those skills seen as necessary for a future transformed by technology.

But the reality is that digital technologies have created more work for teachers, not less. Educational data mining, ostensibly used to highlight areas where students need extra help, is experiencing exponential growth. Collecting and reporting student data has become overwhelming and has meant that teachers struggle to find time to do their job. Educators wonder what data they will be asked to collect and process next about their students.

So how important is digital literacy?

There are two main justifications given for teaching digital technologies in schools. The first is that of multiliteracies: making and creating meaning in a range of modes such as print, body language, music, games and websites. In this way, digital literacy can be seen as just another kind of literacy that needs to be taught in schools.

The other, much more controversial reason, is that digital technology can help improve the way students learn and the way teachers teach.

Digital literacy as literacy

Around 80% of government services are now online signalling a belief that the public is digitally literate enough to access basic information this way – although callers to Centrelink or Medicare may argue this point. And the level of digital literacy required in society will likely grow as Information and Communication Technology (ICT) becomes more entrenched in industry.

Of all the 21st Century skills employers demand, digital literacy is the most rapidly increasing. School graduates don’t consistently have the standard of digital literacy expected and the federal and state governments have produced school curriculums they hope reflect the urgency of the situation. This seems a sound justification to include ICT in our schools. Few would argue that digital literacy is not needed by school graduates.

The Australian Curriculum includes digital technologies as an individual learning area, like history or science, and Information and Communication Technology (ICT) as a general capability to be included in all subjects across the Foundation to Year 10 curriculum. In many schools, the digital technologies strand is also integrated across the curriculum. It’s not hard to imagine ICT in a science classroom where students are already doing things like electric circuits, but there are also many instances where ICT is working in other subjects. For example, my son’s Japanese class regularly Skypes a sister school in Japan to play games, chat in Japanese and English, and exchange cultural food recipes. Even subjects like Health and Physical Education have changed as students collect data and analyse their own fitness with Fitbits (a wearable device that records steps and heart rate). But many educators still see ICT as busy work with little pedagogical value in some lessons.

Can digital technologies improve the way students learn?

 The second justification given for teaching digital technologies in schools is that somehow this will help students “learn better”. This is where the main controversy lies. What evidence do we have that this is true? As computers have been in schools for such a short time and used very differently, it is hard to find conclusive proof that using them and teaching ICT makes a difference.

Declines in NAPLAN results are widely reported in the media, so we don’t seem to have improved learning outcomes, and, worse, the NAP-ICT results indicate that students’ digital competence is actually decreasing, adding evidence to a large body of work that the widespread belief that young people are “digital natives” is just not true.

Some teachers may not want to engage with digital technology, seeing it as an optional extra. The argument goes along the lines of: I learned history without digital technology so why should I include it in my lessons?

Then there is the commonly stated second order barrier, that teachers just don’t have sufficient digital skills to understand how it can be used effectively. For example: 5-7 year olds are expected to learn “how data are represented by changing pixel density (resolution) in a photograph with support and noting the change in file size to successfully email to a friend”. But this is a task many teachers would struggle with.

We believe it is therefore crucial that teacher education specifically includes digital skills and the critical frameworks needed to evaluate the pedagogical worth of digital tasks, so teachers can make informed decisions about including the general capability of ICT, that is digital technologies, in their lessons.

There are also teachers who do have the skills, who can imagine and create amazing, engaging, student centred digital lessons, but choose not to. These are the teachers that can tell us the most about digital technology in education and more research needs to be conducted in this area to understand the processes they go through when deciding to include ICT or not.

Another issue is the wow factor. Teachers who have been at the cutting edge of technology in schools for a while have seen technologies come and go. An example of this is the interactive whiteboard, which now sits sadly in many classrooms, used only as a projection screen.  The increased teacher workload, involving training and time spent creating lessons through the interactive whiteboard software seems to have come and gone. While there may have been an increase in excitement, enthusiasm and engagement as the shiny new toy was brought out to play, there is little evidence that learning was improved. Could the same engagement have been achieved by bringing a puppy into the classroom? Unless there is a clear reason for ICT into the classroom, why would teachers use it?

The future

The pace of change for digital technologies is very fast. Today makerspaces and robotics are not uncommon in schools; augmented and virtual reality are making a comeback; and the Internet of Things (imagine a smart classroom school where the classroom temperature, lighting, and air quality are self-adjusted and then sensors in the room measure the amount of noise and activity to see if they need to be changed) and more sophisticated Artificial Intelligence are predicted to be on the way.

Professional learning needs to work with, rather than on, teachers to help them unpack the digital technologies and ICT capability in the Australian curriculum in a meaningful way. Communities need to work with schools to promote the use of digital technologies, particularly among girls, and remind them that digital technology skills are important in all subjects and for all jobs, not just tech ones. Schools are currently facing opportunities to restructure their already crowded curriculums to promote this.

If we really do want transformation, the gaps in the way Australia deals with digital technologies in schools, in the application of the ICT capability in the Australian curriculum, in teacher education and, especially, in the support for schools and teachers to implement the curriculum need to be filled. In the next 10-20 years’ we will have graduates who have had the new curriculum for their full schooling and hopefully, our future young Australians will have a more consistent level of digital competence and confidence.

 

Amber McLeod is a lecturer and early career researcher in the Faculty of Education, Monash University.  Before joining Monash, Amber was a microbiologist and then taught English as an additional language in Japan, Brunei and Australia.  She is the Director of Pathways Programs and teaches a range of units including digital technologies. Amber’s research focus is on increasing digital competence in the community. This includes investigating attitudes towards ICT, cultural understandings of ICT and ways to increase pre-service teacher digital competence.  Amber is also passionate about the role of ICT in increasing access and equity, in particular, how digital competence can be taught in pathway courses, and its role in successful student transition to university and retention.  

 

Dr Ibrahim Latheef is a lecturer in ICT and digital technologies in education in the Faculty of Education at Monash University. Latheef was a primary teacher for six years and head of school in the Maldives. Latheef is a digital technology enthusiast and is continuously exploring how technologies impact human consciousness. As a result his interest lies in the use of technology as tools in education and the sociocultural and sociohistorical impact on the learners. This specifically includes the subjects of mind and interactivity, cultural historical impact of technologies in education, digital games in education, digital literacy as a way of thinking and mixed realities in education