The following is a transcript from a talk I gave at the Innovate EdTech conference, November 12 2017.
Thank you, it’s a pleasure to be here. I am Junaid, I’m a mathematician and so I like to start my talks with numbers. I can tell you there are only 4400 words separating us all from lunch. I can also tell you that I speak at around 155 words per minute, so if you do the maths on that we should have a couple of minutes left for questions. But do ask any questions as we go, and I’ll just speed up if necessary.
So I am the Director of Education at Whizz Education. We partner with schools and ministries of education across the world to raise standards in learning, through our virtual tutoring service Maths-Whizz, which I’ll say more about during this talk. Our core belief at Whizz is that every child deserves a learning experience that supports their individual needs and pace of learning. So you could say we’re champions of personalised learning.
I say that a little glibly because I think personalised learning is a poorly understood concept in Education, and EdTech in particular. My goal today is to challenge those of us working in EdTech to really think about all that personalised learning entails. I want to cut through the hype and ask whether we’re really up to the challenges of personalised learning. I don’t think we should treat anything in EdTech as a given.
I’ll look at personalised learning through three lenses: algorithms, data and content. The holy trinity of EdTech.
But first, a reminder that personalised learning is not new. It’s tempting to think the idea has emerged in the last few years with the latest wave of digital technologies. But this newspaper ad is from the 1880s and it’s promoting a correspondence course, learning by mail.
You’d sign up to a course, list your goals and preferences, and the course provider would send you tailored content through the post. That might sound familiar. This was the EdTech of its time, so we can dispense right away with the idea that personalised learning is a 21st century innovation. And as we’ll see, in many ways personalised learning hasn’t evolved all that much since these humble beginnings.
Now, I do believe personalised learning is a must. I’ll make the case with a few visuals from our Maths-Whizz tutoring program. My claims may be specific to maths, but I think they apply across most areas of learning and teaching.
So the Maths-Whizz tutor works with primary and early secondary students. It’s a fully automated program that behaves much like a human tutor. Now I am a human tutor, and I can tell you that when I first meet a new student, I issue a diagnostic assessment. I want to identify their strengths and weaknesses in topics, and evaluate their baseline maths abilities. We have a way of measuring that, it’s called Maths Age, it’s much like Reading Age, so the interpretation is natural. We’d expect a seven-year-old to have a Maths Age of 7 and so on. Now, I’m going to show you the Maths Age distribution for a typical class of Year 5 students. This is real data, we have permission from the school to share it, we’ve just anonymised the names.
Each of those icons is a student, and they’re in Year 5, so we’d expect them to have a Maths Age of 10. And yet we see there’s a multi-year gap, stretching to around 4 years. And we see this across the world. Maths-Whizz is in eight countries, we serve schools in contexts as far reaching as rural Kenya and Washington, Dubai and here in London, and we see this huge spread of learning needs wherever we work.
It already underscores the challenge for teachers: how do you deliver a lesson that caters to students’ individual needs? The challenge actually runs deeper than you might think, because when we drill down into students’ individual learning profiles, we can see just how widespread these differences are. Let’s take two students from that cluster, who seem to be getting on just fine overall. Here’s Nancy, not her real name.
You can see the different topics, and these grey bars show where Nancy was placed in her initial assessment. Now she was around 7 at the time and in most topics she was within the expected range of ability. Except for Place Value where she was way behind, more than two years.
And now I’ll show you Luke, who is in the same class as Nancy. Two students, sitting side by side, receiving the same support and instruction, so you might expect a similar learning profile. Well, here’s Luke: now look at Place Value – it was Nancy’s weakest topic, but it was one of Luke’s strongest. His weaker topics were Fractions and Pencil/Paper methods.
This story plays out with every single student: 30 students means 30 distinct learning profiles. What is a teacher to do?
Well, one-size-fits-all, ushering students through the curriculum at the same time, simply won’t do. It is distinctly cold and impersonal to expect students to learn at the same pace. We just don’t develop mathematically at a linear rate. And it is cruel and unusual to penalise students for not fitting the prescribed learning journey of a standardised curriculum.
And so part of personalised learning has to be about respecting students’ individual pace of learning. We can commit to getting every student through the same curriculum, but instead of fixing their pace, we can make that the variable. But how?
That bring us to the first piece of the holy trinity: algorithms. Crafting a learning journey that adapts to each child’s pace of learning is largely an algorithmic challenge. Let’s go back to Nancy, here was her learning profile.
Now we can ask, from this baseline profile, what would a human tutor do? I know what I’d do: I’d focus on Place Value. The mathematician in me knows that maths is like a Jenga tower, if you have shaky foundations in any one area, your overall learning will collapse. So more attention in Place Value. But I wouldn’t ignore the other topics either. I want my students to forge links between topics, to see them as part of an integrated body of knowledge. I’ll also be sure not to move Nancy along in any topic until she’s grasped the current concept. And if I see that she’s really stuck, I’ll offer her remedial support – perhaps we’ll spend time on a more foundational concept to plug her knowledge gaps.
Now, you could take everything I’ve just said, these principles of human tutoring, and encode them. This is the old-fashioned way of doing AI: just code in human behaviours. And by and large, that’s how the Maths-Whizz tutor works. So here is Nancy’s profile two years later, after receiving that 1:1 support from the virtual tutor.
The blue bars show her improvement in each topic. Look at Place Value; it’s no longer a weakness. We’ve not neglected a single topic. And notice how she has a much more rounded learning profile. That, in a nutshell, is the optimum that the algorithm is seeking. It’s the same for Luke.
Like I said, the Tutor is largely hard-coded. But it’s been going over ten years and in that time we’ve amassed a lot of data: every lesson ever attempted by more than half a million of students who have used Maths-Whizz. So we have opportunities to use more modern approaches like Machine Learning to enhance our tutoring algorithms.
Well, we participated in an EU research project called iTalk2Learn, where we worked with partners across Europe, including a Machine Learning lab in Germany, to build a prototype of a next-generation virtual tutor. And part of the thinking was to upgrade our hard-coded algorithm using a recommender system, which recommends lessons in a similar way to how Netflix chooses movies for you based on your viewing habits. The idea is to keep the learner optimally challenged.
It all sounded great in principle, but when we trialled the sequencer against our own, hard-coded version, we found some issues. The data-driven approach completely ignores some of the human judgements that have stood the test of time. For example, the standard, human authored Maths-Whizz tutor would never teach Fractions until students had mastered the basics of multiplication, but the machine learning sequencer couldn’t care less; it’s agnostic to the underlying domain and doesn’t pick up these dependencies. At times it even recommended the same lesson multiple times in a row, which the Maths-Whizz tutor would never do because it is frustrating and disengaging for students. And the lesson learned here is that algorithms are no panacea. We saw some promise, particularly in how efficient machine learning algorithms are, and we’re looking at developing a hybrid sequencer that combines the best of human judgement with these recommender system algorithms.
But algorithms too often sit in a black box: we simply see what goes in and what comes out, and we often have no visibility on the nuts and bolts of how they work. It’s so important that we don’t just defer to the so-called ‘science’ of Data Science, because blind faith in algorithms will lead us astray. Cathy O Neill’s Weapons of Math Destruction will persuade you that algorithms, when left unchecked, will work against our human interests in all walks of life: university admissions, car insurance, the legal system. They perpetuate our biases, our prejudices, many of which we may not even be conscious of.
Education is not immune to the threat of unchecked algorithms. No student or teacher should ever be subjected to an algorithm without knowing the essence of how it works. They don’t need to become Data Scientists to do that; if you can’t explain how an algorithm works in simple terms you have deferred too much to the science. And actually, that’s not science at all.
End users of algorithms will only profit from them when they have transparency, and when they are afforded control over the black box. At Whizz we found that while teachers welcomed the automated behaviours of the Tutor, they also wanted to direct it in different ways, perhaps focusing on a particular topic in a given week. So we have to reconcile the relentless focus the Tutor has on creating individualised learning journeys with the specific behaviours teachers are looking for.
And if it’s done well, the real promise of a Tutor is that it can act as a virtual teaching assistant. A teaching assistant for every child, saving teachers time because they don’t have to assign or mark 30 pieces of work. And the teaching assistant provides constant, real-time feedback on students’ learning through data.
And when you look at the kind of insights now available to teachers, at the click of a button, you can get a sense of how that might transform their practice. Because now they enter the classroom, armed with those insights, which highlight every student’s knowledge gaps, shows how the class is progressing in each topic, and so much more.
But I used a sleight of hand; maybe you noticed it. I have switched seamlessly to talk of “insights”. When actually, all we’re seeing here is data. Numbers. Information. The leap from data to insight can only be made when you have context. I learned this the hard way.
A while back, I visited one of our partner schools over in Seattle and I had the great pleasure of meeting one of our students, a fifth grader who I’ll call Joshua. And Joshua had been rewarded for achieving the highest progress on Maths-Whizz at his school that semester. The principal had given us notice ahead of time, and invited us to look through Joshua’s reports to see what we could find. And I looked through Joshua’s reports and figured he has a consistent work ethic, his favourite topic seems to be Fractions, Shape and Space is an obvious area of difficulty. And all of that was all confirmed when I spoke with Joshua, so I was feeling rather pleased. But after Joshua left, his principal revealed something else about him that I’d missed – something the data would never reveal: and that is that Joshua is homeless. I was stunned: I cannot fathom that in this day and age, a young, hardworking boy has no home to call his own. Well, it turns out that Joshua lives in a hostel, he shares a bedroom with his mother and three siblings. He has no computer and so his mum takes him to the local library every week to get his Maths-Whizz usage up. And when Joshua goes to school he has dedicated time on Maths-Whizz because they are sensitive to his situation.
It’s an amazing story, but I would never get that from the Maths-Whizz reports. Nor should I. Any EdTech that demands those most intimate details of a child’s life must be resisted. Some things are meant to be hidden from data dashboards.
And being vigilant to data privacy means accepting that intelligent tutors and real-time reports only tell part of the story. They can trigger smart questions, like why was Nancy weak in Place Value? But we must entrust teachers to interpret data within their context and to apply their own expert, human judgement to answer those questions. And that’s why teachers must be allowed inside the black box of tutoring algorithms and given the freedom to direct and even over-ride these tools. And I’ll say that if personalised learning is our goal, then data without context and algorithms without human oversight are futile.
Now, when I first came across Maths-Whizz I was curious. As a mathematician, I wondered, what does Maths Age actually mean? I’ve been referencing it for the past 15 minutes. And I can tell you it is so easy for me to extol the virtues of Maths-Whizz without having this core metric scrutinised. I’ve found this is very typical in EdTech. We invest so much time, energy and money into algorithms and data that we’ve abdicated on the most important piece of all: content.
There’s an old trope that pedagogy is the driver, and technology must be the accelerator. Well you can have no notion of pedagogy without looking at the underlying content. Algorithms and data might help craft individualised learning journeys, they might help you as a teacher to plan your next lesson. But seriously, if we’re going to expect students to spend time, week after week, interacting with these systems, we’d better hold ourselves accountable to content. That’s the heart of the learning experience. And I think, in EdTech, content has been shamefully neglected in our thirst for scalable innovation.
There is one particular limitation to the content that feeds virtual tutors: they rely heavily on structured, routine learning tasks. You know the kind: multiple choice, short response items. That’s because these systems need to be able to automatically score students’ responses. You only get the automation, the analytics, by narrowing down the form and style of content.
Now, I should say that this content has its place. With maths, there’s much you can reduce to structured learning. Times tables, multiplication algorithms, the quadratic formula – the fluency and fundamentals, much of that can be taught and assessed using structured tasks. And that’s what Maths Age gives you, by the way – a reliable measure of students’ core mathematical knowledge and skills.
But there is so much more to maths than structured tasks. That’s because maths is essentially an open discipline. What I mean by that is that maths should give us the freedom to explore concepts, to shape and reshape problems, to develop your intuitions, and to build a whole range of problem solving strategies. It’s a holistic endeavour, and really hard to measure. As educators, the onus is on us to focus on the processes of learning, to track the nuances students’ thinking, rather than just capturing the blunt outputs.
Education today struggles with this. We have decided to value what we can measure, and there is a virulent strain of ideology that permeates our models of curriculum and assessment. It says that students should remain seated, in silence, in straight rows, consuming facts and information. Well if that’s all there is to it, I swear, the virtual tutors will suffice.
But there’s so much more to learning than consuming static knowledge. My greatest influence in EdTech is Seymour Papert, and I consider his book, Mindstorms, to be essential reading for anyone working in the space. It was written over thirty years ago and in it, Papert espouses an entirely different view of how computers support learning. There is one edict that stands out, a single principle that is more relevant today than ever before: he suggested that the child should program the computer, not the other way around. And he talks in great detail about what this means – it’s not just a matter of learning how to code, but using the affordances of a computer to explore concepts, to solve problems, to make mistakes, to debug. All the hallmarks of deep learning. Papert spoke of the thousands of forms and functions enabled by computers. And I’ll give a quick nod to Maths-Whizz content here because while the tasks are heavily structured, they use interactive exercises and animations to really bring maths to life. They provide instant feedback in a way that no static textbook ever can.
But it’s a crying shame that the vast majority of online content today takes the form of digitised textbook content rather than native content that taps into the unique affordances of digital. Marshall McLuhan observed that whenever we adopt a new medium of technology, the first form of content resembles what came before. The first recorded films were televised stage productions. And we’re seeing that in EdTech: textbook content that has been digitised simply because it scales and produces lots of data.
EdTech can sure be a paradox: it amplifies the very pedagogies it is intended to liberate us from. A pedagogy that is limited by what can be bluntly measured.
We have this tension between what can be measured – structured learning, basic knowledge and skills – and deeper aspects of learning. And a real breakthrough in AI, when we get really close to simulating human tutoring behaviours, is in being able to track students’ thinking, and to engender exploratory learning. There are magnificient tools out there for exploratory learning – in maths you have apps like Wolfram, Desmos and GeoGebra. We haven’t yet figured out how to combine the rich, open tasks afforded by these apps with the real-time tracking of intelligent tutoring systems like Maths-Whizz. Indeed, I would argue a system is only truly intelligent when it achieves that combination.
So personalised learning cannot be encapsulated by algorithms and data alone because they feed off content that does not have the depth to even do the learning part justice. And what about the personalised part? What does it mean for content to be personalised?
Maths-Whizz content already feels very personal – it’s warm, colourful, and you just can’t beat animations as an engaging way of explaining maths concepts. It’s my kind of maths, for sure. But we’ve learned a lot from our experiences across the world. Whenever we introduce Maths-Whizz to a new market, we have to undertake a localisation effort to make sure it is culturally appropriate. There are obvious elements to this, like translating the language – our understanding of maths emerges from language.
Even the visual representations have to adapt. In Kenya, we had to get rid of all references to black magic. In the Middle East, it was the pigs. So there is an imperative to adapt content right down to the cultural sensitivities of individual communities. But why stop there? If the premise of personalised learning is that every learner is different, surely we need to create content that adapts to their individual needs, their preferences, their interests. So for EdTech to really live up to its promise of personalised learning, we would have to exploit every one of those thousands of forms and functions Papert spoke of. There’s a huge authoring effort required there, a huge effort needed to truly personalise learning for each child.
And is this really the way to go, keeping learners and communities forever encircled within their local frames of reference? Shouldn’t we be promoting universal content that promotes shared ideals and values? That builds empathy between learners, and forces them to consider representations of knowledge that sit outside of their comfort zone? For the worst interpretation of personalised learning, just spend a few hours on social media: what have personalised news feeds ever done for globalisation and empathy?
So I think personalised learning is worth striving for, but we need to consider all the implications. And we need to consider its limits, and the limits of technology. And when we do that, it sharpens our focus on our distinctly human capabilities.
And as human educators, we really need to think about the kinds of teachers we aspire to be. If teaching is knowledge transmission then believe me, the machines are ready to take over. Arthur C Clarke was right when he said that any teacher that can be replaced by technology will be.
We don’t need to feel threatened by that statement, because technology must be part of the solution. I know this because every teacher I’ve met is confronted with this same reality of widespread learning needs. There is no known way of securing those fundamentals, 30 students at a time, without technology. And we’ve seen what the virtual tutor can do: it systematically embeds the building blocks of learning, the fluency and fundamentals.
Our research shows that students only need to spend 45-60 minutes a week on Maths-Whizz to advance their Maths Age by eighteen months in the first year. Now, students spend what, 5 hours or more studying maths each week, in class, at home and wherever else?
So that leads to some exciting questions: how can we integrate a system like Maths-Whizz to enrich the learning that happens offline? How can we strike the right balance – the blend – between online and offline learning?
Let me put it another way: How can we use technology to empower the teacher? How can we support them to use data to plan their lessons, to design rich tasks that bring about the full breadth and depth of learning, that builds off the foundations of the Tutor? How can we maximise the face-to-face time teachers spend with their students? How can we foster collaboration and discussion in the classroom?
These are challenging questions, and they form the basis of our service. We never leave the product behind in a school because we know technology alone just won’t cut it. But perhaps it can be the great enabler; supercharging offline learning experiences. So we work closely with every teacher to understand their specific environment and needs because personalised learning must also embrace the individual needs of every teacher. Some will use Maths-Whizz for homework, others in class, some in the lab, others after school. And all will have their curriculum objectives, their schemes, their assessment goals, that the Tutor can then be integrated to support. It’s the Tutor in service of our educational goals, augmenting teachers’ practice, and enriching the most important variable in all of education, the student-teacher relationship. That’s the real potential of virtual tutoring, as an enabler of richer human experiences.
There are no silver bullets. I’ll finish with an example from rural Kenya, one of the most challenging environments Whizz works in, where we’re delivering Maths-Whizz, to around 100,000 students in marginalised communities. I had the great pleasure of visiting some of our schools out there last year. One moment in particular stuck with me. I’d spent the morning at one of these schools, and saw how they made the most of scarce resources: power and electricity is volatile, and so is the water supply. But they pray and they endeavour. And this was a good day because the water was running, the power was on, so I got to see Maths-Whizz in action. I was ushered to a computer lab, which had been installed as part of our intervention, and I could see 30 children on Maths-Whizz. And no two screens were the same; each student was receiving lessons appropriate to their needs. This was a hint of personalised learning; a joy to witness.
Then I stepped next door, where I saw something entirely different. A hundred or so students tightly crammed into a dusty confine, all of them squinting to see the single lesson that had been projected by the teacher from the one computer available to them. It was one of our Maths-Whizz lessons, which was again great to see, except out of the hundred students, it may have been appropriate to 20 students at best. This was one-size-fits-all on acid. And the teacher played through the lesson, getting the students to recite back the answers as he tried to command their attention. The interactive elements of the content were lost in the chaos of the classroom.
When I stepped outside, I felt I’d glimpsed the crossroads that EdTech is now at. On one side, some of the promises of personalised learning were coming into fruition. On the other, technology was just amplifying flawed pedagogical practices. I left wondering how those two worlds were going to be reconciled.
And the challenge I think we all face in EdTech is understanding that personalised learning, by definition, means different things to different people, and it may require different solutions depending on the environment. We need to understand that technology alone will not be the solitary solution, although it can be a great enabler. That personalised learning has to be situated offline as well as online. That Algorithms, Data and Content can advance our innovation goals, only if they are built on rich human understanding of education.
It’s always helpful to keep our thirst for innovation in check. When AI first broke onto the scene, one of its founding fathers Marvin Minsky declared: “We’re going to make machines intelligent. We are going to make them conscious,” to which his contemporary Douglas Engelbart replied: “you’re going to do all that for the machines? What are you going to do for the people?”
And as we head to lunch I want to leave you all, my contemporaries, with the same question: what will our technologies and innovations do for the students and teachers they are intended to serve?