Bias In, Bias Out: Are AI Systems Neutral or Nurturing Prejudices? (AI in Education: Part 5)

Bias is one of those words we hear a lot, tossed around like confetti in academic journals and casual conversations alike, but when you stop to think about it, it refers to something we all swim in every day without even noticing. At its simplest, bias is really just a mental shortcut. Sometimes, that’s harmless (like my own irrational dislike of chocolate ice cream or my fierce loyalty to dark mode in Word). Harmless until, of course, you meet me at a dinner party and offer dessert. But where bias becomes a real problem is when these little assumptions start shaping how we treat people, and especially when those assumptions creep into classrooms.

And yes, classrooms are full of bias, whether we like to admit it or not. It slips into the way certain textbooks tell history, the differing patience teachers might have for one student’s mistakes over another’s, the casual jokes in the courtyard, even the way praise is handed out. A lot of this happens without malicious intent: teachers aren’t rubbing their hands together planning how to undermine students. But even subtle assumptions can add up, leaving a very different kind of mark on kids than what’s written on a report card.

Now, we’ve all heard of the familiar flavours of bias (if bias were ice cream…and now I promise that’s the last ice cream metaphor). You’ve got your confirmation bias, where you mainly notice things that prove you were right all along, and your affinity bias, where you’re convinced other people are smarter or more trustworthy if they happen to remind you of yourself. Then there are the heavier ones: gender bias, cultural bias, socio-economic bias. These are the ones that show up in classrooms when boys are expected to be rowdy troublemakers and girls are expected to sit politely with neat handwriting. Or when the kid with the “unfamiliar” name or accent gets treated differently, whether that’s in the roll call or in the doubts about how well they’ll “fit in.” Or when there’s an assumption that if a student comes from a lower-income background, they’ll inevitably struggle to keep up. All of these unspoken expectations change who gets opportunities and who gets overlooked, and the messages students absorb about themselves along the way.

And then there’s neurodiversity, which is far too often missed in this conversation. Neurodivergent students, those with ADHD, autism, dyslexia, or any of the thousand different brain-wirings that exist in the world, might not fit into the neat little boxes school systems like to draw. I think we have well and truly established that by this point in the series, particularly in last week’s discussion of personalised education. What we all need to be better at recognising, regardless of what capacity we interact with these students in, is that they often bring incredible strengths, new perspectives, and creative solutions that benefit everyone. The problem is, bias can stop us from seeing that.

A child with ADHD isn’t “difficult” (ok yes sometimes they can get on every last nerve we have… but so can every other student, we just notice it a bit less, and that’s the bias), they just might learn better when movement and activity are part of the lesson. A student with dyslexia isn’t “falling behind” (maybe they are, but perhaps they have been classed as lazy and aren’t receiving the support they need, and there’s that bias coming in again), they may excel in storytelling, critical thinking, or problem-solving if only we look differently at their skills. Autistic students aren’t “lacking” in social behaviour, they may actually contribute unique insights, deep concentration, or innovative ideas. Yet again and again, classroom systems lean into the stereotype rather than the strength, leaving neurodivergent kids told in silent ways that how they think, learn, and interact is somehow less valid.

Between all these biases, I would bet good money that every student is impacted in at least some significant way, regardless of what side of a bias they fall on.

Imagine what that does to a child’s sense of belonging.

Now, take all those biases already swirling in school, and drop computers into the mix. Technology in classrooms has been hailed as an equaliser, but ironically it can just as easily reinforce the same old inequalities, it simply does so behind a screen with a friendly interface. Computers look neutral, but anyone who’s tried to get one to update at a crucial moment knows they are anything but impartial. AI, in particular, adds an extra twist, because it’s trained on data drawn from the real world, which is to say: filled to the brim with human quirks, stereotypes, and, you guessed it, bias. Put bluntly, an AI may look smart, but it’s basically just a mirror to society’s habits, only it reflects them back with more confidence than a teenager on TikTok (I only recently got onto TikTok for ideas for some gymnastics drills, and I make this statement known just how confident teenagers on TikTok really are).

So, what happens in education when we let AI play along? A fair few things that ought to give us pause. If AI is used to “predict” which students will succeed, it might quietly downgrade students from certain backgrounds regardless of individual skill. If it’s tasked with recommending materials, it might push content that centres only certain cultures while sidelining others. If it’s giving feedback, it might overemphasise patterns in past performance that pigeonhole children as “struggling” or “high achieving”, two labels that risk becoming self-fulfilling prophecies. And because so many students see AI as a kind of oracle (“My teacher may have opinions, but ChatGPT? That’s a computer, so it must be right”), these messages carry even more weight. Bias disguised as objectivity is perhaps the most dangerous kind of all.

The consequences are huge. We’re not just talking about skewed marks or mildly unfair group work. We’re talking about students soaking up unconscious messages about their worth, capability, and place in the world. If an Indigenous student never sees their culture reflected in suggested materials, that invisibility sends a message no rubric will capture. If a neurodivergent student is consistently categorised as struggling, it chips away at their confidence and resilience over time. And just as damaging, if another child is consistently built up as the “brightest,” then they risk leaving school less prepared to collaborate, empathise, or understand the strengths of those around them. Bias doesn’t just punch down; it can elevate people to unhelpful places too. Both outcomes stick, long after exams are over.

But before we throw all the laptops out the window, it’s worth remembering that AI isn’t automatically the villain in this story. When designed thoughtfully, it can actually help expose problems we might miss. If used carefully, it can adapt to students’ different needs, recognising unique strengths and providing tailored materials that support them without flattening the diversity in our classrooms. It might, for example, highlight that a particular group of students is consistently under-participating, prompting educators to adjust and rebalance the classroom. It can give all students a chance to engage at their own pace, not on a single rigid timeline. Of course, this works best when teachers understand how AI is making its decisions (because “the algorithm told me so” is not a lesson plan). It also works best when the data feeding these systems actually represents all kinds of learners, not just the majority. Otherwise, we risk designing tools that treat difference as a defect.

The bottom line is that bias is inevitable - it’s part of being human. But that doesn’t mean it gets a free pass in education, especially not when AI is involved. The question isn’t whether bias exists, but whether we notice it, name it, and design systems that push back against it. A fair classroom is not one where every student is forced into the same mould, but one where diversity of background, of experience, and of thought process, is seen as the richness that drives learning forward. If we want AI to truly serve education, it has to be built with those human values in mind: transparency, inclusion, and humility in the face of difference.

Everyone who works in education knows it isn’t only about the content taught; it’s about the identities being shaped along the way. Every bias, every algorithmic suggestion, every interaction tells a child something about who they are, or who they’re not. And the real question for us, as educators, technologists, and humans fumbling along in an AI-shaped world, is: what kinds of messages do we want to leave our kids with? Hopefully, ones that help them see themselves as whole, capable, and valuable, regardless of if they ever understand the appeal of chocolate ice cream.