AI and Ethics Series : Why Bias Auditing is Essential in Ethical use of AI within the NHS
Friday, 5 September 2025
In this article I start to dig more of the detail of why ethics as a framework is hugely important within the NHS. Bias is something we may not consider when we start to look at creating a machine learning model, or when we go out to the market to procure an AI solution, and yet it is Bias that can lead to patient harm, can reinforce health inequality and ultimately lead to a failed project.
Take maternity as an example, Why do some women in the UK face higher risks during childbirth simply because of their ethnicity?
Black & Asian women in the UK are twice as likely to die during pregnancy or childbirth compared to white women. Those aren’t statistics plucked from thin air, they come from MBRRACE-UK(Mothers and Babies: Reducing Risk through Audits and Confidential Enquiries), the national programme that tracks maternal and infant deaths across the UK.
Understanding the Types of Bias And Where They Appear in Maternity Data
Bias appears in various forms within data systems. Here’s how it affects NHS maternity statistics:
- Historical bias: Past inequality is baked into the data. Disproportionate mortality rates among Black and Asian women are well-documented facts, not speculation.
- Representation bias: Minority groups may be underrepresented or misrepresented in datasets, leading to skewed insights.
- Measurement bias: Differences in how symptoms or outcomes are recorded like whose pain is taken seriously can distort the picture.
- Confirmation bias: Clinicians or systems may interpret data through lenses that reinforce existing narratives ignoring exceptions that don’t fit.
Why the NHS Might Use This Data And Why It’s Dangerous to Do It Blindly
Suppose NHS policymakers want to build an AI tool for resource planning and allocation. The goal, to reduce bottlenecks in overstretched maternity wards and ensure resources are distributed fairly. This makes perfect sense, and it will not be long before someone, somewhere tries to build this AI model.
The issues arise where certain groups of patients are underrepresented in the data and we bake this into the model resulting in unequal resource allocation. If this model is deployed nationally then it reinforces health inequality which will lead to the same outcomes which MBRRACE-UK have identified. Only now, instead of those inequalities being regional and different across England, we would standardise this everywhere and make the situation worse.
Bias Auditing in Practice: What Would a Real NHS Audit Look Like?
If the NHS is serious about getting this right, bias auditing would include:
- Data equity check: Are Black and Asian women fully represented in the dataset, or under-counted?
- Outcome equity test: Does the AI produce more errors for some groups than others?
- Fairness metrics: Compare false positives and false negatives by ethnicity.
- Contextual review: Consider how housing, income, and racism impact maternal outcomes, not just clinical factors.
- Mitigation: Use data weighting, fairness-aware algorithms, and human oversight.
This isn’t just a “data science” job. Midwives, obstetricians, patient advocates all need a seat at the table.
Neither is it a job to be done once and then forget, Bias auditing is done continually to help spot and prevent any model drift. Model drift is where the predictive outcomes move away from the data that the model was originally trained on. This can have a negative impact on using any predictive information that the model provides and should result in the model being retrained to ensure it is still accurate and safe to use.
This would be easier if the model was being created within your own organisation, as you would have the experts sitting within your control. What happens though if you were to procure this system, are your procurement terms and conditions going to mandate Bias checking as part of the procurement, will you insist on regular reports from the supplier to ensure the model stays accurate?
The Bigger Lesson
Bias auditing is not about polishing away uncomfortable truths. It’s about recognising them and deciding whether technology can help reduce inequality, or whether building a system might actually make things worse.
But here’s the catch: recognising bias in data is itself still new territory for much of the senior managers in the NHS. Knowing how to spot problems in data, let alone how to fix them, is not widely understood. Without consistent frameworks and national standards, every organisation risks doing its own thing, meaning the quality of bias auditing could become a postcode lottery. For something as serious as health, that’s unacceptable. We need shared approaches, not piecemeal experiments, if we’re serious about tackling inequalities rather than cementing them into the next generation of technology.
Let's be honest with each other, do you think you have the skills in your own organisation to address the full requirements of AI Ethics? The AI ambassador network shows that one of the primary concerns across the NHS is getting governance right.
My own view is we need key roles to be trained for this new reality. Caldicott guardians are great at understanding their remit for patient confidentiality but are likely to feel exposed when asked to sit on an AI ethics committee overseeing the implementation of AI within their organisation. The same is true for my role as the SIRO, which is why I am raising it here.
I hope that these articles will shine a light on what is needed and how we need a focus on a shared framework and training programme, so that those charged with deploying AI within our organisations are confident and competent to do it.
John Uttley – Innovation Director & SIRO, NHS Midlands and Lancashire