AI and Ethics Series : From Ethics to Assurance - Building the NHS 'AI MOT' with an AI TrustMark Review

See LinkedIn Post

In my last article, I explored how Regional AI Ethics Committees (RAIECs) could establish a unified governance model for AI across the NHS, balancing local accountability with national consistency.

That framework answered the initial question: How do we ensure an AI system is safe before deployment?

This article tackles the critical follow-up: How do we keep it safe once it’s live?

Approving an AI system marks the beginning, not the end, of ongoing oversight.

Consider the risk: We would never approve an MRI scanner or a ventilator and then ignore it indefinitely. That would be reckless.

AI in healthcare is not static. Models evolve as data streams shift, vendors update algorithms, and usage patterns change. What is safe and fair today may pose a risk tomorrow.

For the NHS to deploy AI responsibly at scale, governance cannot be a one-off event. It must function like vehicle maintenance, regular, structured, and transparent. It requires an 'AI MOT.'

The long-term need, however, goes beyond a simple analogy. The NHS requires a recognised, trusted symbol of ongoing assurance, a mark that patients, clinicians, and boards can immediately recognise and rely on. This is the concept behind the AI TrustMark Review.

The Case for Continuous Assurance

If RAIECs provide the governance framework, the AI TrustMark Review provides the operating rhythm, the routine that sustains assurance after deployment.

Today, most NHS organisations rely on governance built for conventional IT projects, focusing on static systems like DCB standards and information-governance sign-offs. While necessary, these frameworks fail to capture what happens when algorithms learn, drift, or change hands post-launch.

An AI tool can evolve silently beneath the radar. Who is actively checking that it remains accurate, fair, and explainable?  NHS data changes regularly.  If you have procured a black box AI solution, how do you know changes to data will be noticed by a vendor who is not aware of these changes.  This could cause drift in the model and that could cause harm.

A structured AI TrustMark Review provides the answer: a repeatable, proportionate audit to verify that an initially safe and effective system remains so over time.

What the AI TrustMark Review Includes

Think of the TrustMark as a post-deployment audit combining crucial technical, ethical, and operational checks:

• Performance and Accuracy Check: Comparing current performance against baseline metrics and requiring suppliers to submit evidence of drift or retraining logs.
• Bias and Fairness Audit: Rigorously testing outputs for demographic or geographic disparities, ideally including patient or public reviewers.
• Explainability Review: Confirming the system still delivers interpretable results that clinicians can confidently defend, updating documentation if the logic or interface changes.
• Data and Security Compliance: Verifying data sources, access controls, and retention policies remain lawful and that no unintended data linkages have occurred.
• User Feedback and Safety Reports: Collecting systematic clinician and patient feedback on usability, false positives, or actual harm events, feeding insights directly to ethics and regulatory bodies.
• Public Transparency Log: Publishing a concise, plain-language summary of the AI’s purpose, last review date, and known limitations.

The review should be annual for high-risk systems and biennial for lower-risk ones, flexible, but critically, predictable.

Establishing the Oversight System

The Regional AI Ethics Committees (RAIECs) are ideally positioned to host this process.

Each region could coordinate TrustMark Reviews for a defined portfolio of products (e.g., all imaging AI, all transcription tools), sharing comprehensive findings nationally via a central registry.

• Vendors would be obligated to provide the necessary data and evidence.
• Trusts would verify local implementation and escalate incidents.
• NHS England and the MHRA would analyse national trends, issuing aggregated "AI Safety Bulletins" similar to current Medical Device Alerts.

The result is a living system of oversight that learns and adapts.

Mandating Assurance in Procurement

For all procured solutions, the TrustMark Review must be contractual, not voluntary. Every supplier should commit to:

1. Annual Submission of Assurance Evidence: Including accuracy data, bias testing results, drift analysis, and change logs.
2. Access for Independent Reviewers: Granting non-vendor access to documentation and metrics.
3. Clear Remediation Clauses: Defining processes for suspension or remediation if assurance standards are not met.

Evidence could be uploaded to a secure national assurance portal, where a single RAIEC or expert panel review applies across the country, eliminating duplication.

Each product would feature on a public dashboard showing its status:

✅ Up to date - Assurance confirmed.

⚠️ Due for review - Review cycle imminent.

❌ Suspended - Pending reassessment or remediation.

Trusts could verify the status before purchase or renewal, shifting reliance from vendor promises to verifiable assurance.

Frequency, Timing, and Accountability

The rhythm of assurance must reflect risk.

• High-Risk Systems (diagnostic, triage, or treatment AI): Annual review, with interim checks triggered by retraining, significant data change, or reported incidents.
• Medium-Risk Systems (analytics influencing care): Every 18–24 months, unless a change occurs.
• Low-Risk Systems (admin or summarisation tools): Every 2–3 years or at major version release.

To maintain order, a national review cycle could mandate all vendors submit assurance evidence by May 1st each year. Critically, procurement renewals should align with this schedule: no valid TrustMark, no contract renewal. This transforms assurance from a box-ticking exercise into a practical accountability mechanism.

Independent Assessment and Procurement Reform

Asking suppliers for data is simple; knowing how to interpret it is complex. Most Trusts lack the in-house data science or AI audit capability to verify vendor-supplied evidence.

The solution requires independent expert assessment built directly into procurement frameworks.

• Accredited Assessors: Suppliers must agree to third-party evaluation by accredited assessors approved by NHSE / DHSC. Vendors fund the audit, but the reports are shared with the NHS, not controlled by the vendor.
• National Expert Panels: Specialist panels (e.g., in imaging, NLP) review submissions once, issuing an assurance certificate valid for the review period. Trusts only confirm local implementation, avoiding the need to re-run complex audits.
• Procurement Frameworks: Future frameworks must condition inclusion on passing this independent review and agreeing to the re-assurance cycles. This makes continuous assurance a core cost of doing business in the NHS.

When the annual cycle arrives, the supplier’s independent report is reviewed by the RAIEC, not re-audited locally. This provides assurance, not just software.

The Role of the MHRA

The MHRA's model for regulating AI as a Medical Device is primarily front-loaded. Manufacturers gain a UKCA mark by demonstrating compliance, but post-approval oversight is largely reactive, driven by incident reports. While the MHRA is introducing reforms like "Predetermined Change Control Plans," these are still emerging.

Until then: the MHRA ensures a product is safe to sell; the AI TrustMark Review would ensure it remains safe to use. It complements regulation by proactively filling the assurance gap between approval and real-world performance.

Why Continuous Assurance Matters

Governance is ineffective without ongoing oversight, as even a flawless ethics framework at launch loses value if the system quietly deviates over time.

The AI TrustMark Review delivers three essential benefits:

1. Early Warning: Small deviations are spotted and corrected before they result in harm.
2. Shared Learning: Findings in one region inform and improve deployments nationwide.
3. Public Confidence: Patients and clinicians gain visible proof that AI is inspected, not self-policing.

Conclusion

AI governance should prioritise patient safety. Regular assurance doesn’t impede innovation; it protects it. Each successful TrustMark Review would reinforce confidence that NHS AI is safe, fair, and continually improving, the defining hallmark of a mature digital health system.

#AIinHealthcare #EthicalAI #AIGovernance #NHSInnovation #PatientSafety #ResponsibleAI #DigitalHealth #TrustInAI #DIU

John Uttley – Innovation Director & SIRO, NHS Midlands and Lancashire