- NHS England directed doctors to stop using unregistered AI scribe tools in 2025. DCB0129 and DCB0160 are the legal framework for accountability when AI generates patient record content. A clinician deploying an unapproved tool is outside that framework — and personally exposed if patient harm results.
- The NHS Anima incident (July 2025) shows the three-failure cascade ambient AI creates: AI hallucination, failed clinician review, downstream clinical error. Each layer requires its own governance control. Detecting the error is not governance — preventing it from propagating is.
- Recording patient consultations in two-party consent jurisdictions — most of Canada, California, and roughly twelve US states — without explicit patient consent is a criminal offence. The Sharp Healthcare lawsuit (December 2025) applied this directly to AI scribe recording. Consent framework design is a legal pre-condition for deployment, not a privacy policy update.
- Canada’s AI scribe adoption grew from under 3% of GPs in May 2024 to over 8% by October 2024. Recording without patient consent is illegal in most Canadian provinces. Adoption speed consistently outpaces governance infrastructure — that gap is where patient harm occurs.
- Clinicians who attest to AI-generated records they didn’t have time to genuinely review are becoming ‘liability sinks’ — responsible for outputs they didn’t create, without the workflow support to catch errors. Deploying ambient AI without building genuine review capacity into the schedule is transferring liability without the means to manage it.
Ambient AI scribe tools — large language models that listen to clinical consultations and generate structured documentation in real time — are the fastest-growing category of clinical AI adoption globally. By 2025, NHS pilots were running across dozens of specialties. In Canada, nearly one in four GPs was estimated to be using them. The productivity case is compelling: AI scribes reduce documentation time by 70–90% per appointment in published studies, cut after-hours EHR work significantly, and free clinician attention during the consultation itself. But the governance infrastructure has not kept pace with adoption speed.
The NHS Anima Health Incident: Three Failure Layers
In July 2025, a London patient received an NHS letter inviting him to a diabetic eye screening. He had never been diagnosed with diabetes. His GP practice’s AI tool — Anima Health’s Annie — had generated a fabricated medical summary from his tonsillitis consultation, asserting he had Type 2 diabetes and suspected coronary artery disease, along with a fabricated hospital address. A clinician spotted the error but was distracted and saved the original erroneous version. The error entered the record and triggered the automated downstream process that invites diabetic patients for annual eye screening.
| Failure Layer | What Failed | Governance Response |
|---|---|---|
| AI hallucination | AI generated plausible but false diagnoses and a fabricated address from limited input. | Pre-deployment: medical device classification, DCB0129/DCB0160 clinical safety assessments, hallucination rate testing for clinical content. Post-deployment: MHRA Yellow Card incident reporting. |
| Human review failure | Reviewing clinician spotted the error but failed to complete correction before saving. | Review workflow design: make it structurally difficult to save AI content without completing review. Track review completion rates. Remove one-click acceptance workflows for clinical content. |
| Cascade propagation | Error in record triggered downstream clinical invitation process. | Integration governance: map all downstream automated processes reading from the patient record. Add validation gates where documentation errors could trigger patient-facing clinical actions. |
NHS officials described the event as a one-off case of human error. From a systems governance perspective, this misses the point. A system that relies on no single human distraction ever occurring is not a safe system. The governance question is: what design would have caught the error even if the reviewer was distracted?
Sharp Healthcare and the Consent Litigation Pattern
In December 2025, Sharp Healthcare in California was sued for recording patient consultations using Abridge’s AI scribe without adequate patient consent, alleged to violate California’s Confidentiality of Medical Information Act and wiretapping statutes. California’s two-party consent law requires all parties to a communication to consent to its recording. The lawsuit followed the same legal theory as Turner v. Nuance Communications and Gladstone v. Amazon Web Services in financial services — AI recording and analysis of communications without adequate consent.
Consent Requirements Across Jurisdictions
| Jurisdiction | Consent Obligation for Ambient AI |
|---|---|
| California and two-party consent US states | Explicit patient consent required before recording consultation. Violation is a criminal offence. Generic privacy policy acknowledgment is insufficient. |
| Rest of US | HIPAA Business Associate Agreement required for third-party AI processor receiving PHI. State health privacy laws may add consent requirements. Federal and state wiretapping law applies. |
| Canada (most provinces) | Recording without consent is illegal in most provinces. BC and Ontario ICO issued AI scribe guidance in early 2025. PIPEDA requires knowledge and consent for collection and use of personal information. |
| United Kingdom | UK GDPR lawful basis required for processing health information; explicit consent or legitimate interests assessment for recording. NHS guidance requires DPIA and patient information about AI use. |
| Australia | Recording without consent is a criminal offence in most states and territories under listening devices legislation. Privacy Act and Australian Privacy Principles apply; health information classified as sensitive. |
| EU/EEA | GDPR Article 9 special category processing (health data) requires explicit consent or specific lawful basis. EU AI Act Article 52 requires disclosure when individuals interact with AI systems. |
Designing consent for ambient AI is not the same as updating a patient privacy notice. A meaningful consent tells the patient: that their consultation is being recorded in real time; that audio is processed by an AI system; that the AI generates clinical documentation; that a third-party technology company processes the recording; and how it will be stored, retained, and deleted. Patients must have the option to decline recording, with an equivalent care pathway if they do.
Regulatory Classification: Is Your AI Scribe a Medical Device?
In the UK, whether an AI scribe requires MHRA medical device classification depends on its intended use. If AI outputs are relied upon to inform care decisions — diagnoses, treatment plans, clinical codes, referral letters — rather than simply capturing what the clinician says, it crosses into medical device territory. NHS England clarified in 2025 that many tools in common NHS use should have been seeking higher regulatory classification. In Canada, Health Canada’s February 2025 MLMD guidance applies to AI supporting clinical decisions. An AI scribe with diagnostic suggestion features is within scope. In Australia, the TGA SaMD grace period ended November 2024, and AI clinical note tools with diagnostic features may require ARTG registration.
NHS Clinical Safety Framework
- DCB0129: The supplier must hold current NHS digital clinical safety certification. This is a supplier-level obligation verifying the vendor’s safety processes.
- DCB0160: The deploying organization must complete its own clinical risk assessment before go-live — identifying failure modes, clinical hazards, and mitigations. Maintained as a living document.
- DPIA: A Data Protection Impact Assessment must be completed before processing consultation audio through the AI tool, addressing third-party processor obligations under UK GDPR.
- Clinical Safety Officer: A qualified healthcare professional with clinical safety training must be assigned ongoing clinical risk management responsibility for the tool.
- NHS AVT Supplier Registry: Ambient voice technology products should be procured from NHS-approved suppliers; organizations must confirm compliance before deployment.
Review Workflow Design
The most common failure mode in ambient AI documentation is not AI hallucination — it is inadequate clinician review before content enters the patient record. Governance must address it structurally:
- Make review mandatory in the workflow: AI-generated documentation must not complete to the patient record without explicit clinician attestation. ‘Saved by default’ workflows where inaction results in acceptance are high-risk.
- Design for realistic review time: A summary requiring 15 minutes to review cannot be reviewed in a 1-minute gap between consultations. Build review time into scheduling. If the workflow doesn’t provide adequate time, the review will not be done.
- Train for specific AI failure modes: Clinicians need to know what ambient AI hallucinates — confabulated diagnoses, misattributed symptoms, fabricated dates or addresses.
- Flag high-risk content: Configure review workflows to flag diagnoses, medication changes, referral recommendations, and clinical codes for heightened scrutiny.
- Track completion: Log whether clinicians completed review or used shortcut acceptances. Review completion rate is a governance KPI. Falling rates are an early indicator of review fatigue.
PM Responsibilities for Ambient AI Deployment
- Confirm regulatory classification before procurement. Does the product suggest diagnoses, recommend codes, or generate referral letters? If so, medical device classification may be required.
- Complete DCB0129/DCB0160 or jurisdiction-equivalent before go-live. These are the legal framework under which your organization and clinicians bear accountability for AI-generated content.
- Design consent workflow before go-live. Work with legal and privacy teams to confirm what the specific consent requirement is in each jurisdiction you operate in.
- Scope review workflow design as a project deliverable, not a post-deployment training item.
- Establish performance monitoring: hallucination rates (measured by clinician rejection and correction rates), review completion rates, adverse event reporting pathways.
Right-Sizing for Your Situation
NHS DCB0129/DCB0160 and jurisdiction-equivalent clinical safety standards; DPIA requirements for health data processing; consent framework design for ambient AI; medical device classification basics for AI scribes.
Comprehensive jurisdiction-by-jurisdiction consent requirements; review workflow design methodology; hallucination monitoring program; multi-jurisdiction deployment compliance (UK, Canada, Australia, US); clinical safety officer program.
Enterprise ambient AI governance; NHS Airlock and supplier registry compliance; multi-jurisdiction DPIA program; post-market surveillance for ambient AI medical devices; litigation readiness for consent-related claims.
Framework References
NHS England Guidance on AI-Enabled Ambient Scribing Products (2025) — DCB0129 supplier certification, DCB0160 organizational risk assessment, DPIA, Clinical Safety Officer, AVT Supplier Registry.
NHS AI Airlock Programme (October 2025) — Regulatory sandbox for clinical AI; Phase 2 includes clinical note-taking tools.
BC IPC + Ontario IPC — AI Scribe Guidelines (early 2025) — Provincial guidance on consent obligations for AI scribe use in Canadian healthcare.
AHPRA Guidance on AI in Clinical Practice (August 2024) — Practitioners ultimately responsible for AI-generated content; obligations to verify accuracy and obtain patient consent.
Sharp Healthcare v. Abridge (December 2025) — US ambient AI consent litigation; California two-party consent law applied to AI scribe recording of patient consultations.
Lawton T et al., ‘Clinicians Risk Becoming Liability Sinks for Artificial Intelligence’ (Future Healthcare Journal, 2024) — Analysis of liability allocation when AI-generated documentation errors reach patients without adequate institutional governance support.
GDPR Article 9 (EU/EEA); UK GDPR / Data Protection Act 2018; Privacy Act 1988 (Australia); PIPEDA (Canada) — Data protection obligations for processing health information through ambient AI third-party processors.
This article is part of AIPMO’s Healthcare series. See also: AI Governance in Healthcare | Algorithmic Bias in Clinical AI | Clinical Validation of Healthcare AI