- A regulatory clearance establishes that an AI met an evidence standard at a point in time. It does not guarantee the AI performs in your hospital, with your patients, in your workflow. Independent validation in your deployment environment is a separate and additional obligation.
- Health Canada’s February 2025 MLMD guidance requires training data ‘adequately representative of the Canadian population’ including skin pigmentation and biological sex — and the FDA and EU Article 10 impose equivalent obligations. Population representativeness of training data is a regulatory requirement, not a voluntary equity commitment.
- Adaptive AI that updates after deployment requires a Predetermined Change Control Plan in the US and Canada. Without a PCCP, every material model update potentially triggers a new regulatory submission. Build PCCP thinking into the development roadmap from day one — it is a project design decision, not a regulatory afterthought.
- Post-market surveillance for clinical AI is a regulatory obligation with specific reporting timelines. If your clinical AI produces a wrong output that injures a patient, MHRA Yellow Card, TGA, FDA MedWatch, and EU AI Act serious incident reporting timelines apply. Knowing these pathways before go-live is a non-negotiable deployment requirement.
- An AI with 90% aggregate accuracy may have 70% accuracy for the demographic group where diagnostic errors are most consequential. EU Article 10 makes subgroup performance testing a legal requirement. Setting acceptable thresholds before testing — not after seeing results — is what makes the test meaningful.
Clinical AI validation is the process of demonstrating that an AI system performs safely and effectively in the clinical context in which it will be used. This sounds straightforward. It is not, for three reasons that are specific to AI.
First, AI performance is population-specific. An AI trained on imaging data from one type of healthcare system, one racial demographic, or one clinical presentation pattern may not perform equivalently when deployed in a different context. The training data defines the problem the AI learned to solve. If the deployment population is different from the training population, the AI is solving a different problem than it was trained for.
Second, adaptive AI changes after deployment. Unlike a drug or a static algorithm, an AI that continues to learn from production data can drift from its validated performance profile without any specific trigger event. The validation at deployment may not reflect the AI’s current behaviour months later.
Third, regulatory authorization is not clinical validation. A regulatory clearance or approval says the AI met an evidence standard at a point in time under controlled conditions. It does not say the AI performs well in your hospital, with your patient mix, in your workflow, under your IT infrastructure. Independent clinical validation in the deployment environment is a different and additional obligation.
What Clinical Validation Requires: Five Jurisdictions
| Jurisdiction / Framework | Core Validation Requirements |
|---|---|
| US FDA (January 2025 Draft Guidance) | Pre-market submission including clinical evidence; performance testing by demographic subgroup; Predetermined Change Control Plan for adaptive AI; post-market surveillance program; Total Product Lifecycle (TPLC) approach requiring monitoring in real-world deployment. |
| Health Canada (February 2025 MLMD Guidance) | Class II–IV devices require clinical evidence appropriate to class; training data representative of Canadian population including skin pigmentation and biological sex; PCCP for adaptive AI; incident reporting obligations; peer review for higher-risk devices. |
| UK MHRA + NHS | Medical device classification under UK MDR; clinical evidence package; DCB0129 clinical safety standard (supplier); DCB0160 organisational clinical risk assessment; NHS AI Airlock regulatory sandbox for novel clinical AI; post-deployment monitoring including MHRA Yellow Card adverse event reporting. |
| Australia TGA | SaMD classification; evidence requirements scale with class (Class I–IV); ARTG registration for Class II–IV; post-market vigilance obligations; TGA incident reporting; July 2025 report recommends stricter evidence and monitoring requirements for AI clinical prediction tools. |
| EU (AI Act + MDR/IVDR) | CE marking under MDR/IVDR; clinical evaluation report; MDCG 2025-6 interplay guidance; Article 10 data governance including bias assessment; post-market monitoring; serious incident reporting to EU AI Office and national market surveillance authorities. Full MDAI compliance August 2027. |
The PCCP: Managing Adaptive AI Across Its Lifecycle
Adaptive AI — AI that learns or changes after deployment — is the most consequential regulatory development in clinical AI governance in 2024–2025. The FDA’s Predetermined Change Control Plan framework, adopted in its June 2024 final guidance and extended in the January 2025 draft guidance, allows manufacturers to define in advance what model changes are pre-authorised and what changes require a new submission. Health Canada adopted a similar PCCP framework in its February 2025 MLMD guidance. The EU and Australia are developing equivalent frameworks.
For PMs, the PCCP is a project planning document with regulatory consequences:
- What changes are pre-authorised (can happen without new regulatory submission): defined within bounds of the original PCCP — for example, model retraining on new data within the original patient population and indication.
- What changes require regulatory notification: material changes outside PCCP bounds, changes to intended use, significant performance changes.
- What constitutes a material change requiring a new full submission: changes to intended use, significant changes to performance profile, extension to new patient populations.
- How changes will be tracked and documented: version control, change log, performance monitoring before and after each update.
Without a PCCP, every significant model update potentially triggers a new regulatory authorization process. Building PCCP thinking into the AI development roadmap from the start is a significant project governance investment that pays for itself in regulatory and clinical safety clarity.
Independent Validation: What It Means in Practice
Why Vendor Validation Is Insufficient
Every clinical AI manufacturer provides validation data as part of the regulatory submission. This data was generated by the manufacturer, using data the manufacturer selected, in controlled conditions designed for the submission. It demonstrates that the AI can work. It does not demonstrate that it works in your clinical environment.
Independent external validation — conducted by researchers or clinicians with no financial interest in the AI’s approval, using data from the target deployment population — is the evidence standard that regulators and clinicians increasingly expect for higher-risk clinical AI. The NHS STANDING Together consensus recommendations (Lancet Digital Health, 2025) specifically call for transparent reporting of AI validation across diverse datasets including representation from underserved populations.
Validation in Your Environment
Even where independent external validation exists, deploying health organisations have an additional obligation: confirming the AI performs acceptably in their specific clinical environment. This is sometimes called local validation or site validation. The relevant questions:
- Does the AI’s performance in our patient population match the published validation? Is our demographic and clinical mix similar to the validation population?
- Does the AI integrate correctly with our clinical systems? Does it receive the data it was designed to receive, in the format it expects?
- Does it perform consistently under our IT infrastructure? Are there latency, connectivity, or data quality issues that affect performance?
- Does it behave consistently in our clinical workflow? Are clinicians using it as intended, or are they bypassing or misapplying it in ways that create risk?
Subgroup Performance Validation
Performance by demographic subgroup is the most important dimension of clinical validation from a health equity and patient safety perspective. Aggregate performance metrics can mask significant performance disparities within subgroups. A diagnostic AI with 92% overall accuracy may have 78% accuracy in the demographic group for which diagnostic errors are most consequential.
EU AI Act Article 10 makes subgroup performance analysis a legal requirement for high-risk AI including medical devices. Health Canada’s 2025 MLMD guidance requires representative data including demographic factors. The FDA’s guidance emphasises subgroup analysis as a component of clinical evidence. NHS clinical governance expectations require subgroup performance to be disclosed to clinical users.
Post-Market Surveillance: The Ongoing Obligation
Regulatory authorization is not a one-time gate. Every major clinical AI regulatory framework requires ongoing post-market monitoring and incident reporting:
| Jurisdiction | Post-Market Obligation |
|---|---|
| US | FDA MedWatch reporting for adverse events; post-market surveillance program; TPLC monitoring; PCCP performance tracking. |
| UK | MHRA Yellow Card reporting for medical device incidents and near misses; NHS clinical governance monitoring; NHS England guidance requires performance tracking and incident escalation. |
| Canada | Manufacturer incident reporting to Health Canada for device failures or deterioration in effectiveness; post-market monitoring under PCCP; provincial patient safety reporting. |
| Australia | TGA incident reporting; ARTG post-market vigilance; clinicians required to report to TGA as per ACSQHC guidance (August 2025). |
| EU | Serious incident reporting to EU AI Office and national market surveillance authority; post-market monitoring system under AI Act Article 72; MDR/IVDR vigilance reporting. |
For deploying health organisations (as distinct from AI manufacturers), post-market obligations include: monitoring AI performance in production, detecting and investigating adverse events involving AI, reporting to the appropriate regulatory authority, and notifying the AI manufacturer of identified issues. In the UK, the DCB0160 clinical risk assessment and ongoing safety case include post-deployment monitoring obligations.
PM Responsibilities for Clinical Validation
Planning
- Identify the regulatory classification and authorization pathway in every deployment jurisdiction. This determines the evidence standard and the timeline.
- Determine whether the AI is adaptive. If so, scope PCCP development as a project deliverable before regulatory submission.
- Plan for independent validation in your deployment environment as a project phase, not an afterthought. Budget, timeline, and clinical staff time for site validation must be in the project plan.
Development and Procurement
- For vendor AI: request the full clinical validation package including subgroup performance data by age, sex, race/ethnicity, and clinical presentation. If the vendor cannot provide subgroup performance data, that is a risk assessment decision, not a procurement default.
- Review the manufacturer’s PCCP or equivalent change control plan. Understand what model updates are planned, when, and what clinical safety review they trigger.
- Confirm the manufacturer’s post-market surveillance program and the organisation’s obligations as a deployer under applicable regulatory frameworks.
Deployment and Post-Deployment
- Complete organisational clinical safety assessment (DCB0160 in UK or equivalent) before go-live, incorporating site validation findings.
- Establish performance monitoring from day one: accuracy by demographic subgroup, clinical override rates, adverse events, and near misses.
- Confirm incident reporting pathways to regulators are known, documented, and accessible to clinical staff before go-live.
Key Questions for Clinical AI Validation
- What is the regulatory classification in each deployment jurisdiction, and what evidence standard does it require?
- Is there published subgroup performance data for the demographic groups in our patient population? Does performance hold for the groups where errors are most consequential?
- Has independent external validation been conducted in a comparable clinical context?
- Is the AI adaptive? If so, what is the PCCP or equivalent change control plan? What changes will require regulatory notification?
- Has site validation in our clinical environment been completed and documented before go-live?
- Are post-market monitoring, adverse event reporting pathways, and incident escalation procedures in place before go-live?
Right-Sizing for Your Situation
Regulatory classification basics across US, UK, Canada, Australia, and EU; minimum evidence standards by risk class; site validation methodology; post-market reporting obligations.
Comprehensive jurisdiction-by-jurisdiction validation requirements; PCCP design methodology; subgroup performance validation framework; independent validation design; post-market surveillance program setup; multi-jurisdiction regulatory mapping.
Enterprise clinical AI validation program; EU AI Act Article 10 data governance compliance for MDAI; validation quality management integration; adaptive AI change governance across jurisdictions; validation evidence management for multi-jurisdiction regulatory submissions.
Framework References
FDA Draft Guidance: AI-Enabled Device Software Functions Lifecycle Management (January 2025) — TPLC approach, PCCP framework, subgroup performance analysis, post-market surveillance.
Health Canada Pre-Market Guidance for Machine Learning-Enabled Medical Devices (February 2025) — Clinical evidence by class, representativeness requirement including demographic factors, PCCP, incident reporting.
EU AI Act (Reg. (EU) 2024/1689) Article 10 — Data governance and bias obligations for MDAI including demographic representativeness. MDCG 2025-6 (June 2025) for MDR/IVDR interplay. Full MDAI compliance August 2027.
Australia TGA — Clarifying and Strengthening the Regulation of Medical Device Software including AI (July 2025) — Stricter evidence and monitoring requirements; SaMD grace period ended November 2024.
NHS STANDING Together Consensus Recommendations (Lancet Digital Health, 2025) — Algorithmic transparency standards including subgroup representation reporting for NHS AI deployments.
Royal College of Radiologists — AI Deployment in the NHS: Reviewing Progress and Defining Future Action (April 2025) — UK clinical governance expectations for AI validation, subgroup performance disclosure, post-market surveillance.
NIST AI RMF 1.0 — MEASURE 2.5 (performance evaluation), MEASURE 2.11 (demographic subgroup bias testing), MANAGE 4.1 (continuous monitoring).
This article is part of AIPMO’s Healthcare series. See also: AI Governance in Healthcare | Algorithmic Bias in Clinical AI
To err is AI; to govern, human.