- Traditional insurance AI governance doesn’t transfer to LLMs. A gradient-boosted decision tree is deterministic — the same inputs always produce the same output and feature importance can be measured. A large language model is neither. Governance must be redesigned for GenAI, not just extended.
- EIOPA’s April 2025 GenAI analysis identified three governance gaps traditional frameworks don’t address: reliability (probabilistic outputs), auditability (no feature importance equivalent), and concentration risk (most insurer GenAI depends on a small number of foundation model providers). Each requires a distinct governance response.
- A chatbot that answers coverage questions and makes a misrepresentation creates the same liability as a human agent who says the same thing — but at scale, with confident-sounding language that doesn’t signal uncertainty. EU AI Act Article 52 requires disclosure when customers interact with AI systems. Design so customers know.
- DORA (in force January 2025) requires EU insurers to manage concentration risk among critical ICT providers. Most insurer GenAI deployments share a small number of foundation model providers — a service disruption or model deprecation at one provider hits multiple insurers simultaneously. Assess this risk and build contingency into vendor contracts.
- AI washing — overstating AI capabilities in marketing, investor disclosures, or regulatory filings — is an active enforcement area after the SEC brought actions against investment advisers for misleading AI disclosures in 2024. PMs are responsible for ensuring what the organization says the AI does accurately reflects what it actually does.
Generative AI — large language models, image generation systems, and multimodal AI — is arriving in insurance at a speed that governance frameworks were not designed for. EIOPA’s 2024 Digitalisation Survey found that 50% of European non-life insurers were already using AI, with most using traditional supervised learning. The commoditization of GenAI through accessible APIs has lowered the barrier to adoption dramatically. By 2025, insurtechs and established carriers alike were deploying LLMs for policy document drafting, claims correspondence, customer service, regulatory reporting summarization, and fraud narrative analysis.
The governance problem is that traditional insurance AI governance — validation against ground truth, feature importance analysis for bias testing, change control against a defined model version — was designed for deterministic models. GenAI is not deterministic. The same prompt can produce different outputs. The contribution of specific input variables to a specific output cannot be audited the way gradient-boosted decision tree features can. And the model updates in ways that are not transparent to the deploying insurer when the foundation model provider updates its base model.
EIOPA’s April 2025 analysis — From Traditional AI to Generative AI: Implications for the Insurance Sector — is the most operationally relevant regulatory analysis of this shift. This article builds on EIOPA’s framework.
GenAI Use Cases in Insurance: A Risk-Tiered View
| Use Case | Governance Risk | Risk Tier |
|---|---|---|
| Policy document drafting and summarization | Hallucinated coverage terms; policy summaries that misrepresent actual policy language; customer reliance on AI-generated summary rather than policy document | High — coverage representation risk; potential basis for claim denial dispute |
| Claims correspondence generation | Denial letters with hallucinated or incorrect coverage rationales; correspondence that creates legal admissions; tone that violates unfair claims settlement requirements | High — bad faith exposure; legal admission risk; regulatory compliance |
| Customer service chatbots (coverage questions) | Coverage misrepresentation by chatbot; customers relying on AI response for coverage decisions; EU AI Act Article 52 disclosure obligation | High — misrepresentation liability; disclosure obligation |
| Underwriting narrative and declination letters | AI-generated rationales that do not reflect the actual underwriting decision or introduce discriminatory language | High — accuracy obligation; anti-discrimination compliance |
| Fraud investigation narrative analysis | Hallucinated connections or fabricated details in investigation reports; use of AI-generated conclusions in denial decisions | High — accuracy requirement; bad faith exposure if denial based on incorrect AI narrative |
| Regulatory reporting and compliance documentation | AI summarization errors in regulatory filings; compliance documents not accurately reflecting actual practices | Medium — regulatory accuracy obligation; potential misleading disclosure |
| Internal productivity (drafting, summarization) | Hallucinated facts in internal documents; inaccurate competitive analysis; confidential data sent to foundation model providers | Medium — data governance; operational accuracy |
The Three Governance Gaps GenAI Creates
The Reliability Problem
Traditional insurance AI models are deterministic: the same inputs produce the same output every time. This is the foundation of traditional model governance — you can test the model on a held-out dataset, confirm its performance, and rely on that performance persisting in production because the model does not change.
GenAI is probabilistic. The same prompt can produce different outputs. An LLM that generates a policy summary, a claims denial letter, or an underwriting rationale today will not necessarily generate the same document tomorrow, even with the same inputs. This creates a governance gap: validation of a GenAI output at a point in time does not predict the distribution of future outputs with the same reliability as traditional model validation.
The governance response: define the acceptable output parameters for each GenAI use case, implement automated output validation to flag outputs outside acceptable parameters before they are used or delivered to customers, and establish human review protocols for high-risk output categories (customer-facing coverage statements, claims denial communications, formal regulatory documents).
The Auditability Problem
Traditional model governance relies on the ability to explain why a model produced a specific output: feature importance scores, SHAP values, decision path documentation. These tools allow auditors, regulators, and litigants to verify that a specific input set produced a specific output for documented reasons.
LLMs do not provide equivalent auditability. The contribution of specific input tokens to a specific output cannot be measured with the reliability of gradient-boosted decision tree feature importance. A claims denial letter generated by an LLM that contains an incorrect coverage statement cannot be audited to determine whether the incorrect statement came from a hallucination, a training data artifact, a prompt design flaw, or context window interference.
The governance response: for high-risk GenAI outputs, retain the prompt, the model version, the context window, and the output as a complete audit record. This record establishes what inputs produced what output and at what model version — the closest equivalent to traditional model audit trails. Store this record for the litigation limitation period applicable to the document type.
The Concentration Risk Problem
Most insurer GenAI deployments use foundation models provided by a small number of large technology companies. When a foundation model provider updates its model, every insurer using that model experiences a change in AI behavior simultaneously — without any notification that the change occurred and without any ability to re-validate before the change takes effect.
EIOPA explicitly identified this as a risk in its April 2025 analysis: insurer adoption of GenAI increases dependence on a reduced number of third-party service providers. DORA, in force January 2025, requires EU insurers to assess and manage concentration risk among critical ICT providers. A foundation model provider that several major EU insurers depend on for claims processing or policy documentation is, in practical terms, a critical ICT provider for each of them.
The governance response: assess foundation model provider concentration risk as part of the AIS Program and DORA third-party risk governance. Maintain contingency plans for provider disruption, model version deprecation, or terms changes. Establish notification requirements in vendor contracts for material model updates.
Customer-Facing GenAI: The Misrepresentation Risk
Chatbots, virtual assistants, and AI-generated customer communications are among the most common GenAI applications in insurance. They are also among the highest-risk from a regulatory and liability perspective.
Insurance is a contract. What the insurer communicates to the policyholder about coverage creates representations that can become legally binding or create estoppel. A human agent who tells a customer that their policy covers a specific loss creates a misrepresentation claim if it does not. An LLM chatbot that tells the same customer the same thing creates the same risk — but at a scale and with a confidence of presentation that makes the customer more likely to rely on it, not less.
- Disclosure: EU AI Act Article 52 requires that customers be informed when they are interacting with an AI system, unless this is obvious. EIOPA’s Opinion requires that AI-generated outputs be explainable to customers in clear and comprehensible language. For customer-facing GenAI: tell the customer they are interacting with an AI; design the interaction so the AI’s limitations are apparent, not obscured by confident-sounding language.
- Coverage statement accuracy: GenAI chatbots that answer coverage questions must be constrained to accurate policy information. The technical solution is retrieval-augmented generation (RAG) that grounds the model’s responses in the specific policy terms — but even RAG-grounded systems require human review of high-stakes coverage statement patterns.
- Escalation pathways: Customer-facing GenAI must have clear escalation pathways to human agents for complex coverage questions, claims reporting, and any query where the customer indicates dissatisfaction with the AI response. Deploying GenAI without an accessible human escalation path is a Consumer Duty failure in the UK, an IDD fair treatment failure in the EU, and a potential unfair claims settlement practices issue in the US.
AI Washing: The Disclosure Risk
AI washing — overstating the capabilities of AI systems in marketing materials, investor disclosures, or regulatory filings — is an emerging enforcement area. The SEC brought enforcement actions against investment advisers in 2024 for misleading AI disclosures. EIOPA’s Chair noted at the September 2025 Singapore conference that the governance of AI is “rather fragmented and challenging to grasp at a global level,” creating conditions where misrepresentation about AI capabilities can persist undetected.
For insurers, AI washing risk appears in specific contexts:
- Marketing: claiming AI-driven pricing is more accurate or fairer than it is; claiming AI claims processing is faster and error-free without disclosing error rates and human review requirements.
- Investor and regulatory disclosures: overstating the operational benefits or readiness of GenAI implementations; understating the governance gaps and validation limitations.
- Procurement and sales: vendors claiming GenAI capabilities that the model does not reliably deliver; insurers representing AI governance programs to regulators that do not reflect actual practice.
The governance obligation is straightforward: describe AI systems accurately in all communications. PMs are responsible for ensuring that project documentation, launch communications, and ongoing operational descriptions of what the AI does are accurate.
The FCA AI Live Testing Opportunity
The FCA launched AI Live Testing in October 2025, allowing UK financial firms including insurers to validate AI models under direct regulatory engagement before wider deployment. This is a significant opportunity for insurers developing novel GenAI applications in high-risk areas — claims processing, underwriting, customer-facing advice — to validate their governance approach with regulator visibility before deployment, reducing the risk of enforcement action based on unknown regulatory expectations.
The FCA’s Supercharged Sandbox (launched June 2025) similarly provides a testing environment for AI experiments under regulatory oversight. For UK insurers uncertain whether a GenAI application meets Consumer Duty obligations, these programmes are the most direct path to regulatory clarity.
PM Responsibilities for GenAI in Insurance
- Risk-tier every GenAI use case before deployment. Customer-facing coverage representations, claims denial communications, and formal regulatory documents require the highest governance intensity. Internal productivity tools require less. Do not apply uniform governance to all GenAI — it will be either too onerous for low-risk uses or too light for high-risk ones.
- Design output validation before deployment for high-risk use cases. What does an acceptable output look like? What automated checks can flag unacceptable outputs before they reach customers or regulators? What human review process applies to flagged outputs?
- Establish the audit record architecture before deployment. For high-risk GenAI outputs: retain prompt, model version, context, and output. Store for the applicable limitation period. This is the discovery record if there is litigation.
- Assess foundation model provider concentration risk as part of the AIS Program and DORA third-party risk governance. Map which insurers share common foundation model providers and what the impact of provider disruption or model deprecation would be.
- Brief legal, compliance, and marketing on AI washing risk. Require review of AI capability claims in marketing materials, investor disclosures, regulatory filings, and procurement submissions before publication.
Right-Sizing Your AI Governance Approach
GenAI vs. traditional AI governance gaps; use case risk tiering; output validation basics; customer-facing disclosure requirements (EU AI Act Article 52, FCA Consumer Duty); AI washing risk fundamentals.
Comprehensive reliability and auditability gap analysis; output validation framework design; RAG-grounded customer-facing AI architecture; foundation model concentration risk assessment; DORA third-party governance for GenAI vendors; FCA AI Live Testing programme.
Enterprise GenAI governance program; EIOPA GenAI guidance alignment; DORA critical ICT provider framework for foundation model concentration risk; EU AI Act GPAI model obligations for insurers; audit record architecture; AI washing risk management program.
The AI Governance Advisor can help you design a risk-tiered governance framework for your GenAI use cases — start with a free Essential account.
Framework References
EIOPA ‘From Traditional AI to Generative AI: Implications for the Insurance Sector’ (April 2025) — Three governance gaps for GenAI: reliability, auditability, concentration risk. Primary EIOPA analysis of GenAI-specific insurance governance.
EU AI Act (Reg. (EU) 2024/1689) — Article 52 (disclosure when customers interact with AI systems); Title VI (obligations for general-purpose AI models including LLMs, effective August 2025); Recital 99 (AI system deployment by insurers).
DORA (Reg. (EU) 2022/2554, in force January 17, 2025) — Third-party ICT risk governance; concentration risk among critical ICT providers including foundation model vendors; mandatory audit rights; incident reporting; exit provisions.
EIOPA Opinion on AI Governance and Risk Management (August 6, 2025) — Principles-based governance for all insurance AI; explainability to customers; redress mechanisms; applies to GenAI as much as traditional AI.
FCA AI Live Testing / Supercharged Sandbox (October 2025) — UK regulatory testing environment for AI in financial services including insurance; mechanism for novel GenAI governance validation before deployment.
NAIC Model Bulletin: Use of Artificial Intelligence Systems by Insurers (December 2023) — AIS Program requirements apply to GenAI; documentation and governance obligations extend to LLM deployments in regulated insurance processes.
SEC enforcement actions on AI washing (2024) — Precedent for enforcement against misleading AI capability claims in regulated industries; applicable risk for insurance AI marketing, investor, and regulatory disclosures.
This article is part of AIPMO’s Insurance series. See also: AI Governance in Insurance | AI in Insurance Underwriting | AI Governance in Financial Services | AI Governance in Healthcare