- The tool you have been tracking has a new name. What the NAIC called the AI Systems Evaluation Tool is now the AI Risk Evaluation Supplement. The renaming is not cosmetic. It reflects the working group’s decision to position the document as a supplement to existing examiner handbooks rather than a freestanding instrument, which means it inherits the handbooks’ definitions, materiality thresholds, and inquiry protocols.
- The comment window on version 5.0 closed September 29, 2026. The pilot has finished, NAIC staff plan to revise the document through October, and the Fall National Meeting in November is where adoption gets considered. The window for arguing about scope has shut; what is left is getting ready for the version that goes to Fall.
- Regulators now explicitly ask for a Model Inventory. Earlier versions implied the request. Version 5.0 lists it. An inventory that exists as a maintained artifact is a different thing from an inventory a team could assemble if asked, and the difference shows up under exam-clock pressure.
- The checklist asks for document names and page numbers. Exhibit B’s checklist version now asks companies to provide the document name and page number supporting each response, so regulators can connect answers to the documents provided. That is a documentation-architecture requirement wearing the costume of a form field, and it is the clearest signal yet of what audit-ready means in practice.
- This is not only a market conduct tool. Pilot states are using the supplement in market conduct exams and reviews, in financial analysis, and in financial exams. A governance program scoped only to consumer-facing models is answering half the question, because the supplement asks separately about models with material financial impact.
For two years, insurer AI governance has been a document problem in the abstract. The NAIC Model Bulletin told carriers to maintain a written AI Systems Program. Twenty-six jurisdictions adopted that expectation. Nobody was quite sure what a regulator would actually ask for when the time came.
That question now has a published answer, and it runs to four exhibits. The NAIC’s Big Data and Artificial Intelligence (H) Working Group has spent 2026 piloting a structured examination document with twelve states, and the current draft — version 5.0, whose comment window closed September 29 — is specific in a way the bulletin never was. It asks for an inventory. It asks which models touch consumers and which move money. And in its checklist form, it asks the company to say which document, and which page, supports each answer.
That last detail is the one worth sitting with. A governance program can be genuinely good and still fail that request, because pointing to page 14 of a named document requires that the document exist, that it be current, and that someone know where the answer lives. Most programs are built to satisfy an auditor who will read the whole binder. This one is built for an examiner who wants the citation.
A Rename That Tells You Something
Through the pilot, the document was the AI Systems Evaluation Tool. It is now the AI Risk Evaluation Supplement, and NAIC staff have explained the change plainly: during earlier comment periods, stakeholders pushed back on how broadly the document might be used, so text was added emphasizing that it is a supplement to existing handbooks. Decisions about who receives inquiries are made under handbook guidance, not under the supplement itself.
Three consequences follow, and they matter more than the name.
- Materiality is now aligned to the Financial Condition Examiners Handbook definition. Carriers already living with that definition in financial exams do not get a second, AI-specific version of it.
- Risk assessment centers on inherent risk — the risk before mitigating controls are considered. Your controls are the answer to a different question. They do not lower the risk rating that determines how much attention a model receives.
- Machine learning guidance was added, with generalized linear models offered as an example, and regulators may limit further inquiry after Exhibit A responses come in. Scope gets set early, from what you report first.
The working group heard the industry’s concern about scope creep and responded by tightening the document’s stated purpose rather than by narrowing what it asks. For a program manager, that is the more demanding outcome: the questions stayed specific, and the framing got more institutionally anchored.
What the Pilot Actually Was
Twelve states ran the pilot from March through September 2026: California, Colorado, Connecticut, Florida, Iowa, Louisiana, Maryland, Pennsylvania, Rhode Island, Vermont, Virginia, and Wisconsin. They used it across property and casualty, life, and health, focused on domestic insurers, and applied proportionality — more time on high-risk AI systems that could cause serious consumer or financial harm, less on low-risk back-office systems.
Two design choices in that pilot are worth a carrier’s attention. First, participating states coordinated on calls to avoid repeating requests, which tells you the multistate exam pattern is being rehearsed now rather than improvised later. Second, each jurisdiction retained authority to modify the document for its own needs. The supplement is a common spine, not a uniform questionnaire, and a carrier writing in several pilot states should expect family resemblance rather than identical requests.
Proportionality is the carrier’s lever, and it only works if the inventory is real. If a program cannot demonstrate which systems are high-risk and which are low-risk back-office tools, it cannot make the proportionality argument, and the examiner’s own read of materiality becomes the operative one.
The Four Exhibits, as Deliverables
The supplement is organized as Exhibits A through D. Read as an examiner would, they are four different artifacts with four different natural owners — which is exactly why programs that assign all of it to one compliance lead tend to struggle.
| Exhibit | What it asks for | The artifact it implies |
|---|---|---|
| A | Scoping and inventory. AI systems versus AI models, model types, and separate reporting for models with direct consumer impact and models with material financial impact. Regulators now explicitly request a Model Inventory. | A maintained model inventory with impact classification — owned by the PMO or model risk function, not assembled on demand. |
| B | The AIS Program itself, in narrative or checklist form. Includes new questions on explainability and transparency, on materiality, and on how the company oversees third-party models. | A current, versioned program document whose sections map to the questions — plus a citation index for the checklist version. |
| C | The per-model record: use case and purpose, model type, how the model was developed, and separately its risks and its limitations, rated high, moderate, or low. | A model documentation template applied consistently, with risk and limitation treated as distinct fields. |
| D | The data layer: data sets in use, which models rely on them, and how the information is used. Regulators may request a data dictionary, and may customize the exhibit by line of business. | A data dictionary and lineage mapping from data set to model — usually the least mature artifact in a carrier’s AI program. |
Exhibit D deserves a particular note. It asks about data types including ethnicity and race, and the regulator instructions acknowledge that some data types may not be pertinent to every line of business. The exhibit is not an accusation. It is a mapping request, and the carriers that handle it well will be the ones that can already say which models consume which fields without a discovery exercise.
What Changed in Version 5.0
NAIC staff published a summary of the most impactful changes alongside the draft. Several are pilot-driven refinements of interest mainly to the states. These are the ones that change what a carrier has to have ready.
- Implied requests were made explicit. The prior version implied several asks; they are now listed, and the Model Inventory is the most notable of them.
- Exhibit B’s checklist version asks for document name and page number, so responses can be tied to the documents provided.
- Two new checklist questions arrived: one on materiality, and one on how a company oversees third-party models. Vendor oversight is no longer a paragraph in the program narrative; it is a direct question.
- A new narrative inquiry covers explainability and transparency, alongside added guidance for responding on models with direct consumer impact.
- Exhibit C split model risks from model limitations into separate fields, with a high, moderate, or low rating on limitations. Conflating the two now produces a visibly incomplete answer.
- Definitions were added for agentic AI, for AI Model drawing on NIST and executive order language, for AIS Program drawing on the Model Bulletin, for direct consumer impact, for material financial impact, and for third parties. The agentic AI definition arrives before the exhibits do much with it — worth noting, not yet worth restructuring a program around.
The Scoping Lever Most Carriers Will Miss
Buried in the Exhibit A changes is a provision with more operational consequence than its placement suggests. Either the regulator specifies a materiality threshold for Exhibit A responses, or the company is asked to specify the threshold it used to scope its own responses — and to disclose it.
That is a governance decision with teeth. Set the threshold too low and the carrier volunteers a wide surface for inquiry. Set it too high and the company is explaining, on the record, why a model it excluded did not meet its own stated bar. The threshold is also disclosed, which means it becomes a durable artifact that a future examiner can compare against the inventory.
This should not be decided by whoever is assigned to complete the form. It belongs to the same governance body that approves the program document, it should be documented with its rationale, and it should be applied consistently across exhibits. A carrier that can show a reasoned, minuted threshold decision is in a materially stronger position than one that can only show a number.
What to Do Before November
The pilot has closed and the comment window shut on September 29. The working group meets October 8 to continue public discussion, and the NAIC’s published timeline has the updated document considered for adoption at the Fall National Meeting in November. None of that requires a carrier to guess at the final text, because the work that pays off is the same under any plausible version.
- Run Exhibit A against your own inventory as a dry run. Not as a compliance exercise — as a test of whether the inventory can produce model type, direct consumer impact, and material financial impact without a two-week scramble.
- Build the citation index. For each question in the checklist, name the document and page that answers it. Where no document answers it, that gap is the finding, and you found it rather than an examiner.
- Make the materiality threshold a governance decision, minuted, with rationale, before anyone needs it under deadline.
- Separate risk from limitation in your model documentation template now, so Exhibit C is a retrieval task rather than a drafting task.
- Assign Exhibit D an owner. Data lineage tends to be the artifact with no natural home, which is why it is usually the one that is missing.
The honest framing for a PMO is that none of this is new work invented by the NAIC. It is the documentation discipline that a governance program was supposed to produce anyway, now with a published format and a date attached. The difference between carriers in 2027 will not be who has a program. It will be who can cite it.
Framework References
AI Risk Evaluation Supplement, version 5.0 (NAIC Big Data and Artificial Intelligence (H) Working Group, exposed August 31, 2026; comment period closed September 29, 2026) — Exhibits A through D and definitions cited.
AI Risk Evaluation Supplement — Summary of Changes from version 4.0 (NAIC staff, 2026) — Source for the version 5.0 changes described here.
AI Systems Evaluation Tool Pilot: Pilot Project Summary (NAIC, 2026) — Participating states, timeline, proportionality approach, and November adoption consideration.
Big Data and Artificial Intelligence (H) Working Group Summer National Meeting Summary (NAIC, August 14, 2026) — Pilot start date, participating state count, and scope across market conduct and financial work.
NAIC Model Bulletin: Use of Artificial Intelligence Systems by Insurers (NAIC, December 4, 2023) — Source of the AIS Program expectation the supplement examines against.
Implementation of NAIC Model Bulletin: Use of Artificial Intelligence Systems by Insurers (NAIC, status as of August 31, 2026) — Adoption status: 25 states and the District of Columbia, plus four states under their own frameworks.
This article is part of AIPMO.co’s Insurance series. See also: NAIC AI Bulletin Adoption: Q2 2026 State-by-State Status | AI Governance in Insurance | AI in Insurance Underwriting | AI Governance in Financial Services
To err is AI; to govern, human.