AI Vendor Risk Assessment: What Goes in the DPA

AI vendors changed what a data processing agreement needs to cover, and most legal templates haven't caught up. The DPA that was fine in 2023 for a SaaS vendor doesn't address the things that actually matter when the vendor is going to send your data through a foundation model: whether your inputs train someone else's model, who the sub-processors are, what happens when the underlying model gets updated, and how deletion works when the data is now in vectors and embeddings that don't behave like rows in a database. I review a lot of these for clients, and I see the same five gaps over and over. This is the practical list of what should be in your AI vendor DPA before you sign it.

Training Data Carve-Out

The most important clause is the training carve-out. The vendor must contractually agree that customer data, prompts, outputs, and any derived artifacts are not used to train, fine-tune, evaluate, or improve any model — theirs, an upstream provider's, or anyone else's. The right language is unambiguous: 'Customer Data shall not be used to train any model, including but not limited to fine-tuning, reinforcement learning, retrieval augmentation, evaluation, or any other training-adjacent purpose, without Customer's prior written consent.' If the vendor wants telemetry to improve their product, that needs to be a separate, opt-in clause that excludes content. A surprising number of AI vendor MSAs in 2026 still default to training opt-in, and the only way to find out is to read the DPA word-for-word. Our AI Vendor Risk Scorecard walks through exactly this on a per-vendor basis.

Sub-Processor Disclosure

AI vendors typically route through at least one foundation model provider (OpenAI, Anthropic, Google, Cohere) and often through additional infrastructure (vector databases, observability platforms, embedding services). Every one of those is a sub-processor, and every one has its own data residency, training policies, and breach notification timelines. The DPA needs to name the sub-processors, require advance notice of changes, and give the customer a right to object. 'Vendor will maintain a current list of Sub-Processors at [URL] and will notify Customer at least thirty (30) days before authorizing any new Sub-Processor that processes Customer Data, during which Customer may object in writing.' If the vendor won't disclose, that's the answer.

Model-Update Notification

When the underlying model changes, the behavior of the system changes. That matters for risk assessment, bias monitoring, and any downstream automated decision-making. The DPA should require notification of material model changes — version upgrades, training-data shifts, new alignment techniques — with enough lead time for the customer to re-validate. The NIST AI Risk Management Framework treats model drift as a first-class risk; your contract should reflect that.

Deletion, Retention, and Embeddings

Deletion is harder than it sounds for AI systems. Prompts and outputs can be logged across multiple systems. Embeddings derived from customer data may persist in vector indexes even after the source documents are deleted. Cached responses can live in CDN layers. A real deletion clause spells out: retention periods for prompts, outputs, and logs (typically 30 days); deletion of embeddings and derived representations within a defined window of source deletion; certification of deletion within 60 days of contract termination; and an explicit statement that no copies are retained in backups beyond a defined backup-rotation period. 'On termination, Vendor shall, within sixty (60) days, delete all Customer Data, including derived representations, embeddings, and cached outputs, and provide written certification of such deletion.'

Incident Notification With AI-Specific Triggers

Standard breach notification clauses contemplate data exfiltration. AI systems have additional incident types that need to be in the notification clause: prompt injection that resulted in unauthorized data access; model jailbreaks that produced policy-violating outputs from your account; PII appearing in shared evaluation sets; downstream training contamination; and known-bad outputs that the vendor remediated. Notification should be required within 72 hours regardless of category, and the customer should have a right to participate in the root-cause analysis if Customer Data was implicated.

Liability That Reflects AI Risk

Standard SaaS liability caps don't reflect AI-specific risk. The damage from a foundation-model leak of regulated data can dwarf the annual contract value by orders of magnitude. Push for an uncapped indemnity for vendor breach of the training carve-out, gross negligence in handling regulated data, and IP infringement in model outputs (the last one matters because foundation models occasionally regurgitate training data). Cap negotiation is real and the vendor will push back, but the asymmetry of the underlying risk justifies the ask.

Audit Rights and Documentation

Audit rights for AI vendors should include access to: model cards and system cards for any model touching Customer Data; red-team results; bias-testing methodology and results; SOC 2 reports (or ISO 27001, ISO 42001 once it's mature); penetration tests; and the vendor's own AI governance documentation. The right phrasing isn't 'reasonable audit rights' — it's 'access on reasonable notice to the documentation reasonably necessary to verify compliance with this Agreement, including but not limited to [list].'

What I Tell Clients to Walk Away From

If a vendor refuses to put the training carve-out in writing, walk. If sub-processor disclosure is refused, walk. If deletion is described as 'commercially reasonable efforts,' negotiate or walk. If the liability cap is one month of fees and the data is regulated, negotiate. These aren't extreme positions — they're the floor for any mid-market US company that's serious about AI governance. The reason vendors push back is that they sell to companies that don't push back. The companies that do push back get reasonable terms, often without much friction, because the legal team has a draft ready for the customers who actually read.

Tying It Back to Governance

The DPA is one artifact in a larger AI governance program. The vendor risk assessment, the bias audit, the incident response runbook, and the AI inventory all need to interconnect. If you don't have the surrounding program, the DPA is a control without a process. Our work on AI Governance & Compliance treats vendor management as one of six core capabilities, alongside policy, inventory, monitoring, incident response, and training. The DPA is where the program meets the contract — and where most programs stop short of where they need to be.