Your AI Vendor Said It Was Tested. Ask Which Model.
“We test our models for safety” is not an approval record.
An operating business needs to know which model was tested, which version, under what conditions, against which risks, with what results, and whether the evidence existed before that version entered production. Without those details, a vendor’s safety language may be sincere but still unusable for procurement, change control, incident review, or audit.
A new analysis of public model disclosures makes the documentation gap visible. Its business lesson applies beyond any one country or developer: approve the exact system you intend to use, not the vendor’s reputation, a family name, or a general statement about responsible AI.
What changed
Documented study finding: On October 8, 2026, SemiAnalysis published a review of 857 identifiable releases from nine leading Chinese AI developers between 2021 and September 15, 2026. The dataset included 741 product models and 116 research models.
The researchers reported that 31 releases—3.6%—had a published safety-evaluation result that could be matched to the specific model. Only nine releases—1.1%—had such a result available at or before release. They found no public safety disclosure for 813 releases.
What counted: SemiAnalysis required a quantitative or substantive result tied to a named model. It included findings involving harmful output, jailbreak resistance, toxicity, privacy, refusal behavior, or dangerous capabilities. A generic statement that a model was “safety-trained” or “evaluated” did not qualify.
Independent reporting: Reuters reported the findings on October 9 and emphasized that the study measured public disclosure—not whether developers performed private testing. Reuters also noted that the report did not provide comparable disclosure rates for U.S. developers.
Study limitations: SemiAnalysis said “not found” was limited to the developer materials its researchers checked and did not mean “not tested.” It also warned that companies name and release variants at different levels of granularity, so company-level rates should be treated as indicative rather than a definitive ranking.
What remains unresolved: Public sources do not establish the quality of every undisclosed internal evaluation, whether customers received private evidence under contract, how often published tests reflect real deployment configurations, or whether disclosure itself correlates with lower incident rates. The study documents an evidence-availability gap; it does not prove that 96.4% of the models were unsafe.
A company policy is not model evidence
Vendors often publish responsible-AI principles, security programs, model cards, system cards, red-team summaries, or compliance statements. Those materials can be useful. They do not automatically apply to every model size, snapshot, fine-tune, hosting configuration, tool connection, or later update.
A test performed on a flagship model should not be inherited by a smaller variant without evidence. A base-model evaluation may not describe a fine-tuned customer-service model. A chatbot test may not describe an agent that can send email, change an order, run code, or update an ERP. A result from one release may become stale after a silent model update.
This is why vendor and model risk review needs two linked records: one for the provider and one for the exact production system. The provider record addresses the company, contracts, security, data use, financial viability, incident practices, and dependencies. The system record addresses the model version, configuration, integrations, permissions, tests, operating limits, and approved use.
Require a version-specific evidence packet
Before approving a model for a material workflow, collect an evidence packet that answers:
- Identity: provider, model family, exact version, release date, API or weight identifier, region, hosting method, and configuration.
- Change state: whether the model is fixed, aliased, continuously updated, fine-tuned, routed among models, or modified by retrieval and tools.
- Evaluation scope: which safety, security, privacy, accuracy, bias, reliability, and misuse risks were tested.
- Method: evaluation datasets, scenarios, graders, thresholds, sampling, red-team process, and test environment.
- Results: measured findings before and after mitigations, failure modes, uncertainty, exclusions, and known limitations.
- Deployment relevance: whether the tests included the permissions, tools, data, context length, automation level, and user population planned for production.
- Independent evidence: third-party assessments, customer testing rights, certifications, penetration tests, or other review appropriate to the risk.
- Response commitments: change notices, incident notification, remediation, version pinning, export, rollback, and termination support.
The packet does not need to expose proprietary model weights or confidential training data. It needs enough traceable evidence for the buyer to understand what was tested, what was not, and whether the result is relevant to the intended workflow.
Test the workflow, not just the model
Vendor evidence is one input. The buyer still has to run acceptance tests against its own process, data, permissions, integrations, and failure consequences.
For an industrial distributor, test part-number ambiguity, supersessions, contract pricing, inventory location, lead time, product compatibility, export restrictions, credit status, and approval thresholds. For eCommerce, test catalog facts, claims, restricted products, promotion rules, returns, customer identity, tax, and payment boundaries. For manufacturing, test released revisions, material status, capacity, maintenance constraints, quality limits, safety rules, and change authorization.
Use deterministic validation for facts that must be exact. The model may draft, classify, compare, or recommend, but authoritative systems should validate identifiers, prices, quantities, tolerances, dates, approvals, and transaction state before action.
Record expected results and pass/fail criteria before testing. A demonstration chosen by the vendor can show capability; it cannot substitute for production acceptance.
Match evidence depth to operating risk
Not every AI use needs the same review. A useful tiering model is:
- Low risk: brainstorming, formatting, internal drafts, or summaries with no sensitive data and no direct system action.
- Moderate risk: recommendations or content that can affect customers, operations, or decisions but passes through qualified human review.
- High risk: autonomous or semi-autonomous changes to money, inventory, production, customer commitments, regulated records, employee decisions, security, or safety.
As risk rises, require more specific evidence, broader acceptance testing, narrower permissions, stronger monitoring, more independent review, and faster rollback. This keeps governance proportional and supports adoption instead of forcing every experiment through the same process.
Do not allow silent model substitution
Contracts and technical architecture should identify what constitutes a material model change. That can include a new version, alias update, provider routing change, fine-tune, safety-policy change, tool permission, retrieval source, context-window change, or fallback model.
Require advance notice where feasible, a change record, regression testing, approval for high-risk workflows, and a defined path to pin or return to a known-good version. If the provider cannot offer version stability, the workflow needs stronger monitoring, smaller action authority, and a tested alternative.
Connect model changes to business-system change control. A model update is not merely a software release when it can alter classifications, recommendations, tool selection, or transaction behavior.
Separate evaluation from approval
The person sponsoring a tool should not be the only person deciding whether its evidence is sufficient. Assign distinct roles:
- a business owner who defines the outcome and accepts process risk;
- a technical owner who validates integration, configuration, and recovery;
- a data or security reviewer who examines access, retention, and threats;
- a workflow expert who validates outputs and exceptions; and
- an accountable approver for high-impact deployment or change.
Use human approval controls where the system can create consequential commitments. Approval should capture the evidence reviewed, unresolved limitations, operating boundaries, monitoring plan, and expiration date.
Turn limitations into operating controls
A disclosed weakness does not always mean a model is unusable. It may mean the model needs a narrower job.
If a model struggles with exact numeric reasoning, calculate totals outside the model. If it is vulnerable to untrusted instructions, isolate external content and restrict tools. If it cannot reliably distinguish product revisions, require deterministic revision checks. If it produces inconsistent marketing claims, require a claims library and publication approval. If it cannot support a safety-critical decision, keep that decision outside its authority.
This constructive approach turns evaluation evidence into workflow design. The right question is not simply “Is the model safe?” It is “Under which conditions can this system produce useful work with controlled consequences?”
Monitor the production evidence
Pre-release tests are snapshots. Production monitoring should measure:
- acceptance, correction, and override rates;
- error and exception categories;
- unauthorized tool or data-access attempts;
- policy refusals and inappropriate non-refusals;
- customer or employee complaints;
- latency, availability, and fallback usage;
- model, prompt, connector, and knowledge-source changes;
- human-review time and rework; and
- incidents, near misses, containment time, and recovery time.
Set thresholds that trigger investigation, reduced authority, rollback, or suspension. A dashboard without an owner and response rule is observation, not control.
Put an expiration date on approval
Model approval should expire. Recertify after material changes, significant incidents, new permissions, new data classes, expanded user groups, different jurisdictions, or a defined time interval.
At recertification, compare the current system to the approved evidence packet. Confirm the model version, configuration, tools, data flows, controls, test results, vendor terms, and manual fallback. Close obsolete approvals so an old review cannot be used to justify a different production system.
Own the evidence gaps
A vendor may be unable or unwilling to provide every requested artifact. Treat each missing item as an explicit exception with a named owner, rationale, compensating controls, expiration, and decision authority.
Possible compensating controls include reduced permissions, draft-only use, smaller transaction limits, mandatory human review, independent testing, isolated data, additional monitoring, shorter approval duration, or a parallel manual process. For a high-risk gap without an adequate control, the right answer may be “not yet.”
Document that decision in the organization’s AI governance register. If an incident occurs, the business should be able to show what it knew, what it did not know, who accepted the limitation, and how the workflow was constrained.
Plan rollback before deployment
Maintain a known-good model or non-AI process, preserved configuration, versioned prompts and rules, recoverable data changes, and the authority to suspend automation. Test the manual path under realistic volume rather than assuming employees can absorb the work.
The incident-response plan should identify who can disable the model, revoke tools, isolate data, freeze affected transactions, notify stakeholders, investigate evidence, restore records, and approve recovery. Vendor notification is one input; it should not be the only way the business learns that behavior changed.
The management question
Before approving an AI system, ask: Can we match the evidence to the exact model, configuration, tools, permissions, and workflow entering production—and can we detect when any of those change?
If the answer is yes, safety documentation becomes an operating asset: it defines tests, permissions, monitoring, exception handling, and rollback. If the answer is no, the company is approving a label rather than a system.
