Texas AG settles with healthcare AI vendor over hallucination-rate claims

Texas's Attorney General reached a settlement with a healthcare AI vendor whose advertised accuracy figures the state's own investigation found likely overstated — one of the earliest state actions against an AI company's own marketing claims, using an existing consumer-protection statute rather than a new AI-specific law.

What happened

Attorney General Ken Paxton's office announced on September 18, 2024 that it had resolved an investigation into Pieces Technologies, a Dallas-based healthcare AI company whose generative-AI product summarizes patient condition and treatment data for hospital staff at Texas hospitals. The office's investigation centered on Pieces' own marketing of an error rate, or "severe hallucination rate," of "<1 per 100,000" — a figure the office said it found was likely inaccurate.

Legal basis and terms

The office pursued the matter under the Texas Deceptive Trade Practices-Consumer Protection Act, a general consumer-protection statute rather than an AI-specific one. The agreement is not a financial penalty: Pieces agreed to accurately disclose the extent of its products' accuracy and to ensure hospital staff using the product understand how much to rely on it. Attorney General Paxton said in the release: "AI companies offering products used in high-risk settings owe it to the public and to their clients to be transparent about their risks, limitations, and appropriate use."

Why it matters to security leaders

This settlement predates most AI-specific state statutes and shows a state consumer-protection law reaching an AI vendor's own accuracy marketing, not a data-security or privacy claim — a distinct exposure for any security or compliance leader evaluating an AI vendor's published performance claims before deployment. It is also one of the earliest examples of a pattern that continued through 2026: state attorneys general applying existing deceptive-trade-practices authority to AI products rather than waiting on AI-specific legislation.

See the regulation tracker for how this state consumer-protection action compares to federal and international AI rules, and the FTC AI-related enforcement tracker for the equivalent federal-level record.