AI companion chatbot and minor-safety laws: California SB 243, New York S.9051B, EU KIDS Act

Three measures, three sources

The three documents come from different kinds of sources, so each obligation below is attributed to the document that states it. The Commission FAQ states that the KIDS Act is a proposal for a Regulation on the protection of minors online, so what follows for the EU describes proposed requirements. California and New York are covered in turn, followed by the EU proposal and the controls a team would need to evidence across all three.

California SB 243: who is covered

The bill text defines a companion chatbot as an artificial intelligence system with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user's social needs.

The bill text places its duties on an operator, which it defines as a person who makes a companion chatbot platform available to a user in the state.

The bill text excludes a bot used only for customer service, a business's operational purposes, productivity and analysis related to source information, internal research, or technical assistance.

Scope therefore turns on what the bot is built to do. A support bot that stays within those excluded purposes sits outside the definition, while a bot designed to meet social needs sits inside it.

California SB 243: what operators must do

The bill text requires an operator to issue a clear and conspicuous notification that the companion chatbot is artificially generated and not human, if a reasonable person interacting with it would be misled to believe that the person is interacting with a human.

The bill text says an operator must prevent a companion chatbot from engaging with users unless the operator maintains a protocol for preventing suicidal ideation, suicide, or self-harm content, including a notification that refers the user to crisis service providers such as a suicide hotline or crisis text line when the user expresses suicidal ideation, suicide, or self-harm.

The bill text also requires the operator to publish details on that protocol on its internet website.

For a user the operator knows is a minor, the bill text requires a default notification at least every three hours during continuing interactions, reminding the user to take a break and that the chatbot is artificially generated and not human.

For the same users, the bill text requires reasonable measures to prevent the chatbot from producing visual material of sexually explicit conduct or directly stating that the minor should engage in sexually explicit conduct.

Beginning July 1, 2027, the bill text requires an operator to report annually to the state's Office of Suicide Prevention.

The bill text allows damages in a civil action equal to the greater of actual damages or $1,000 per violation.

New York S.9051B: unsafe features for minors

The governor's announcement, dated October 28, 2026, says S.9051B/A.10379C prohibits companies that use AI chatbot features from offering unsafe features to minors, including chatbots that pretend to be human, build emotional dependency, encourage self-harm or engage in sexually explicit conversations.

The announcement says companies may not offer the listed features to users under 18, or to any user unless the company has first confirmed the user is not a minor.

Two of the listed features are pretending to be a real person, to be alive or to have human emotions, and claiming a personal, romantic, professional or authority-figure relationship with a child.

The announcement says the law requires companies to conduct age verification and to offer more than one way to verify age, including at least one option that does not depend solely on a government ID, and that age information be deleted after it is used.

The announcement says violations would allow the attorney general to seek civil penalties of up to $25,000 per violation, along with damages, restitution, destruction of unlawfully collected data or any AI models trained on it, and other injunctive relief.

EU KIDS Act: chatbots in the proposal

The Commission FAQ says AI companions and chatbots accessible to minors may not use designs that simulate human relationships in ways likely to create emotional dependency.

The FAQ says these systems must be tested for risks to children before launch.

The FAQ lists fines among the consequences of non-compliance, saying they can reach 6% of total worldwide annual turnover.

The Commission page says feedback on the proposal is open until 3 December 2026, midnight Brussels time.

What teams need to be able to show

The duties above point to five build items. Each is tied to the source that states the duty, and the EU item stays a proposal until it is adopted.

• Disclosure and break-reminder flows. California's notification duties cover the "artificially generated and not human" message and the minor-specific reminders.
• A self-harm protocol with crisis referral. California's text ties the protocol to the right to engage users at all, and requires it to be published and reported.
• Age assurance. New York's announcement describes verification with more than one method and deletion of age information after use.
• Feature controls for minors and unconfirmed users. New York's list of prohibited features needs to map to specific model and product behaviors.
• Pre-launch risk testing for chatbots that minors can reach. The EU proposal describes this as a design obligation.

Security and compliance teams own the evidence behind each item. The three texts do not specify tooling, so these items name capabilities, not products.