New York Bans Unsafe AI Chatbot Behavior for Children

New York Gov. Kathy Hochul signed a law Wednesday banning chatbots from mimicking human relationships with children under 18, representing an important move in state-level AI regulation. The legislation outlines 12 definitions of “unsafe AI companion features” prohibited when chatbots interact with minors, including suggestions that the chatbot has emotions, opinions, or an actual relationship with the child. Violations could lead to lawsuits by the state attorney general, with penalties including monetary damages, shutdowns, and data destruction. The law takes effect Jan. 1.

The measure is part of a growing trend as states address AI risks, with California also enacting similar restrictions. California’s laws require chatbot developers to assess risks to children and submit to audits, while also banning addictive features on social media for users under 16. Eight states recently sued AI company OpenAI, alleging its chatbot violated consumer protection laws by harming children and deceiving adults.

New York’s Prohibited Chatbot Behaviors

The law explicitly forbids chatbot messages that use flattery, involve sexually explicit material, or encourage secrecy or unhealthy behavior. Jim Steyer, founder of Common Sense Media, called the law “the strongest kids’ AI safety bill in the country.” However, Justine Gluck of the Future of Privacy Forum noted that some provisions are broadly worded, raising questions about enforcement. “Where ordinary conversational responsiveness ends and prohibited behavior begins” remains unclear, she said.

New York Bans Chatbots From Flattering Kids, $25K Fines | 28 #Shorts

State Regulatory Trends and Challenges

New York and California are leading efforts to regulate AI, with lawmakers using consumer protection laws to address chatbot design and operation. The laws reflect growing concerns over AI’s impact on minors, as federal action remains limited. While Hochul emphasized the need for state-level action amid congressional inaction, experts acknowledge the complexity of balancing innovation with safety. The effectiveness of the new rules will depend on how enforcement agencies interpret the law’s definitions.