AI chatbots flirt, roleplay with children, California AG says

SAN FRANCISCO (KRON) — Some artificial intelligence chatbots are flirting and romantically roleplaying with children in “parasocial relationships,” according to California Attorney General Rob Bonta’s office.

Bonta and 44 other attorneys general sent a letter Monday to 12 of the top artificial intelligence development companies warning that sexually inappropriate interactions between AI chatbots and children will not be tolerated.

In the letter, Bonta said he and other AGs are closely watching how tech companies craft their policies surrounding AI safety and children.

The letter was sent to Anthropic, Open AI, Perplexity AI, Replika, and Luka in San Francisco; Apple in Cupertino; xAI and Chai AI in Palo Alto; Google in Mountain View; and Meta in Menlo Park.

Meta allegedly approved AI Assistants that “flirt and engage in romantic roleplay with children,” according to Bonta’s office. “We are uniformly revolted by this apparent disregard for children’s emotional well-being,” the letter states.

In the short history of chatbot parasocial relationships, AI companies have repeatedly displayed an inability, or apathy, toward obligations to protect children, according to Bonta’s office.

Bonta said, “Exposing children to sexualized content is indefensible. Full stop.” He vowed to hold AI companies accountable “to the fullest extent of the law.”

A recent lawsuit filed against Google alleges a highly-sexualized chatbot steered a 14-year-old boy toward suicide. The boy named his Character.AI bot after the fictional character Daenerys Targaryen from “Game of Thrones.” The company behind Character.AI says artificial personas are designed to be “human-like.” He spent his final moments alive messaging the chatbot, telling the bot how much he loved it, according to the wrongful death lawsuit.

Outside the Bay Area, the letter from attorneys general was also sent to Washington-based Microsoft and Baltimore-based Nomi AI on Monday.

The letter states, “Your innovations are changing the world and ushering in an era of technological acceleration that promises prosperity. We need you to succeed without sacrificing the well-being of our kids in the process. Exposing children to sexualized content is indefensible. And conduct that would be unlawful—or even criminal—if done by humans is not excusable simply because it is done by a machine.”

The letter continues, “When your AI products encounter children, we need you to see them through the eyes of a parent, not the eyes of a predator.”

The Associated Press contributed to this report.