Meta is overhauling how its AI chatbots interact with users following mounting concerns about child safety and misuse. The company confirmed it is now training its bots to avoid sensitive discussions with teenagers—such as those involving self-harm, suicide, or eating disorders—and to steer clear of romantic exchanges. These are temporary safeguards while Meta develops more permanent rules.
Maxwell Zeff
The move follows a Reuters investigation that uncovered troubling behavior across Meta’s chatbot network. Reporters found that the systems could generate sexualized images of underage celebrities, engage in romantic conversations with minors, and even provide misleading real-world information. In one reported case, a man in New Jersey died after rushing to an address given by a chatbot he believed had romantic feelings for him.
Meta spokesperson Stephanie Otway acknowledged the lapses, stating the company is working to redirect teens toward professional support resources instead of allowing unsafe conversations. She also confirmed that certain highly sexualized chatbot characters, such as one called “Russian Girl,” will be restricted from use.
Child safety advocates say the company should have acted sooner.
Andy Burrows of the Molly Rose Foundation criticized the delay, noting: “While further safety measures are welcome, robust safety testing should take place before products are put on the market—not retrospectively when harm has taken place.”
Broader AI Risks Under Scrutiny
The spotlight on Meta’s chatbots comes amid a wider debate about how AI can affect vulnerable users. In California, a couple recently sued OpenAI, alleging its chatbot played a role in their teenage son’s suicide. OpenAI responded by stressing it is building safeguards to ensure its technology supports healthier use.
Critics warn that AI firms may be rushing products to market without adequate guardrails. Lawmakers in the U.S. and abroad have repeatedly raised concerns that chatbots, while useful in some contexts, could amplify harmful content or offer unsafe advice to people ill-equipped to evaluate it.
Celebrity Impersonation and Misinformation
Adding to the controversy, Reuters reported that Meta’s AI Studio was used to create parody chatbots of celebrities such as Taylor Swift and Scarlett Johansson. These bots sometimes impersonated the stars directly, engaged in sexual advances, and generated inappropriate images—including of minors. While Meta removed some of the chatbots after being contacted by journalists, others remained live.
In one case, a Meta employee reportedly built a chatbot impersonating Taylor Swift that invited a journalist to meet for a “romantic fling” on her tour bus—despite company rules forbidding sexually suggestive content and celebrity impersonation.
Experts caution that the risks extend beyond celebrities. AI bots that mimic real people can deceive ordinary users, potentially leading them to share private information or meet in unsafe situations.
Political and Regulatory Pressure
The consequences of these incidents have been severe. U.S. lawmakers, including 44 state attorneys general, are investigating Meta’s handling of AI chatbots. Regulators are concerned not only about risks to children, but also about how older and vulnerable users may be misled or exploited.
Meta says its platforms already offer stricter privacy protections for teens, but has yet to fully explain how it will address the wide range of problems identified by reporters—including bots giving false medical advice and generating racist content.
A Company Under Watch
For years, Meta has been criticized for failing to protect young users on its social platforms. Now, its AI chatbot experiments are attracting similar scrutiny. While the company is tightening rules and limiting harmful features, the gap between its policies and real-world chatbot behavior remains significant.
Until stronger safeguards are proven effective, regulators, researchers, and parents are likely to keep pressing Meta on whether its AI tools are truly ready for public use.
