According to Reuters, on Saturday, December 27, China’s cyber regulator issued draft rules for public comment aimed at AI services that simulate human personalities and emotional interaction. The proposed regulations would apply to any public-facing AI in China that presents human-like traits, thinking patterns, and communication styles through text, image, audio, or video. Providers would be required to warn users against excessive use and intervene when signs of addiction appear. The rules mandate safety responsibilities across the product lifecycle, including systems for algorithm review and data security. Furthermore, services are barred from generating content that endangers national security, spreads rumors, or promotes violence and obscenity.
China’s AI Anxiety
Here’s the thing: this isn’t just about safety. It’s about control. China has been aggressively pushing its own AI development, but it wants that growth on its own terms. The state is clearly nervous about software that can form deep, persuasive emotional bonds with users. Why? Because that kind of influence is powerful, and it’s influence they don’t directly control. So they’re drawing a bright red line. You can build an AI companion, but it better not say the wrong thing or make someone too dependent. The requirement to “assess users’ emotions and their level of dependence” is a huge ask. How do you even do that at scale without being incredibly invasive? It seems like the goal is to make the technical and compliance burden so high that only the most aligned, state-friendly companies can play in this space.
The Addiction Paradox
Now, the focus on addiction is fascinating. On one hand, it’s a legitimate concern. We’ve all seen how sticky social media can be; an AI designed to be your perfect empathetic friend could be far more potent. But there’s a paradox here. The business model for many of these services relies on engagement and dependency. Telling a company it must build a captivating product but also actively sabotage its own retention metrics when it works too well is a tough sell. It basically puts AI developers in the role of therapist and censor simultaneously. I think we’ll see a lot of box-ticking compliance—a pop-up warning you might get addicted—rather than meaningful intervention. The real enforcement will come down on the content red lines, not the psychological ones.
A Broader Play for Control
Look, this draft is another piece in a larger puzzle. China has been methodically building a regulatory framework for its tech ecosystem, from data security to algorithms. This move specifically targets the “consumer-facing” edge of AI. It’s a warning shot to startups and giants alike: the wild west phase for emotionally intelligent AI is over. The beneficiaries? Probably the large, established tech firms that have the resources to navigate these complex rules and maintain strong government relations. For everyone else, the cost of compliance just got a lot steeper. This is about shaping the narrative and the technology itself before it becomes a mainstream social force. And honestly, it’s a preview of debates other countries will probably have soon enough.
