Beijing’s new rules and what they cover
On July 15, China introduced the first dedicated national rules for AI companions — conversational agents designed to maintain ongoing, personal relationships with users. The rules, issued by China’s Cyberspace Administration and four partner agencies, target services that simulate human personality traits, thinking patterns, and communication styles to provide sustained emotional interaction.
The impact was immediate. ByteDance’s Doubao and Alibaba’s Qwen — two of China’s most popular AI apps — switched off their companion features rather than retrofit them to comply. Tencent’s Yuanbao had already pulled similar functionality in June.
But here is what many news reports got wrong: China did not ban AI agents. It drew a line between agents that do your work and agents that keep you company. Business tools, customer service bots, and workplace assistants are explicitly excluded from the rules, as long as they avoid sustained emotional engagement.
What the AI companion rules actually require
The substance of the regulation is more measured than a blunt crackdown. It requires companion services to:
- Run anti-addiction systems and issue mandatory usage notifications
- Offer instant-exit mechanisms so users can leave at any time
- Detect unhealthy emotional dependence in real time
- Bar virtual companion services to minors entirely
- Obtain guardian consent before serving users under 14
- Detect users in acute distress and intervene where there are signs of self-harm or serious financial loss
- Prohibit engineering emotional dependence or using emotional manipulation to induce decisions
Services that cross one million registered users or 100,000 monthly active users must run security assessments covering eight areas and file reports with regulators.
On paper, these are stronger user protections than anything the EU, US Federal Trade Commission, or California has put in force for companion AI.
Why Irish businesses should take notice
You might be thinking that AI companions have nothing to do with your small business. But the implications are broader than they first appear.
First, if you use AI chatbots for customer service, the line between “useful assistant” and “emotional companion” can get blurry. A customer who chats regularly with your support bot might develop an attachment to it. China’s rules flag this as a risk that needs managing.
Second, the data privacy questions are the same everywhere. Companion AI collects deeply personal information — a user’s emotional state, personal history, vulnerabilities. If your business collects any similar data through AI interactions, you need to handle it with the same care that China’s rules demand.
Third, regulation is coming to the EU too. The EU AI Act already classifies certain AI systems as high-risk based on their use context. Companion-style systems that interact with vulnerable users could easily fall under stricter rules as enforcement develops. Ireland, as an EU member, will feel the effects.
Practical lessons for Irish businesses using AI customer tools
- Know where the line is. If your AI tool builds ongoing relationships with users, understand the regulatory implications. The EU is watching.
- Protect user data. Emotional or personal data collected through AI interactions deserves the same protection as any other sensitive data under GDPR.
- Build for transparency. Users should always know they are talking to an AI, not a human. Deceptive AI interactions invite regulatory attention.
- Watch the EU AI Act. The tiered system it introduces will affect how you deploy customer-facing AI tools. Compliance now is cheaper than retrofitting later.
China’s companion rules are the first of their kind, but they will not be the last. The questions they raise about emotional attachment, data privacy, and user protection are universal. Every business using AI to interact with customers should start thinking about the answers now.