Malaysia and Indonesia Block X’s AI Chatbot Over Deepfake Abuse
Malaysia and Indonesia have restricted access to an artificial intelligence chatbot linked to X, following growing concerns that the tool is being misused to generate sexually explicit fake images of real individuals.
The decision marks the first known full national blocks of the AI chatbot, reflecting escalating regulatory action against AI systems that authorities say are advancing faster than safety and content-moderation frameworks.
Authorities Cite Risks to Women and Children
Regulators in both Malaysia and Indonesia said the AI tool enables users to create and manipulate images, but is increasingly being used to transform real photos into sexualised or revealing content without consent.
Officials warned that the technology poses particular risks to women and children, highlighting how easily it can be abused to create pornographic deepfakes that spread rapidly online.
Malaysia Says X Failed to Address Core Safety Issues
Malaysia’s communications regulator stated that it had previously warned X about the misuse of its AI technology. However, authorities said the company’s response relied too heavily on user reporting mechanisms, rather than addressing the underlying ability of the tool to generate harmful content.
Regulators noted that once explicit fake images are created, reporting systems often fail to prevent further sharing, allowing the material to continue circulating and amplifying harm.
Indonesia Frames Ban Around Human Rights and Dignity
Indonesia’s digital affairs ministry framed its decision around human dignity and human rights, warning that AI-generated non-consensual sexual content undermines public trust and endangers vulnerable groups.
The government has formally requested X to provide details on how the chatbot is controlled, moderated, and restricted, signalling that future access could depend on stronger safeguards.
Consistent With Indonesia’s Hardline Content Policy
The ban also aligns with Indonesia’s long-standing approach to online content regulation. The country has previously blocked platforms such as Pornhub and OnlyFans, viewing AI-generated sexual images as a continuation of the same problem, now delivered through more advanced technology.
Scrutiny Expands Beyond Southeast Asia
The controversy surrounding the chatbot is spreading beyond Southeast Asia. In the United Kingdom, regulators are assessing whether X is complying with online safety obligations, while political leaders have publicly criticised the use of technology to generate explicit fake images.
The chatbot is associated with Elon Musk, whose companies are increasingly facing regulatory pressure over content moderation, AI governance, and platform accountability.
Users Say Reporting Often Worsens the Harm
For affected users, the consequences are deeply personal. Individuals whose images were altered using the AI tool say that reporting the content often did little to stop its spread. In some cases, encouraging others to report the posts led to greater visibility, exposing more people to the fake images and worsening the emotional damage.
A Turning Point for AI Regulation
The backlash is forcing X to reassess how its AI systems operate, as governments begin to treat AI tools as regulated digital products rather than experimental features.
Regulators say the issue extends far beyond a single chatbot, exposing gaps in online safety laws and highlighting the urgent need for clearer rules governing AI-generated content, consent, and accountability.
As more governments weigh similar actions, the bans by Malaysia and Indonesia may signal a broader global shift toward stricter oversight of generative AI technologies.
