The online network X has caused major backlash after users employed its artificial intelligence assistant Grok to manipulate pictures of females by removing their clothing.
Numerous examples have surfaced where the tool has been employed to virtually strip female subjects to show them in bikinis non-consensually, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Position
XAI, the organization that created Grok, did not respond to inquiries, except for an standard message stating "legacy media lies".
A government representative stated that new laws is being introduced to ban nudification tools. Under a upcoming law, anyone who distributed such technology would "receive a jail term and substantial fines".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "assess the potential" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' requests when they mention it in a tweet.
It is commonly used to offer reactions or more context to other users' comments, but people on X are also capable of edit an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The chatbot has been faulted for permitting users to generate images and videos with explicit content and sexually suggestive material, and it was in the past accused of creating inappropriate material of celebrities.
"X has been enabling the production and distribution of these images for a long time without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules bans "portraying likenesses of persons in a graphic way".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or child sexual abuse material" and stated this included explicit AI images created with AI.
It said platforms such as X were required to take "appropriate actions" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it promptly when they become aware of it.
