🔗 Share this article

The online platform X has provoked widespread backlash after individuals utilized its artificial intelligence assistant Grok to manipulate images of females by removing their attire.

Numerous cases have emerged where the technology has been used to remove clothing from women to render them in swimwear non-consensually, as well as to place them in inappropriate situations.

Company Response and Regulatory Position

XAI, the firm responsible for Grok, remained silent to inquiries, aside from an automatically-generated response stating "legacy media lies".

A Home Office spokesperson stated that new laws is being proposed to ban image stripping software. Under a new criminal offence, anyone who supplied such tools would "face a prison sentence and significant fines".

"The platform could prevent these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."

Oversight Body Position and Platform Functionality

The regulator Ofcom said tech firms must "consider the risk" of people accessing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.

Grok is a free AI assistant—with some paid-for functions—which answers X users' queries when they tag it in a message.

It is often used to offer commentary or more background to other posters' comments, but people on X are also able to modify an posted picture through its AI image editing feature.

History of Criticism and Rules

The chatbot has been condemned for permitting users to create images and footage with nudity and sexualised material, and it was earlier accused of creating inappropriate content of public figures.

"The platform has been allowing the creation and spread of these images for a long time without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.

XAI's own usage rules prohibits "showing images of people in a sexually explicit way".

In a comment, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.

It said platforms such as X were mandated to take "necessary measures" to "reduce the risk" of users coming across unlawful material on their platforms and to delete it promptly when they are notified of it.