California officials have opened an investigation into xAI’s AI chatbot Grok. They’re reacting to increasing outcry over its capacity to create pedophilia-promoting images. Grok is an AI tool developed by the tech firm founded by Elon Musk. It allows users to generate original images from scratch, simply by entering a text prompt. The platform has faced backlash for how it can be misused to create non-consensual, pornographic content.
The examination follows nearly immediately on the heels of xAI’s decision to curtail Grok’s image-generating capabilities. Today, those features are only available to paying subscribers. This change hasn’t done much to calm concerns among state leaders and legislators, who say the system is fraught with dangers.
Senator Ron Wyden of Oregon has voiced concerns regarding the application of Section 230 of the Communications Decency Act of 1996. He went on to clarify the role this legal provision plays in granting immunity to online platforms from accountability for what their users create. It doesn’t include the content that the platforms themselves produce.
“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” – Attorney General Rob Bonta
California Governor Gavin Newsom responded to xAI’s actions by describing them as “vile.” He argued that the platform has pervasively morphed into a “breeding ground for predators.” His comments still reflect a mounting trepidation among state employees. At the same time, they are very concerned at the implications of AI technologies such as Grok.
Elon Musk has defended Grok, claiming he is “not aware of any naked underage images generated by Grok. Literally zero.” He cautioned that Grok does not “magically create art” and only produces output when a user requests it. Experts such as Professor James Grimmelmann, from Cornell University, are not convinced by xAI’s legal arguments. He recognized that the Section 230 protections do not extend to when the platform’s own actions create the content.
“Only protects sites from liability for third-party content from users, not content the sites themselves produce,” – Prof James Grimmelmann
Concerns are increasing. In a letter to major app stores—including Apple and Google—three Democratic senators recently demanded that X and Grok be banned from their platforms entirely. The senators’ actions are the latest example of a developing, broad, bipartisan congressional effort to rein in rapidly advancing AI technologies that could produce dangerous results.
In the UK, Prime Minister Sir Keir Starmer has addressed the situation, warning that if Musk’s social media platform X cannot effectively control Grok, regulatory actions may follow. Starmer emphasized the dangers of unfettered AI tools, adding, “If X can’t rein in Grok, we will.”
Grok is criticized in ways that most other AI tools are. Firms such as OpenAI and Google have found themselves on the backfoot in cases of digitally undressing people. As public outcry and fear over the implications of non-consensual image generation grow, regulators are beginning to act. UK comms watchdog Ofcom, which recently opened its own investigation into Grok, is standing shoulder to shoulder with California officials in their attempts to address these urgent needs.