Deepfake technology, emerging as the latest trend throughout our K-12 schools and campuses, has educators and parents worried. Tanya Horeck, a professor of film and feminist media studies at Anglia Ruskin University, is on the fact-finding mission. We hope that she is able to shine a light on the full magnitude of this disturbing trend. Her investigations reveal a disturbing trend: deepfakes and nudifying technologies are increasingly being utilized among schoolchildren, often with devastating consequences.
The consequences of that neglect were made glaringly apparent in several recent incidents. When male students at her school created a pornographic deepfake video of a 15-year-old girl, she became the victim. An example of a disturbing trend While this incident may seem extreme, it’s not an outlier. About 20 of the girls—mostly in their mid-teens—have had a similar deepfake ordeal. Most alarmingly, the youngest victim was only 11 years old. The problem cuts much deeper than the UK. In the Westfield High School deepfake case described above, more than 30 adolescent girls were identified by student victims’ parents to school authorities in the United States, implicating perps still prowling social media platforms like Snapchat.
A new poll of over 4,300 secondary school teachers in England revealed that… It found that one-in-ten teachers were aware of students at their school creating “deepfake, sexually explicit videos” in the last school year. This statistic highlights the need to deal with this problem in educational settings, specifically.
They’re so shocking, deepfake images that anomaly. Her shock was not at the deepfake images themselves, but that of all people, Ms. Not only does she convey a profound emotional effect on victims. As she explains, “Of all the different kind of abuse I get, these are the ones that affect me the most – they stay with me.” As a creator, it is difficult to articulate why, other than to say that it feels like you. It’s almost like somebody has hijacked you. You have no agency and no control over how you can improve or remediate—you can’t enact any change.
And if Horeck’s conversations with headteachers are anything to go by, most of the cases we know about have seen boys make deepfakes of girls. Boys have been the targets, too in some of the cases. One example brought to public attention by the charity Everyone’s Invited (EI) was of a boy who was targeted with an AI-generated sexualised image created by another boy. In Spain, 15 boys were given probation after being found guilty of making counterfeit nude pictures of their female classmates.
Significant legal repercussions are surfacing as well. Criminal inquiries by police have dominated headlines the last few weeks at two elite, private schools in the UK. They are looking into the production and spread of deepfake pornographic images. Indeed, data shows that 75 percent of deepfake cases are children, 14 years or younger. Alarmingly, one in ten of these cases is 11-year-olds.
Dame Rachel de Souza, the children’s commissioner for England, has demanded a ban on nudification apps, such as ClothOff. Perhaps most saliently, she underscores the urgent need for protective measures in the face of these technologies. Horeck highlights a case where a student collected images of classmates to create deepfakes, adding to the complexity of addressing this issue.
In Australia, about 50 high school students at Bacchus Marsh Grammar in Victoria reported that their images had been faked and distributed. One parent told us that her child was so traumatized by the graphic sexual images that she not only vomited—but bled! While these harrowing situations directly concern young students’ physical safety, they shed light on the emotional and psychological impact of being victimized.
As educator Seth James points out, whenever new technology has come to schools, it has posed new challenges. “For everyone working in schools, it feels like new sets of challenges and risks are constantly being thrown up by technological developments,” he states. He compares the deluge of AI-powered technologies to “the next train coming down the track.”
The Department for Education has heard about these challenges and their impact and is now working to ameliorate them. A spokesperson stated, “Our new relationships, sex and health education guidance will make sure that all young people understand healthy relationships, sexual ethics and the dangers of online content such as pornography and deepfake.” Beyond this campaign, the department is funding outreach materials to assist teachers explain the laws and harms in response to online content.
Horeck explains there is a lack of uniformity across the country in schools’ response to these incidents. Some teachers advocate for immediate police involvement and expulsion for offenders, while others push for restorative justice approaches that emphasize understanding and dialogue with students. She says it’s troubling that there isn’t more clear guidance for educators who are navigating these situations in confusing contexts.
“So there seems to be some kind of inconsistency and uncertainty on how to deal with these cases – but I think it’s really hard for teachers because they’re not getting clear guidance.” – Tanya Horeck
In a 2019 collaboration with educational researcher Jessica Ringrose, Ringrose elucidated youth’s accessibility to these technologies. The stakes These things are not hypothetical – people are creating and distributing images without consent. These technologies are at people’s fingertips. And I mean, it’s not like it’s difficult or anything for any kid to get their hands on these things,” she adds.
Amidst these challenges, educators see a growing demand for thoughtful teaching focused on digital ethics and citizenship. Dolly Padalia, a teacher at Center for Creative Leadership, urges schools to treat these challenges as essential parts of their curriculum. “You actually have to teach people to use it properly,” she asserts.
As discussions continue within educational institutions about how best to approach deepfake incidents, headteachers express concerns about normalization among students. One headteacher stated, “It worries me that it’s so normalized. He obviously wasn’t hiding it. He didn’t feel this was something he shouldn’t be doing.”