Generative AI in dating apps: women face increased risks

A recent survey conducted by Sumsub has highlighted the risks that generative AI, particularly deepfakes, pose to users of dating apps, with a particular focus on the increased risk to women.

In a survey involving 2,000 British dating app users, nearly half of the respondents indicated a belief that AI-generated content raises the risk of harm in the realm of dating. Of those surveyed, 46% asserted that deepfakes on these platforms increase the likelihood of experiencing violence, such as sexual abuse, while 49% voiced concerns about an increased chance of emotional and mental harm.

The survey further underscores that while deepfakes and related scams impact all users, women are especially vulnerable. Almost half of the women surveyed acknowledged the heightened risk of mental abuse and harm, and the risks of physical abuse were recognised by 48%. This compares with 45% and 43% of men, respectively, who reported similar concerns. Survey results also showed variances in perception between men and women in other areas related to the use of artificial content on these platforms. For instance, 32% of women strongly agreed that safety schemes like Ask for Angela become more critical due to deepfakes, compared to 26% of men.

Additionally, 64% of women reported experiencing deception by a deepfake or knowing someone who has, against 57% of male respondents. Concerns over the broader societal impact of these technologies were expressed by 88% of women compared to 84% of men.

When considering the use of AI images on dating applications, 60% of women stated they had never considered using such content and were opposed to the idea, outpacing the 52% of men who shared this view. Also, fewer women were likely to use dating apps where AI images are prevalent, with the figure standing at 45% compared to 40% of men.

The survey also revealed that 58% of all respondents believed that artificial content increases the risk of awkward encounters and being deceived by false identities, a phenomenon often referred to as 'catfishing'.

Natália Fritzen, AI Compliance & Policy Specialist, stated, "Deepfakes and other AI generated content in the hands of nefarious actors is a danger to everyone with an online presence - but girls and women are particularly at risk. This is true online, where women the world over have had deepfaked nudes of them artificially created and circulated. In person, new and additional risks arise when generative AI can be used on dating apps to mask identities and create misplaced trust - with the aim of causing physical, emotional, or economic harm."

"While anyone can have bad intentions towards anyone else, it's women that remain the main victims of gender-based violence. The prevalence of convincing artificial content online, and the growing ease of creating and distributing it, is a new weapon in the arsenal of those looking to manipulate, deceive and harm women," Fritzen added. "The internet desperately needs stronger safety guard rails to protect vulnerable users from AI-led harm and manipulation - and while Governments banning the creation and dissemination of deepfaked pornography is a good start, more needs to be done by regulators and Big Tech companies to stop it at the source."