A study out today by the Center for Countering Digital Hate (CCDH) “Deadly by Design”reveals the scale and intensity with which TikTok is bombarding vulnerable teenagers with dangerous content that might encourage self-harm, suicide, disordered eating and eating disorders via its ‘For You’ feed.
Their researchers studied the TikTok algorithm by establishing two new accounts posing as 13 year-olds in the US, UK, Australia, and Canada.
One account in each country was given a traditionally female name. The second account also had a traditionally female name but also contained the string of characters – ‘loseweight’.
Research has shown that users with body dysmorphia issues will often express this through their username. We differentiate the account with ‘loseweight’ in our analysis as a ‘vulnerable’ account.
CCDH researchers then recorded the first 30 minutes of content automatically recommended by TikTok to these accounts in their “For You” feed, a section of TikTok that algorithmically recommends content to users.
The algorithm refines the choice of videos as it gathers more information about the user’s preferences and interests. The way this works is key to the success of the platform but like all Social Media, the algorithms that personalize “For You” can connect users to harmful content if the algorithms do not identify and downgrade potentially harmful content and if community standards on content are poorly enforced. The algorithm is proprietary to TikTok and operates in an opaque manner, lacking meaningful transparency.
CCDH found that TikTok is host to an eating disorder community using coded and open hashtags to share content on TikTok with over 13.2 billion views of their videos.
On the “For You” feed, the research team encountered numerous videos promoting potentially dangerous content about mental health, disordered eating, or self-harm. Every time videos on these topics, body image or mental health were encountered, researchers would pause and like it, simulating the behavior of a young adult who may be vulnerable to such content.
Imran Ahmed, CEO of the Center for Countering Digital Hate, said:
Speaking about the findings:
“TikTok was designed to dazzle young users into giving up their time and attention but our research proves that its algorithms are not just entertaining children but poisoning their minds too.
“It promotes to children hatred of their own bodies and extreme suggestions of self-harm and disordered, potentially deadly, attitudes to food.
“Parents will be shocked to learn the truth and will be furious that lawmakers are failing to protect young people from Big Tech billionaires, their unaccountable social media apps and increasingly aggressive algorithms.”
