How replies to Trump’s tweets and some machine learning can help us combat social media spam

New Twitter research analyzing replies to U.S. candidates for president demonstrates the promise of a new methodology for categorizing posts on the micro-blogging platform.

In A New, Novel Method For Clustering Tweets Andy Patel,  a researcher at F-Secure’s Artificial Intelligence Center of Excellence and Alexandros Kornilakis, a PhD student at the University of Crete’s FORTH-ICS institute, document the experiments that have led to what they believe is the most sophisticated method of clustering tweets available.

“The problem with trying to address misuse of a platform like Twitter is the enormous amount of text to process. Thousands of new posts are published every second,” Andy said.

The torrential downpour of content flooding social media gives bad actors cover to spread disinformation, hoaxes, lies, scams, and fake news. The inability of sites to clamp down on artificial, malicious, or exploitative behavior creates a marketplace for likes, views, subscribes, reviews, and even fake accounts, all available for purchase with a quick web search.

As the importance of social media as an information source continues to grow, researchers are looking for ways to address this problem. Many are now developing methods for identifying and categorizing different types of behavior on social media to help them pinpoint potential instances of abuse.

What is clustering?

Clustering is the process of using a machine learning mechanism to group phrases or passages—or, in this case, tweets—into “buckets” based on their topic or subject matter.

The clustering mechanism Andy and Alexandros developed produced unexpected and surprising results, and demonstrated versatility beyond any methodology they’d previously tried.

Trump targeted by toxic tweets

The researchers tested their method by looking at over 1 million replies to tweets posted by leading Democratic candidates for president along with the account of congresswoman Alexandra Ocasio-Cortez, who has an extremely large and active following, and President Donald Trump, the incumbent in the 2020 U.S. presidential election.

“Influential politicians’ mentions are like an extreme version of what an average Twitter user deals with. They inevitably receive lots of engagement and it’s often very positive or very negative,” Andy said. “We classified the tweets by identifiers like subject-verb-object triplets and overall sentiment.”

As explained in the report’s appendix, the methodology allowed them to generate an average sentiment score. Posts were classified as positive (those that express agreement with the author of the original post), negative (defined as those that express disagreement with the author), or toxic (toxic posts are more strongly worded than negative posts and tend to express hostility or downright hatred) based on different identifiers.

Using clustering they found the most common negative themes of the replies to the Democrats included:

• you are an idiot/moron/liar/traitor (or similar)
• you will never be president
• Trump will win the next election

Positive themes included:

• We love you
• You got this
• You have my vote

“The results are only as good as the sentiment analysis library we used. However, they remain consistent across both datasets studied,” Andy said. “Percentage-wise, @AndrewYang received by far the most positive replies, and @AOC and @SenWarren received the largest ratio of toxic replies.”

But none of the Democrats’ accounts attracted as much toxicity as the current president of the United States. His results were:

The most common negative themes in Trump’s replies included:

• You are an idiot/liar/disgrace/criminal/#impotus
• You are not our president
• You have no idea / you know nothing
• You should just shut up
• You can’t stop lying
• References to Vladimir Putin

The positive themes were:

• God bless you, Mr. President
• We love you
• You are the best president

A smarter way to take on spam

“One surprising outcome of our experiments was evidence that our method is readily able to identify and group sentences with similar subject matter, but different phraseology,” Andy said. “For instance, phrases containing variations on ‘the pot calls the kettle black’, such as ‘name is pot and he say’s you’re black’, ‘kettle meet black’, and ‘pot and kettle situation’ all ended up in the same cluster. We found several similar instances of this type of behavior during experimentation.”

The method also showed a knack for detecting what most people would call spam.

“Experiments also demonstrated that the methodology we developed is highly effective at identifying close-to-identical tweets, commonly associated with reply-spam, account automation, and coordinated disinformation campaigns.”

What can we learn from these clusters?

The research was conducted as part of EU Horizon 2020 projects PROTASIS and SHERPA, and F-Secure’s Project Blackfin. F-Secure’s interest in experimenting with cutting-edge natural language processing (NLP) techniques extends to several domains including spam detection, online content classification, and automated analysis of the complex command lines used in adversarial attacks.

But Andy hopes this method can also help reduce the misuse of social media, such as tweets from coordinated disinformation efforts, spammers that seek to exploit the replies of accounts with a lot of followers, or bots and other types of automation.

“For instance, our methodology automatically identified and grouped tweets pushing a hoax that the Australian bush fires had been caused by arsonists.”

In an emergency situation, being able to identify and respond to misinformation could directly aid rescue and response efforts—or, at least, prevent disinformation from harming those efforts.

“Additional research is necessary but with some more development there could be a range of potential applications. This methodology could be used for automated filtering or removal of spam, disinformation and other toxic content. This could be done by assigning quality scores to accounts based on how often they post toxic content or harass users.”

Andy suggests that it’s possible that this mechanism could even help transform the way people tweet.

“With the ability to identify and classify tweets readily, Twitter replies could begin to be cleaned up the way that email inboxes have been with effective spam filtering.”

Based on their research, the Alexandros and Andy created an interactive website for individuals to explore. They also published their research, including their code, on Github.