Live Coverage Of A Disinformation Operation Against The 2019 EU Parliamentary Elections

I recently worked with investigative journalists from Yle, attempting to uncover disinformation on social media around the May 2019 European elections. This work was also part of F-Secure’s participation in the SHERPA project, which involves developing an understanding of adversarial attacks against machine learning systems – in this case, recommendation systems on social networks. My contribution to the project was to analyze Twitter traffic for manipulation and poisoning attacks.

In the initial phase of the project we ran a script to capture tweets matching any of the official EU election hashtags. We collected data over the course of about 10 days. While analyzing the collected data, we stumbled upon some highly political tweets that received a great deal of suspiciously inorganic engagement – the tweets had received more retweets than likes. Here’s an example of one of them:

Replies to that tweet show that it was a lie. The incident in the video didn’t happen in Italy, but the tweet attempted to present it that way. It worked – thousands of far-right accounts retweeted it.

Here’s another example. A few days ago this tweet had received over 6000 retweets.

While writing this article, both of the tweets above were deleted by the accounts’ owners.

The suspicious tweets we identified originated from two accounts – NewsCompact and PartisanDE. These accounts link to each other in their descriptions. For instance,  PartisanDE identifies itself as the CEO of NewsCompact. The language used in tweets published by these accounts is clearly not written by native English speakers. Both accounts receive a great deal of engagement from far-right Twitter profiles (many of which I’m familiar with from past research). They share a great deal of politically inflammatory content, racist content, and mistruths. The accounts also share many URLs to sources such as VoiceOfEurope, RT, Sputnik, and some other non-authoritative “news” domains.

Both accounts follow, and are followed by many very new Twitter accounts. For example, here’s a histogram of the ages of accounts following PartisanDE. The graph below shows that a great deal of accounts created during the last year follow PartisanDE. The graph for NewsCompact is very similar.

By the looks of it, both accounts readily follow back accounts that follow them. As such, they both follow very many new accounts, and the age distributions graphs of accounts they follow look very similar to the above. There is a clear overlap between their followers and followed.

PartisanDE is a relatively new account (about a year and a half old), whereas NewsCompact is six years old, and has built up a following, apparently in spurts, during that time, as illustrated below.

We decided to dig a little deeper into what these accounts were doing, so we set up a script to follow just the two accounts. The script received tweets from those accounts, and tweets where either of the accounts were mentioned (retweeted, replied to, quoted, or @mentioned). We allowed the script to run for a few days, and then took a look at the data it had gathered.

Visualizing the retweets interactions captured by our script shows that the accounts received engagement from largely separate sources, with a few (rather high volume) accounts (in between the two in the visualization below) interacting with both.

Mapping interactions with tweets gives this visualization:

In the above, tweets are denoted by the nodes with 8-digit numerical labels. Tweet IDs are much longer, so the node labels only contain the last 8 digits of the actual tweet ID (for readability). The thick blue lines denote tweets from PartisanDE. The thick brown lines denote tweets from NewsCompact. This visualization shows that, by-and-large, each tweet received engagement from a separate group of users, with only slight overlap – i.e. each published tweet received engagement (retweets) from a separate, unique group of accounts. This is especially obvious for the pink, green, and orange portions of the above graph.

In addition to plenty of engagement from “completely legitimate” pro-leave, football fan, father of two sons, love Europe, hate the EU profiles, we found that plenty of content from these two accounts was also being amplified by accounts that looked like they shouldn’t be engaging with that content. In one example, we witnessed accounts with Korean, Japanese, Arabic, and continential European language profiles, and accounts self-identifying as US Trump supporters retweeting content about Tommy Robinson. We took a look at the details of all accounts that had engaged with NewsCompact and PartisanDE in the collected data, and found quite a variety of Twitter profiles. Here are some examples.

We found several Korean language accounts that have published a very large number of tweets, like this one:

We found several Japanese accounts, including this one that that likes to frequently retweet McDonalds Japan, and content about Freemasons:

We found accounts that tweet a lot about crypto currencies, like this one:

We found this sockpuppet account that has a Finnish-looking name, is supposedly based in Paris. The account retweets content in a huge variety of languages.

The avatar picture on the JukkaIsorinne account successfully reverse-image searched directly on Google.

We also found this account that identifies as Indian, and has a keen interest in Russian military hardware:

Hopefully the above examples paint a picture of the variety of accounts that engage with NewsCompact and PartisanDE. All of these accounts retweeted highly political content related to the EU. However, this isn’t really the interesting part of the story. By sorting the list of accounts in our dataset by creation date, we found 200 accounts that were created on the 20th and 21st May 2019 (this Monday and Tuesday, so two days ago at time of writing). I uploaded a full list of the user IDs of these accounts to my github repo. Here’s a small random selection of them:

All of these accounts retweeted the same tweet from PartisanDE. The tweet includes a video stamped with the leave.eu logo (the official campaign operation for the “leave” side of Brexit during the 2016 referendum):

As you may have noticed from the image above, at the time of writing, this tweet had received 388 retweets. Two hundred of those retweets came from the new accounts identified in this research. Most of the new accounts were similar. They mostly didn’t have followers or accounts following them, and had only published a handful of tweets. Here’s one example:

The accounts retweeted a variety of completely different content, and it is fairly obvious that they belong to a “purchase retweets” service. This indicates that PartisanDE paid for a service to retweet at least one of their politically motivated tweets at least 200 times.

Out of interest, we decided to check what else these 200 accounts had retweeted. Here are some of the other accounts they’ve retweeted in the past two days:

The list above includes verified accounts (such as justinsuntron, paulwrblanchard, and Mark_Beech), accounts selling products (inlcuding books and health supplements), people in the music industry, a fashion blogger, finance “experts”, “social influencers” (surprise, surprise), and even “CEOs”. All of the below are purchasing retweets from the same company as PartisanD. Apparently verified accounts can freely purchase retweets (and probably followers) with impunity.

We also found this interesting account that was created two days ago has already published almost 800 far-right tweets.

NewsCompact and PartisanDE were both in the top three most engaged accounts in the EU election conversation space on Twitter two weeks ago. This blog post conclusively illustrates that these two accounts are heavily fabricating engagement and, at least PartisanDE is also purchasing retweets. “Twitter Marketing” services that allow users to pay for retweets don’t seem to care about the political implications of the services they provide.

Since both accounts post content that is far-right, racist, and highly partisan, one has to wonder what effect they’ve had on the EU election conversation space on Twitter over the past few weeks. I wouldn’t be surprised if what I’ve found is just the tip of the iceberg.