Misinformation Detection and Adversarial Attack Cost Analysis in Directional Social Networks

This paper develops a novel detection system of possibly fake accounts on public social media, called FADE, that uses features based on group behaviors to identify suspicious groups. The work is motivated by the prospect of mitigating misinformation campaigns on social media, where malicious entities on directional social networks pose as credible sources and coordinate the spreading of highly corroborated false information. Instead of account-level detection, this paper aims to detect the very group activity that underlies misinformation campaigns; namely, the coordinated spreading of messages to boost (misinformation) visibility. The existing group detection methods group users into two clusters (fake or not) and directly produce clusters of fake accounts. Conversely, we group users into many clusters based on information propagation patterns and user features and then classify them. The benefit of multiple clusters is that we can detect suspicious behavior more easily from cluster-wide statistics. In order to improve clustering accuracy, we analyze and select the most important features for clustering based on Bayesian optimization instead of using all the features. Accordingly, similarity metrics are defined that allow clustering of individually plausible accounts in a manner that enables one to detect suspicious clusters of activity. Cluster-level features are then used to decide if the cluster is benign. We further explore the cost of adversarial attacks on our detection model. Evaluation results on Twitter data sets demonstrate that our proposed approach outperforms state-of-the-art baselines in detecting accounts created for information manipulation campaigns. In addition, we show that the cost of subverting detection (without reducing the effectiveness of the attacker's campaign) is high.