Intelligent Explainable Multimodal Framework for Fake News Detection Using BERT, CNN, and Graph Neural Networks

Online social networks (OSNs) have grown very fast and has helped to revolutionize digital communications, where people can exchange real-time information among the members in their own social networks across the globe. However, rapid communication has also paved the way for rumours and fake news, which pose a threat to not only the democratic system but of course to public health, the monetary flow and social cohesion as well. Older techniques like human labelling, as well as rule-based systems and simple machine learning algorithms cannot withstand the multimodal, dynamic and copiously available nature of fake news in these days and times. In this paper, to address these limitations, we propose Intelligent Explainable Multimodal Fake News Detection Framework (IEMFNDF) that uses text data, image data, propagation data and user behavioral data in which helps of the advanced deep learning architectures is used for this problem. We propose this framework consisting of six well-distinguished layers, namely, Data Acquisition layer, Multimodal Preprocessing layer, Deep Feature Extraction layer, Cross-Modal Attention Fusion layer, Explainable Learning & Ensemble Classification layer. Semantic textual analysis is performed using transformer-based BERT models and visual discriminative representation of images and video frames is obtained using two models namely VGG16 and Convolutional Neural Networks (CNNs). Using Graph Neural Networks (GNNs), abnormal propagation behaviors in rumours can be identified by decomposing their diffusion information patterns as well as the social interaction structure. Multimodal fusion mechanism adaptively combines different modalities of representations extracted. Moreover, an explainable AI module generates attention maps and offers visualizations of feature importance, thus improving transparency and interpretability. The proposed framework has been then tested to experiment on the benchmark FakeNewsNet, Twitter15 and Twitter16 datasets and it obtained a significant improvement with respect to unimodal and conventional multimodal frameworks in terms of accuracy, precision, recall, F1-score and early detection of rumors. The proposed system shows its higher resilient to malicious misinformation attacks and variety of platform contents. The results show the effectiveness of the integration of a BERT, VGG16, CNN and GNN in a single explainable multi modal framework, which can be extended and adapted for scalable and real-time misinformation detection in OSNs.