Search

Information sharing on social media has become a common practice for people around the world. Since it is difficult to check user-generated content on social media, huge amounts of rumors and misinformation are being spread with authentic information. On the one hand, most of the social platforms identify rumors through manual fact-checking, which is very inefficient. On the other hand, with an emerging form of misinformation that contains inconsistent image–text pairs, it would be beneficial if we could compare the meaning of multimodal content within the same post for detecting image–text inconsistency. In this paper, we propose a novel approach to misinformation detection by multimodal feature fusion with transformers and credibility assessment with self-attention-based Bi-RNN networks. Firstly, captions are derived from images using an image captioning module to obtain their semantic descriptions. These are compared with surrounding text by fine-tuning transformers for consistency check in semantics. Then, to further aggregate sentiment features into text representation, we fine-tune a separate transformer for text sentiment classification, where the output is concatenated to augment text embeddings. Finally, Multi-Cell Bi-GRUs with self-attention are used to train the credibility assessment model for misinformation detection. From the experimental results on tweets, the best performance with an accuracy of 0.904 and an F1-score of 0.921 can be obtained when applying feature fusion of augmented embeddings with sentiment classification results. This shows the potential of the innovative way of applying transformers in our proposed approach to misinformation detection. Further investigation is needed to validate the performance on various types of multimodal discrepancies.

ABSTRACT: At first, we investigated whether both β-endorphin release level in the hypothalamus and body temperature can be altered after intracerebroventricular (i.c.v.) injection of either lipopolysaccharide (LPS), interleukin-1β (IL-1β), or prostaglandin E2 (PGE2) in rats. It was found that in the rat, i.c.v. administration of either LPS (0.5 μg in 10 μl), IL-1β (10 ng in 10 μl), or PGE2 (200 ng in 10 μl), in addition to producing fever, upregulated the immunoreactivity of β-endorphin in the preoptic anterior hypothalamus of rat brain. Secondarily, we assessed whether the fever induced by either LPS, IL-1β, or PGE2 can be altered by pretreatment with buprenorphine (an opioid receptor antagonist). The results revealed that i.c.v. administration of buprenorphine (1 – 10 μg in 10 μl) alone had an insignificant effect on the body temperature. However, the fever induced by i.c.v. injection of either LPS, IL-1β, or PGE2 was significantly attenuated by pretreatment with i.c.v. injection of buprenorphine 1 h before the pyrogen injection in rats. The results suggest that pyrogens enhance β-endorphin release in the hypothalamus and trigger fever which can be attenuated by buprenorphine, an opioid receptor antagonist.

Information sharing on social media has become a common practice for people around the world. Since it is difficult to check user-generated content on social media, huge amounts of rumors and misinformation are being spread with authentic information. On the one hand, most of the social platforms identify rumors through manual fact-checking, which is very inefficient. On the other hand, with an emerging form of misinformation that contains inconsistent image–text pairs, it would be beneficial if we could compare the meaning of multimodal content within the same post for detecting image–text inconsistency. In this paper, we propose a novel approach to misinformation detection by multimodal feature fusion with transformers and credibility assessment with self-attention-based Bi-RNN networks. Firstly, captions are derived from images using an image captioning module to obtain their semantic descriptions. These are compared with surrounding text by fine-tuning transformers for consistency check in semantics. Then, to further aggregate sentiment features into text representation, we fine-tune a separate transformer for text sentiment classification, where the output is concatenated to augment text embeddings. Finally, Multi-Cell Bi-GRUs with self-attention are used to train the credibility assessment model for misinformation detection. From the experimental results on tweets, the best performance with an accuracy of 0.904 and an F1-score of 0.921 can be obtained when applying feature fusion of augmented embeddings with sentiment classification results. This shows the potential of the innovative way of applying transformers in our proposed approach to misinformation detection. Further investigation is needed to validate the performance on various types of multimodal discrepancies. [ABSTRACT FROM AUTHOR]