This is a paper list (working in progress) about Misinformation Video Detection
-
The Challenges of Studying Misinformation on Video-Sharing Platforms During Crises and Mass-Convergence Events.
Sukrit Venkatagiri, Joseph S. Schafer, Stephen Prochaska. CHI 2023 Workshop on Building Credibility, Trust, and Safety on VideoSharing Platforms, April 23–28, 2023, Hamburg, Germany.[pdf]
-
Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities.
Sara Abdali. arXiv preprint arXiv:2203.13883 (2022). [pdf]
-
A Survey on Multimodal Disinformation Detection.
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimitar Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, Preslav Nakov. arXiv preprint arXiv:2103.12541 (2021) [pdf]
-
Exploring the role of visual content in fake news detection.
Juan Cao, Peng Qi, Qiang Sheng, Tianyun Yang, Junbo Guo, Jintao Li. Disinformation, Misinformation, and Fake News in Social Media: Emerging Research Challenges and Opportunities (2020): 141-161. [pdf]
-
A Survey on Video-Based Fake News Detection Techniques.
Ronak Agrawal, Dilip Kumar Sharma. 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom). IEEE, 2021. [pdf]
-
A comprehensive survey on passive techniques for digital video forgery detection.
Nitin Arvind Shelke, Singara Singh Kasana. Multimed Tools Appl 80, 6247–6310 (2021) [pdf]
-
Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection.
Junke Wang, Zhenxin Li, Chao Zhang, Jingjing Chen, Zuxuan Wu, Larry S. Davis, Yu-Gang Jiang. ArXiv, 2022. [pdf]
-
A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities.
Xinyi Zhou, Reza Zafarani. ACM Computing Surveys (CSUR) 53.5 (2020): 1-40. [pdf]
-
Fake news detection on social media: A data mining perspective.
Kai Shu,Amy Sliva,Suhang Wang,Jiliang Tang,Huan Liu. ACM SIGKDD Explorations Newsletter,Volume 19,Issue 1,June 2017, pp 22–36. [pdf]
This section contains the pilot works that analyze misinformation video on online platform, including the impact of video modality and the propagation feature on recommendation-dominated plaforms.
-
[PACMHCI-2020] Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube.
Eslam Hussein, Prerna Juneja, Tanushree Mitra. [pdf]
- This audit experiments investigate whether personalization (based on age, gender, geolocation, or watch history) contributes to amplifying misinformation.
-
[PNAS-2021] The (minimal) persuasive advantage of political video over text.
Chloe Wittenberg, Ben M. Tappin, Adam J. Berinsky, David G. Rand. [pdf]
- This paper tests the assumption that video is more compelling than text.
-
[JCMC-2021] Seeing is believing: Is video modality more powerful in spreading fake news via online messaging apps?
S. Shyam Sundar, Maria D. Molina , Eugene Cho. [pdf]
- This works finds that video is processed more superficially, and therefore users believe in it more readily and share it with others.
-
[TORS-2022] Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles.
Ivan Srba, Robert Moro, Matus Tomlein, Branislav Pecher, Jakub Simko, Elena Stefancova, Michal Kompan, Andrea Hrckova, Juraj Podrouzek, Adrian Gavornik, Maria Bielikova. [pdf]
- This paper presents results of an auditing study performed over YouTube aimed at investigating how fast a user can get into a misinformation filter bubble, but also what it takes to “burst the bubble”.
-
[IJCAI-2022] Black-box Audit of YouTube's Video Recommendation: Investigation of Misinformation Filter Bubble Dynamics (Extended Abstract).
Matus Tomlein, Branislav Pecher, Jakub Simko, Ivan Srba, Robert Moro, Elena Stefancova, Michal Kompan, Andrea Hrckova, Juraj Podrouzek, Maria Bielikova. [pdf]
- This paper describes a black-box sockpuppeting audit which was carried out to investigate the creation and bursting dynamics of misinformation flter bubbles on YouTube.
This section contains the pilot works that detect misinformation video at signal level, i.e., detect with the clues of editing traces or generating traces.
-
[MS-2023] Fake COVID-19 videos detector based on frames and audio watermarking.
Nesrine Tarhouni ,Salma Masmoudi, Maha Charfeddine, Chokri Ben Amar. [pdf]
- This paper presents a fake video detector based on combining audio and frames watermarking which makes possible the detection of modifications in the two video channels and assures a fast detection of fake video.
-
[arXiv-2022] VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces.
Tai D. Nguyen, Shengbang Fang, Matthew C. Stamm. [pdf]
- This paper designs VideoFACT to exploit forensic traces’ contextual dependencies upon scene content as well as spatial dependencies and proposes datasets that are composed not only of standard video manipulations but also advanced AI-based content manipilation techniques.
-
[MMM-2019] Detecting tampered videos with multimedia forensics and deep learning.
Markos Zampoglou, Foteini Markatopoulou, Gregoire Mercier, Despoina Touska, Evlampios Apostolidis, Symeon Papadopoulos, Roger Cozien, Ioannis Patras, Vasileios Mezaris, Ioannis Kompatsiaris. [pdf]
-
[MTA-2017] Malicious inter-frame video tampering detection in MPEG videos using time and spatial domain analysis of quantization effects.
Javad Abbasi Aghamaleki, Alireza Behrad. [pdf]
-
[JFS-2018] Authentication of surveillance videos: detecting frame duplication based on residual frame.
Sondos M. Fadl, Qi Han, Qiong Li. [pdf]
-
[CSPA-2017] A comprehensive approach for exposing inter-frame video forgeries.
K Sitara, B M Mehtre. [pdf]
-
[SCAIE-2018] Detection clone an object movement using an optical flow approach.
Omar Ismael Al-Sanjary, Ahmed Abdullah Ahmed, Adam Amril Bin Jaharadak, Musab AM Ali, and Hewa Majeed Zanga. [pdf]
-
[ICIEAM-2016] Implementation of image/video copy-move forgery detection using brute-force matching.
Neema Antony, Binet Rose Devassy. [pdf]
-
[NCA-2020] Video tampering localisation using features learned from authentic content.
Pamela Johnston, Eyad Elyan, Chrisina Jayne. [pdf]
-
[AECE-2019] Spatial video forgery detection and localization using texture analysis of consecutive frames.
Mubbashar Saddique, Khurshid Asghar, Usama Ijaz Bajwa, Muhammad Hussain,Zulfiqar Habib. [pdf]
-
[TIFS-2016] ESPRIT-Hilbert-based audio tampering detection with SVM classifier for forensic analysis via electrical network frequency .
Reis, Paulo Max Gil Innocencio , da Costa, Joao Paulo Carvalho Lustosa , Miranda, Ricardo Kehrle , Del Galdo, Giovanni. [pdf]
- In this paper, a new technique to detect adulterations in audio recordings is proposed by exploiting abnormal variations in the electrical network frequency (ENF) signal eventually embedded in a questioned audio recording.
-
[CSUR-2021] The creation and detection of deepfakes: A survey.
Yisroel Mirsky, Wenke Lee. [pdf]
-
[FCST-2023] Overview of Facial Deepfake Video Detection Methods.
Zhang lu, Lu Tianliang, Du Yanhui. [pdf]
-
[DDAM-2022] Lessons Learned from ASVSpoof and Remaining Challenges.
Junichi Yamagishi. [pdf]
-
[ICASSP-2022] Fake audio detection based on unsupervised pretraining models.
Zhiqiang Lv, Shanshan Zhang,Kai Tang, Pengfei Hu. [pdf]
- This work presents the authors' systems for the ADD2022 challenge, which is the first audio deep synthesis detection challenge. They explored using unsupervised pretraining models to build fake audio detection systems.
-
[ICCV-2021] Joint audio-visual deepfake detection.
Yipin Zhou, Ser-Nam Lim. [pdf]
- This work proposes a novel visual / auditory deepfake joint detection task and show that exploiting the intrinsic synchronization between the visual and auditory modalities could benefit deepfake detection.
-
[MM-2020] Emotions don't lie: An audio-visual deepfake detection method using affective cues.
Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, Dinesh Manocha. [pdf]
-
[TCSVT-2021] Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network.
Juan Hu ,XinLiao ,WeiWang, ZhengQin. [pdf]
- The work propose a two-stream method by analyzing the frame-level and temporality-level of compressed Deepfake videos. The frame-level stream can prune the redundant connections to prevent the invalid connections from affecting the final prediction. The temporality-level stream is utilized to capture temporal features to detect the temporal consistency.
-
[arXiv-2023] Anti-Compression Contrastive Facial Forgery Detection.
Jiajun Huang, Xinqi Zhu, Chengbin Du, Siqi Ma, Surya Nepal, Chang Xu. [pdf]
- The authors propose a novel anti-compression forgery detection framework by maintaining closer relations within data under different compression levels.
This section contains the pilot works that utilize multimodal features to detect misinformation video at Semantic/Intent Level.
-
[MFSec-2017] Web Video Verification using Contextual Cues.
Olga Papadopoulou, Markos Zampoglou, Symeon Papadopoulos, Yiannis Kompatsiaris. [pdf] [repo]
- Propose an annotated dataset of real and fake videos(FVC)
- Use video comment credibility and video metadata features for fake video detection.
-
[ICMI-2019] Towards Automatic Detection of Misinformation in Online Medical Videos.
Rui Hou, Verónica Pérez-Rosas, Stacy Loeb, Rada Mihalcea. [pdf]
- Explore the use of linguistic, acoustic, and user engagement features to identify misinformation.
-
[ECIR-2019] Misleading Metadata Detection on YouTube.
Priyank Palod, Ayush Patwari, Sudhanshu Bahety, Saurabh Bagchi, Pawan Goyal. [pdf] [repo]
- The work presents VAVD, a new dataset for research on fake videos.
- Propose UCNet , a deep learning based approach using comments and simple features extracted from title and social context to identify fake videos.
-
[ACL Workshop-2020] NLP-based Feature Extraction for the Detection of COVID-19 Misinformation Videos on YouTube.
Juan Carlos Medina Serrano, Orestis Papakyriakopoulos, Simon Hegelich. [pdf] [repo]
- Create a multi-label classifier based on transfer-learning that can categorize conspiratorial content; use the percentage of conspiracy comments and the first hundred comments as tf-idf features in the classifier.
-
[arXiv-2021] Misinformation Detection on YouTube Using Video Captions.
Raj Jagtap, Abhinav Kumar, Rahul Goel, Shakshi Sharma, Rajesh Sharma, Clint P. George. [pdf] [repo]
- This work exploited the YouTube captions to understand the content of the videos using multiple pre-trained word embeddings.
-
[Scientific Reports-2022] A CNN-based Misleading Video Detection Model.
Xiaojun Li, Xvhao Xiao, Jia Li, Changhua Hu, JunpingYao , Shaochen Li. [pdf]
- In this paper, three categories of features (content features, uploader features and environment features) are proposed to construct a convolutional neural network (CNN) for misleading video detection.
-
[CIKM-2021] Using Topic Modeling and Adversarial Neural Networks for Fake News Video Detection.
Hyewon Choi, Youngjoong Ko. [pdf]
- The work proposes a topic agnostic fake news video detection model based on adversarial learning and topic modeling. The stance difference estimation (using Gibbs sampling-based LDA) between title/description and comments on topic modeling was used to dynamically adjust the encoding ratio of the comments.
-
[PRL-2022] Effective fake news video detection using domain knowledge and multimodal data fusion on youtube.
Hyewon Choi, Youngjoong Ko. [pdf]
- The framework is Similar to 6-[CIKM-2021], but remove the adversarial learning and topic modeling.This paper use domain knowledge to perform learning by reflecting the potential meaning of comments and use the linear combination to adjust the encoding rate for each characteristic of the video.
-
[BigData-2021] A Multimodal Misinformation Detector for COVID-19 Short Videos on TikTok.
Lanyu Shang, Ziyi Kou, Yang Zhang, Dong Wang. [pdf]
- The work develops TikTec, a multimodal misinformation detection framework that explicitly exploits the captions extracted from the audio track and the visual frame to accurately capture the key information from the distractive video content, and effectively learns the composed misinformation that is jointly conveyed by the visual and audio content.
-
[MMM-2022] Multi-modal semantic inconsistency detection in social media news posts.
Scott McCrae, Kehan Wang, Avideh Zakhor. [pdf]
- The work develops a multi-modal fusion framework to identify mismatches between videos and captions in social media posts by leveraging an ensemble method based on textual analysis of the caption, automatic audio transcription, semantic video analysis, object detection, named entity consistency, and facial verification.
-
[arXiv-2022] Misinformation Detection in Social Media Video Posts.
Kehan Wang, David Chan, Seth Z. Zhao, John Canny, Avideh Zakhor. [pdf]
- This work proposes two new methods for detecting semantic inconsistencies within short-form social media video posts, based on contrastive learning and masked language modeling.
-
[MMSP Workshop-2022] Multimodal Semantic Mismatch Detection in Social Media Posts.
Kehan Wang, Seth Z. Zhao, David Chan, Avideh Zakhor, John Canny. [pdf]
- This work uses language, video and audio models to extract dense features from each modality, and explore transformer architecture together with contrastive learning methods.
-
[AAAI-2023] FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms
Peng Qi, Yuyan Bu, Juan Cao, Wei Ji, Ruihao Shui,Junbin Xiao, Danding Wang, Tat-Seng Chua. [pdf] [repo]
- This paper proposes the largest Chinese short video dataset about fake news named FakeSV and provides a new multimodal baseline method SV-FEND.
-
[EACL-2023] COVID-VTS: Fact Extraction and Verification on Short Video Platforms.
Fuxiao Liu, Yaser Yacoob, Abhinav Shrivastava. [pdf] [repo]
- This work introduces a new benchmark, COVIDs-VTS, for fact-checking multi-modal information, as well as proposes TwtrDetective, an effective model incorporating cross-media consistency checking to detect token-level malicious tampering in different modalities, and generate explanations.
-
[ChineseCSCW-2022] Video Rumor Classification Based on Multi-modal Theme and Keyframe Fusion.
Jinpeng You, Yanghao Lin, Dazhen Lin, and Donglin Cao. [pdf]
- This work propose a multi-modal fusion model based on theme and keyframe. It can effectively capture the theme and key-frame information of the three modalities in the video, and carry out rumor detection from them.
-
[ACL-2023] Improving Fake News Video Detection by Correlating with Neighbors.
Peng Qi, Yuyang Zhao, Yufeng Shen,Wei Ji, Juan Cao, and Tat-Seng Chua. [pdf] [repo]
- This work propose the NEED framework, which exploits the neighborhood relationship explicitly and implicitly to enhance the fake news video detection. The authors also formulate a new multimodal inference task and propose a novel model that utilizes the consistency to leverage factual information to rectify false negative predictions.
-
[arXiv-2023] Multimodal Short Video Rumor Detection System Based on Contrastive Learning.
Yuxing Yang, Junhao Zhao, Siyi Wang, Xiangyu Min, Pengchao Wang, and Haizhou Wang. [pdf]
- This paper proposes a framework for detecting and classifying short video rumors based on multimodal feature fusion: the authors use the TSN (Temporal Segment Networks) to extract video visual features; then OCR and ASR technologies are exploited to extract text features; BERT model is utilized to fuse text and video visual features. Contrastive learning is applied for the final prediction.
-
[arXiv-2023] Identifying Misinformation on YouTube through Transcript Contextual Analysis with Transformer Models.
Christos Christodoulou, Nikos Salamanos, Pantelitsa Leonidou, Michail Papadakis, Michael Sirivianos. [pdf] [repo]
- This paper converts the conventional vide classification task into a text classification task by leveraging the textual content derived from the video transcripts and presents a methodology for detecting misinformation on YouTube using different fine-tuned transformer models and few-shot learning.
-
[EMNLP-2023] Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines.
Yoo Yeon Sung, Jordan Boyd-Graber, Naeemul Hassan. [pdf] [repo]
- This paper presents VMH, a dataset of misleading headlines from social media videos and analyze multimodal baselines for detecting misleading headlines.
-
[MTA-2023] Emotion aided multi-task framework for video embedded misinformation detection.
Rina Kumari, Vipin Gupta, Nischal Ashok, Tirthankar Ghosal, Asif Ekbal. [pdf]
- This paper develops a video-based multimodal fake news detection dataset named FakeClips and introduces a deep multitask framework dedicated to video-embedded multimodal fake news detection in which fake news detection is the main task and emotion recognition is the auxiliary task.
-
[ArXiv-2024] FakeClaim: A Multiple Platform-driven Dataset for Identification of Fake News on 2023 Israel-Hamas War.
Gautam Kishore Shahi, Amit Kumar Jaiswal, Thomas Mandl. [pdf] [repo]
- This paper constructs a dataset of factual claims from different platforms and fake YouTube videos on the 2023 Israel-Hamas war for automatic fake YouTube video classification. A fine-tuned pretrain model is leveraged to classify fake videos within the subset of YouTube videos with textual information and user comments.
-
[TCSS-2024] Cross-Modal Attention Network for Detecting Multimodal Misinformation From Multiple Platforms.
Zhiwei Guo, Yang Li, Zhenguo Yang, Xiaoping Li, Lap-Kei Lee, Qing Li, Wenyin Liu [pdf]
- This paper constructs a multiplatform multimodal misinformation(3M) dataset by collecting data from TikTok(Chinese version) and Weibo and designs a crossmodal attention misinformation detection(CAMD) network. Extensive experiments on the 3M dataset validate the effectiveness of the proposed CAMD model.
-
[COLING-2024] Interpretable Short Video Rumor Detection based on Modality Tampering.
Kaixuan Wu, Yanghao Lin, Donglin Cao, Dazhen Lin [pdf]
- This paper considers how to detect rumors from the perspective of modality tampering. The authors propose a short video rumor detection framework by designing two pretraining tasks: modality tampering detection and inter-modal matching and design an interpretability mechanism to make the rumor detection results more reasonable by backtracking the model’s decision-making process.
-
[SSRN-2022]A Novel Method for Detecting Misinformation in Videos, Utilizing Reverse Image Search, Semantic Analysis, and Sentiment Comparison of Metadata.
Dhanvi Ganti. [pdf]
- This paper proposes a three-step method to detect video-based misinformation. First, this method detects deepfakes first and then applies semantic analysis to detect shifts in the meaning and intent of the associated metadata of both videos. The last step entails a sentiment comparison to detect shifts in emotion.
This section contains the pilot works that discuss areas related to misinformation video detection.
-
[Soft Computing-2022] Intelligent techniques for deception detection: a survey and critical study.
Haya Alaskar, Zohra Sbaï, Wasiq Khan, Abir Hussain, Arwa Alrawais. [pdf]
-
[CICLing-2018] A deep learning approach for multimodal deception detection.
Gangeshwar Krishnamurthy, Navonil Majumder, Soujanya Poria, Erik Cambria. [pdf]
-
[AAAI-2018] Deception Detection in Videos.
Zhe Wu, Bharat Singh, Larry Davis, V. Subrahmanian. [PDF]
-
[Applied Intelligence-2021] A unified approach for detection of Clickbait videos on YouTube using cognitive evidences.
Deepika Varshney, Dinesh Kumar Vishwakarma. [pdf]
-
[ICPR-2022] VidHarm: A Clip Based Dataset for Harmful Content Detection.
Johan Edstedt, Amanda Berg, Michael Felsberg, Johan Karlsson, Francisca Benavente, Anette Novak, Gustav Grund Pihlgren. [pdf]
This section contains the pilot works that related to critical open issues and future directions for misinformation video detection.
-
[ICWSM-2022] Cross-Platform Multimodal Misinformation: Taxonomy, Characteristics and Detection for Textual Posts and Videos.
Nicholas Micallef, Marcelo Sandoval-Castañeda, Adi Cohen, Mustaque Ahamad, Srijan Kumar, Nasir Memon. [pdf]
-
[SIGIR-2022] Generalizing to the future: Mitigating entity bias in fake news detection.
Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, Fuzhen Zhuang. [pdf]
-
[AAAI-2021] Embracing domain differences in fake news: Cross-domain fake news detection using multi-modal data.
Amila Silva, Ling Luo, Shanika Karunasekera, Christopher Leckie. [PDF]
-
[CIKM-2021] MDFEND: Multidomain fake news detection.
Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang,Jintao Li. [pdf]
-
[TKDE-2022] Memory-guided multi-view multi-domain fake news detection.
Yongchun Zhu, Qiang Sheng, Juan Cao, Qiong Nan, Kai Shu, Minghui Wu, Jindong Wang, Fuzhen Zhuang. [pdf]
-
[IP&M-2022] Characterizing multi-domain false news and underlying user effects on Chinese Weibo.
Qiang Sheng, Juan Cao, H. Russell Bernard, Kai Shu, Jintao Li, Huan Liu. [pdf]
-
[NAACL-2018] Fever: a large-scale dataset for fact extraction and verification .
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal [pdf]
-
[CVPR-2022] Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources.
Sahar Abdelnabi, Rakibul Hasan, Mario Fritz. [pdf]
-
[CVPR-2020] Multi-modal graph neural network for joint reasoning on vision and scene text.
Difei Gao, Ke Li, Ruiping Wang, Shiguang Shan, Xilin Chen. [pdf]
-
[NIPS-2022] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou [pdf]
-
[WWW-2022]Veracity-aware and Event-driven Personalized News Recommendation for Fake News Mitigation.
Shoujin Wang, Xiaofei Xu, Xiuzhen Zhang, Yan Wang, Wenzhuo Song. [pdf]
-
[AAAI-2020] Weak Supervision for Fake News Detection via Reinforcement Learning.
Yaqing Wang, Weifeng Yang, Fenglong Ma, Jin Xu, Bin Zhong, Qiang Deng, Jing Gao. [pdf]
DeepFake Detector : WeVerify Project ; Sensity
Reverse Image Search : Google ; Baidu ; Bing ; Yandex ; Tineye ; ImageRaider ; Duplichecker
Video Verification Plugin : InVID Verification Plugin
This curated list of works on misinformation video detection is based on our survey. If you find it helpful, please cite as follows:
@inproceedings{mvdsurvey,
title={Combating Online Misinformation Videos: Characterization, Detection, and Future Directions},
author={Bu, Yuyan and Sheng, Qiang and Cao, Juan and Qi, Peng and Wang, Danding and Li, Jintao},
booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
year={2023},
doi={10.1145/3581783.3612426},
publisher = {Association for Computing Machinery},
}