Video Deepfake Detection using Hybrid CNN-Transformer Models

Abstract

Deepfakes on a mass scale have been enabled by advancements in deep generative models. Conventional deepfake methods based on manually designed features or independent CNN models are found to have limitations in generalising across different types of modification methods and levels of compression. In this paper, a combined CNN-Transformer approach for exact deepfakes video detection has been used. In this combined approach, the Transformer component has been used for representing relations among different lifelike videos. On the other hand, for spatial mapping of registered artefacts in deepfake images, a CNN component has been used. This combined approach has been found to perform better at detecting deepfakes than other models when tested on the FaceForensics++, DFDC, and Celeb-DF image datasets. Bias in data consideration in testing and research ethics has been considered in this paper for future research on exact multimedia forensics.