Figure 3 (b) Distribution of eligible studies by publication year
Related Figures (42)
Summary of existing available surveys and review papers on deepfake detection and generation using AI. Table 1 Summary of research questions and key motivations includes in this study. Table 2 Fig. 1. Text analysis visualization of search string generates using the VOYANT tool. Fig. 2. Keyword co-occurrences visualization analysis of eligible studies using the VOSviewer tool. Fig. 3. Number of publications on deepfake detection and generation, by publication year and research area. 3.2.1. Deepfake generation methods reenactment, and attribute manipulation are not new problems: videos and images have been forged since their creation. In face-swapping, an individualā€™s face in an input video is swapped with the face that appears in the intended video (Ding et al., 2020), as shown in Fig. 7(a). Gener- ally, a face-swapping method comprises three stages (Ding et al., 2022; Coccomini et al., 2022; Sun et al., 2023). Firstly, these methods detect a face in the input image and then choose a target image from a library that is comparable to the input image. Deepfake generation refers to a digital media manipulation method that overcomes the substantial limitations of traditional forgery gener- ation frameworks that are designed to generate forgeries or fraudulent content by minimizing traces of manipulation or fingerprints. It has been widely used for detecting forgeries, such as inconsistent biometrics or compacted artifacts. Deepfake generation uses deep neural networks (DNNs) to extract input characteristics and produce synthetic fake yet hyper-realistic content (Kolagati et al., 2022). Detecting deepfakes is more challenging than detecting traditional manipulation of digital media due to the minimal divergence between the real and counterfeit data boundaries (Devasthale & Sural, 2022; Kawa & Syga, 2021). In the next stage, the mouth, nose, ears, and eyes are swapped and fine-tuned to match the appearance of the source image. Lastly, com- bined candidate substitution occurs that involves calculating the dis- tance based on the overlapping area. Fig. 4. Flow diagram of the PRISMA protocol for reporting systematic reviews (Page et al., 2021); the figure summarizes the data selection and screening process. Fig. 5. Taxonomy of deepfake detection and generation approaches. Fig. 6. Taxonomy of deepfake generation approaches using GAN and its variants. (c) Expression Manipulation with smiling face and opening eye using SC-GAN Overview of face reenactment and faceswap generation methods. Table 3 (continued) Fig. 8. Example of different databases (with real and synthetic images) taken from Casia Dataset (2020). Fig. 9. Generic framework using GAN based on Ding et al. (2022). Fig. 10. Example of synthetic faces and attributes manipulation created using GAN variants (Tran et al., 2019; Hou et al., 2023). Overview of facial and video synthetic generation methods. Table 4 (continued) parameter. The classifier F is trained on input images and labels bo. To identify facial characteristics bo, the optimization function Ry is defined efficiently to preserve the visual quality of the image. Salama and Hel-Or (2020) developed a method to identify source generators that created fake images using multi-class classification. The study designed a multi- class model to create an image generator profile to evaluate generated images, as shown in Fig. 13. The study performs well in generating synthetic image profiles but fails to preserve distinguishing facial fea- tures. Ding et al. (2020) presented a method that automatically elimi- nates facial masks and synthesizes affected areas while preserving facial Fig. 12. Deepfake generation process using autoencoder and GAN (Lee & Kim, 2021). Fig. 11. Generic framework to create synthetic video using StyleGAN2 (Singh et al., 2020) Fig. 13. Framework for image re-generation and classification (Liu et al., 2021), Fig. 15. Real and synthesized faces taken from hybrid fake faces datasets (Guo et al., 2021). Fig. 16. Framework for deepfake image/video detection (Ismail, Elpeltagy, Zaki, et al., 2021). Overview of faceswap and face reenactment deepfake detection methods. Table 5 Table 5 (continued ) Table 5 (continued) Fig. 17. Generic pre-processing pipeline for audio-visual deepfake detection (Ilyas et al., 2023). Fig. 18. Framework flow of real and deepfake image detection (He et al., 2021). Overview of synthetic face and audio-visual detection methods. Table 6 Table 6 (continued ) seeking to optimize their approaches in this rapidly advancing field. methods were trained on the CelebA and RaFD datasets (75 % of the dataset for training and 25 % for validation), which utilize a more streamlined set of frames than standard benchmark methods. Likewise, JinTian (2022), Deng et al. (2020), and Iperov, (2022) presented Dis- coFaceGAN and DeepFaceLive-based frameworks to generate high- quality face swap videos. Further, they analyze the latent space trans- formation using 3D adversarial learning. The frameworks demonstrated superior performance to benchmark methods, with higher computa- tional overhead. Fig. 19. Deepfake video training, generation and detection Flow (Mitra et al., 2020). Fig. 20. Example of classification workflow (Mitra et al., 2020). Overview of deepfake generation and detection tools and software. Table 7 (continued) Fig. 21. Generic workflow of deepfakes generation using different auto-encoders (Zendran & Rusiecki, 2021). Using an autoencoder is a key method for generating deepfakes. An autoencoder is a kind of Artificial Neural Networks (ANN) that learns to re-generate certain inputs using a permitted method. To train the encoder M: Z"-Z! and decoder N : Z'Z" and to meet the condition described in eq (15) (Zendran & Rusiecki, 2021) 3.4.1.1. Reenactment and faceswap. Even though current reenactments, attribute manipulation and faceswap-based techniques perform well in generating forged faces and feature swapping. For example, Mitra et al. (2022), Liu et al. (2022), and Yang et al. (2021) developed methods of Trends in deepfake generation, and recommendations. Recommendations and trends relating to deepfake detection. Fig. 22. Policy measures to mitigate Al-based deepfake impacts. Data availability human-machine partnerships in deepfake detection. Active methods can play a pivotal role in disrupting the potential endless cycle of AI- based deepfake creation and detection. These methods take a proac- tive approach by implementing measures to prevent the creation of deceptive videos rather than solely relying on detection after the fact. Techniques like digital watermarking can embed unique identifiers within authentic videos, making it harder to manipulate them. Future research should focus on developing and enhancing active methods to effectively counteract the advancements in deepfake technology, thus mitigating the perpetual arms race between deepfake creation and detection algorithms. By addressing these research gaps and pursuing these avenues of investigation, we can collectively work towards more resilient and practical frameworks for detecting and mitigating the spread of deepfake content, essential in safeguarding individuals, com- munities, and societies from the potential harms of manipulated media.
