An Intelligent Approach to Detect and Classify Facial Image Forgery

Abstract

In the era of digital image manipulation and the pervasive presence of social media, the verification of facial image authenticity has become an essential concern. Facial retouching in supporting documents can have adverse effects, undermining the credibility and authenticity of the information presented. In official identification documents, such as passports or driver's licenses, retouching can hinder accurate identification and security measures, potentially leading to identity fraud and security risks. This study presents a comprehensive investigation into the classification of retouched face images using a fine-tuned pre-trained VGG16 & ResNet50 model with ImageNet weight. We explore the impact of different train-test split strategies on the performance of the model and also evaluate the effectiveness of two distinct optimizers namely Adam and RMSprop. The model generalizability has been checked over two standard datasets ND-IIITD retouched faces and MDRF (Multi Demography Retouched Faces- Caucasian samples). The experiment results indicate that the ResNet50 model, fine-tuned with the RMSprop optimizer, attains a maximum accuracy of 98.52% for ND-IIITD and an impressive 99.17% for MDRF (Caucasian). In addition, an examination of various train-test split ratios over these datasets reveals the 80%-20% split ratio as the optimal choice for the approach. Moreover, the experiments show that this method effectively performs on both balanced and imbalanced datasets, emphasizing its robustness and adaptability. In conclusion, the intelligent approach leverages transfer learning and model selection offers a robust solution for the automated detection and classification of facial retouching. This contribution not only enhances image authenticity and trustworthiness in the digital age but also emphasizes the importance of considering various factors, such as model selection and dataset characteristics and hyperparameters in achieving optimal results in this field.