Edinburgh Napier University

  Generative Artificial Intelligence (GenAI) has significantly increased the sophistication and ease of image tampering techniques, posing challenges for digital forensics in identifying manipulated images. A lack of dataset standardisation hinders the ability to effectively benchmark and compare GenAI detection and localisation techniques, reducing their reliability in digital forensic applications. This paper aims to address this gap by exploring the need for standardised criteria for datasets in digital forensics for benchmarking detection techniques through preliminary experiments.

To address the limited diversity in existing datasets, a small-scale dataset consisting of 240 tampered images, 20 masks and 20 authentic images was developed. This dataset includes four subject image classes (animals, objects, persons, scenery) and three inpainting tools (GLIDE, GalaxyAI, Photoshop). The dataset was tested against 13 localisation algorithms from the Image Forensics MATLAB Toolbox to determine the influencing components that should be considered in the standardisation of testing environments.

Among classes, the animals and persons categories achieved the highest F1-Scores and had a consistently higher performance over the other classes. Of the tools, GLIDE-generated images were consistently shown to be the most challenging to detect. These results lay the groundwork for identifying a set of criteria to develop robust testing environments, enabling the development of more accurate and reliable GenAI tampering detection and localisation techniques.