The Face of Deception: The Impact of AI-Generated Photos on Malicious Social Bots

  • Maxim Kolomeets
  • , Han Wu
  • , Lei Shi*
  • , Aad Van Moorsel
  • *Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

In this research, we investigate the influence of utilizing artificial intelligence (AI)-generated photographs on malicious bots that engage in disinformation, fraud, reputation manipulation, and other types of malicious activity on social networks. Our research aims to compare the performance metrics of social bots that employ AI photos with those that use other types of photographs. To accomplish this, we analyzed a dataset with 13 748 measurements of 11 423 bots from the VK social network and identified 73 cases where bots employed generative adversarial network (GAN)-photos and 84 cases where bots employed diffusion or transformers photos. We conducted a qualitative comparison of these bots using metrics such as price, survival rate, quality, speed, and human trust. Our study findings indicate that bots that use AI-photos exhibit less danger and lower levels of sophistication compared to other types: AIenhanced bots are less expensive, less popular on exchange platforms, of inferior quality, less likely to be operated by humans, and, as a consequence, faster and more susceptible to being blocked by social networks. We also did not observe any significant difference between GAN-based and diffusion/transformersbased bots, indicating that diffusion/transformers models did not contribute to increased bot sophistication compared to GAN models. Our contributions include a proposed methodology for evaluating the impact of photos on bot sophistication, along with a publicly available dataset for other researchers to study and analyze bots. Our research findings suggest a contradiction to theoretical expectations: in practice, bots using AI-generated photos pose less danger.

Original languageEnglish
Pages (from-to)1080-1091
Number of pages12
JournalIEEE Transactions on Computational Social Systems
Volume12
Issue number3
Early online date9 Oct 2024
DOIs
Publication statusPublished - 2 Jun 2025

Bibliographical note

Publisher Copyright:
© 2024 IEEE.

Keywords

  • Artificial intelligence (AI)-generated photographs
  • bot evolution
  • diffusion
  • disinformation
  • generative adversarial network (GAN)
  • social bots
  • social networks

ASJC Scopus subject areas

  • Modelling and Simulation
  • Social Sciences (miscellaneous)
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'The Face of Deception: The Impact of AI-Generated Photos on Malicious Social Bots'. Together they form a unique fingerprint.

Cite this