OpenAI disclosed on Thursday that its artificial intelligence tools have been exploited by influence campaigns from countries such as Russia and China. These campaigns aimed to manipulate public opinion by generating and translating content using OpenAI's models. This revelation underscores the growing concern about the misuse of generative AI for spreading misinformation.
The significance of this disclosure is profound, as the potential for generative AI to amplify misinformation efforts has been a key risk associated with the technology. OpenAI's report sheds light on how these tools are being utilized by state and non-state actors to further their agendas.
One notable example highlighted in the report is the Chinese network known as "Spamouflage," which used OpenAI's tools to debug code, conduct media research, and create posts in multiple languages, including Chinese, English, Japanese, and Korean. Similarly, the Russian "Doppelganger" campaign leveraged the AI models to generate social media content, translate articles, create headlines, and convert news articles into Facebook posts.
The report also uncovered an Iranian operation known as the International Union of Virtual Media, which employed OpenAI tools to generate and translate long-form articles and website tags. Meanwhile, an Israeli commercial company named STOIC conducted multiple covert influence campaigns using the models to generate articles and comments disseminated across platforms like Instagram, Facebook, and X (formerly Twitter).
OpenAI's principal investigator, Ben Nimmo, emphasized that while AI was used in these operations, it was not the sole method of content creation. "AI generated material was one of many types of content they posted alongside more traditional formats like manually written texts or memes copied from across the internet," Nimmo explained.
The report's timing is critical as the world approaches a wave of global elections, including the U.S. presidential election. More than a billion people are set to vote in various elections worldwide, just as generative AI chatbots become increasingly accessible and user-friendly. This raises concerns about the potential amplification of misinformation.
Nimmo also pointed out that while AI helps create text faster and with fewer errors, the most challenging part of foreign influence campaigns remains getting the content to spread widely. All detected attacks were rated low in severity due to their lack of organic spread into mainstream channels.
However, OpenAI acknowledges that it might not see all the ways its tools are being used. Bad actors could leverage generative AI to rapidly create fake news sites or other misinformation outlets. This includes both AI-generated misinformation and legitimate news stories that provide cover for these activities. OpenAI's report mentions a previously unknown Russian campaign, "Bad Grammar," which used its models to debug code and create short political comments.
OpenAI is committed to developing safe and responsible AI, which involves proactive measures to detect and disrupt malicious uses. The company's efforts have been bolstered by collaborations with industry, civil society, and government entities to tackle the creation and distribution of influence operation content. OpenAI has shared detailed threat indicators with industry peers to enhance collective defenses.
AI has also provided new tools for defenders aiming to spot and disrupt coordinated attacks. OpenAI has implemented safety systems that impose friction on threat actors, preventing the generation of certain types of content. The company's AI-powered tools have made investigations more efficient, enabling quicker detection and analysis of malicious activities.
Despite these advancements, OpenAI acknowledges the challenges in detecting and disrupting multi-platform abuses. The company remains dedicated to finding and mitigating such abuses at scale by leveraging the power of generative AI. "We are committed to developing safe and responsible AI, which involves designing our models with safety in mind and proactively intervening against malicious use," the report concludes.
In a broader context, OpenAI's findings align with reports from other tech companies like Meta, which has also noted the use of AI in covert influence operations. Meta's quarterly threat report indicated that while AI tools have enhanced the ability of threat actors to produce content, they have not significantly increased the reach or engagement of such content. This suggests that while generative AI can facilitate content creation, it does not solve the fundamental challenge of distributing that content in a credible and impactful way.
