OpenAI Disrupts Russian and Iranian Influence Operations Using AI-Generated Personas
OpenAI has shut down sophisticated influence operations from Russia and Iran that leveraged AI tools like ChatGPT to create fake journalists and think tanks, successfully planting narratives in mainstream media.
OpenAI announced on Thursday the disruption of two significant influence operations originating from Russia and Iran. These operations utilized advanced AI tools, including ChatGPT, to generate convincing fake journalist personas and establish covert think tanks. The goal was to seed propaganda and specific narratives into legitimate news outlets, a tactic that proved more impactful than typical influence campaigns.
The Russian operation, dubbed "Dark Clark" by OpenAI, focused on manipulating political discourse and public opinion within Latin America. The actors created a fake persona, Mia Clark, who was presented as the leader of the "Social Research Center," a fabricated think tank. This entity served as a front to disseminate propaganda, primarily aimed at damaging Ukraine's reputation in the region. The campaign also delved into sensitive political issues within Argentina and Bolivia, employing fake audio clips and misleading news stories to sow discord and influence public perception. OpenAI's analysis suggests the group was actively managing the "Social Research Center," using AI to discuss operational details like employee wages, indicating a high level of sophistication and control.
OpenAI researchers noted that "Dark Clark" was the most complex attempt to run a front identity they had encountered in two and a half years. The operation's propaganda efforts were significant enough to prompt fact-checks and official denials from major media organizations and government officials in Latin America. Examples include a TikTok video falsely claiming the Ecuadorian government was recruiting citizens to fight for Ukraine and a fake audio clip targeting the Bolivian government during protests. While not attributed to a specific Russian agency, the operators showed a notable interest in "Politology," a successor to the Wagner Group, suggesting potential links to Russian mercenary and intelligence circles.
Simultaneously, a separate Iranian campaign employed similar AI tools to push articles related to U.S.-Iran relations. This operation involved creating at least seven distinct fake journalist personas who managed to place dozens of articles across approximately a dozen online outlets globally. These outlets were generally smaller to medium-sized, with a focus on international affairs and the Middle East. While the articles themselves did not generate significant engagement, their publication in legitimate outlets lent them an air of credibility.
OpenAI rated the Russian campaign at a 5 and the Iranian campaign at a 4 on their impact scale, with 6 being the highest. This indicates a significantly higher level of success compared to most influence campaigns, which typically score 1 or 2. The effectiveness of these AI-driven operations lies in their ability to generate plausible content at scale, making them difficult to detect and counter with traditional methods.
The findings highlight a growing trend of state-sponsored actors weaponizing generative AI for influence operations. The ability to create realistic personas, generate vast amounts of text, and mimic journalistic styles allows these actors to bypass traditional defenses and reach wider audiences. This poses a considerable challenge for social media platforms, news organizations, and cybersecurity firms tasked with identifying and mitigating such threats.
OpenAI's proactive disruption of these campaigns underscores the evolving threat landscape and the critical role of AI detection and analysis in cybersecurity. The company's ongoing efforts aim to stay ahead of malicious actors who are increasingly adopting AI technologies to enhance their influence and disinformation campaigns.