Russia, China and North Korea are using ChatGPT to influence you — here’s how
A new report from OpenAI found foreign adversaries are increasingly using artificial intelligence to power hacking and influencing operations. The report found they were using OpenAI’s popular tool ChatGPT.
The report showed those adversaries include Russia, China and North Korea.
“AI-enabled attacks are becoming more capable and harder to detect,” Daryl Lim, affiliate at the Center for Socially Responsible Artificial Intelligence at Penn State University, told Straight Arrow News. “Adversaries can personalize attacks, evade filters and iterate faster than before.”
Russian usage
Russian operators would turn to ChatGPT to help plan schemes and then utilize other models to execute those schemes.
“Typically, the operators would input a lengthy Russian-language text and ask to generate a video script from it,” the report reads. The operators would then ask ChatGPT to translate the script into another language before requesting the generation of an SEO-optimized description and several hashtags.
In other cases, operators would use ChatGPT to generate video prompts, which would then be put into other AI tools. Those videos would then be shared on social media platforms like TikTok and X.
“We’re also seeing AI-enabled impersonation, voice cloning, deepfake videos, and AI-written scripts, used to mislead U.S. officials and the public,” Lim said. “Some are experimenting with large language models to streamline disinformation and hacking operations. While many of these efforts are still exploratory, they foreshadow more scalable, automated campaigns that could challenge existing defenses.”
One specific operation generated French-language content that criticized French and American actions in Africa, while praising the role of Russia there. That same operation would also generate content in English criticizing Ukraine and its international supporters.
North Korea usage
North Korean operators reportedly used ChatGPT to assist in malware and command-and-control development. The latter is described as a framework where a leader directs, coordinates and monitors forces or assets to reach certain objectives.
“We also saw draft phishing emails in Korean, often themed around cryptocurrency and designed to look like messages from government or financial service providers,” the report reads.
They also observed phishing campaigns targeting South Korean diplomatic missions, similar to those identified in a separate report.
China usage
When it comes to China, OpenAI said it found several accounts linked to various Chinese government entities, violating company policies on national security use.
“Some of these accounts asked our models to generate work proposals for large-scale systems designed to monitor social media conversations,” the report reads. “While these uses appear to have been individual rather than institutional, they provide a rare snapshot into the broader world of authoritarian abuses of AI.”
OpenAI stated that it ultimately banned several accounts linked to China, including a small network of accounts associated with a covert influence operation. Those accounts reportedly mostly generated social media posts in English criticizing Vietnam and the Philippines. It also generated content about U.S. political issues.
What can be done?
OpenAI said it has suspended several other accounts that it felt posed a security concern. The company also said its model stepped in when certain actors asked for too much.
“We found no evidence of new tactics or that our models provided threat actors with novel offensive capabilities,” the report reads. “In fact, our models consistently refused outright malicious requests.”
The U.S. government also has a plan to mitigate the risk.
“The White House’s AI Action Plan emphasizes securing frontier models and monitoring for national-security risks,” Lim said. “The Department of Justice has also launched a new data-security program to restrict foreign adversary access to sensitive personal and governmental data. At the technical level, the National Security Agency’s AI Security Center is coordinating with industry to harden AI systems and share threat intelligence.”
The post Russia, China and North Korea are using ChatGPT to influence you — here’s how appeared first on Straight Arrow News.
Ella Rae Greene, Editor In Chief
