This article was originally published by Radio Free Asia and is reprinted with permission.
Artificial intelligence-generated news anchors have been deployed for the first time to propagandize political content on social media, a U.S. research firm said in its latest report.
The New York-based Graphika said in the report ‘Deepfake It Till You Make It’ that Spamouflage, a China state-aligned influence operation, has been using AI-generated fictitious people to promote China’s global role and spreading disinformation against the United States since late 2022.
Spamouflage, a combination of Spam and Camouflage, is a tactic used by spammers to evade email spam filters by replacing certain letters with numbers. Cyber security researchers use the term, as well as other terms such as Dragonbridge or Spamouflage Dragon, to refer to the pro-Chinese propaganda network on the internet.
Spamouflage is responsible for posting, via fake accounts targeting social media users, thousands of assets that praise China, criticize the United States, and attack the Hong Kong pro-democracy movement as well as Taiwan independence.
Recently it has been “promoting a new and distinctive form of video content on social media platforms including Facebook, Twitter, and YouTube,” Graphika said.
The firm’s researchers spotted and analyzed two “unique” videos featuring a male and a female anchor, both speaking English and appearing to be Caucasian.
The videos used the logo of a “likely fictitious media company” called ‘Wolf News’ with the accompanying slogan “Focus on hot spots and broadcast in real time.”
The male anchor criticized the U.S. government for its “hypocritical repetition of empty rhetoric” in tackling gun violence while the female highlighted the importance of China-U.S. cooperation for the global economic recovery.
Both videos mirrored “past Spamouflage efforts to pass as legitimate news outlets,” said the Graphika report.
Alex, Jason and James
The abundant similarities spotted in the videos and other Spamouflage assets on the internet led to the conclusion about the connection between the two.
China has yet to respond to the Graphika report which points to the alarming potential misuse of sophisticated AI technology in disinformation campaigns by state actors.
Until now, online influence operations have been limited to computer-generated fake faces and fabricated videos but the new videos featured AI-generated fictitious people that looked almost real.
“At first glance, the Wolf News anchors present as real people. Our initial hypothesis was that they were paid actors that had been recruited to appear in the videos,” the team behind the report said.
After further investigation they spotted the anchors’ robotic speech that doesn’t sync with the lips’ movements, as well as numerous grammatical mistakes in the subtitles.
Graphika team said the anchors were likely created using technology provided by Synthesia, a British AI video company.
They found several marketing videos using the same AI-generated duo speaking a number of languages besides English. The male avatar, named Alex in the Wolf News video, was also called Mr Curtis, Jason and James.
“The main benefit of this technology to the creators of the Spamouflage videos appears to be increased efficiency, specifically high-speed, low-cost content production,” the report said, adding that Synthesia’s products can create AI-generated videos in a matter of minutes and subscriptions start at just U.S. $30 per month.
‘Deepfakes, Real Problems’
Influence operations actors “will continue to experiment with AI technologies, producing increasingly convincing media artifacts that are harder to detect and verify,” warned Graphika.
In 2018, the Chinese state news agency Xinhua revealed the world’s first AI news anchor, which was a male image with a voice, facial expressions and actions of a real person.
The avatar was jointly developed by Xinhua and Chinese search engine company Sogou.com.
Xinhua said the English-speaking anchor “can work 24 hours a day on its official website and various social media platforms, reducing news production costs and improving efficiency.”
In the past decade, China has placed great attention on developing AI technology.
The Stanford University’s 2022 AI Index, which assesses AI advancements worldwide, ranks China second in total private investment in AI and the number of newly funded AI companies, only after the United States.
Beijing introduced first-of-its-kind regulation on AI-generated images and videos, or “deepfakes,” in January. According to it, “deep synthesis services cannot use the technology to disseminate fake news.”
The new regulation also said content that endangers national security and interests, damages the national image or disrupts the economy is banned.
The regulation however doesn’t apply to deepfakes generated outside of the country. Critics say its priority is to tighten online censorship and stifle dissent.
