AI-generated fake news gets Chinese man arrested

A Chinese man has been arrested by police in north-western Gansu province for allegedly spreading AI-generated fake news about a train crash through the use of AI technology. This marks China’s first reported arrest for ChatGPT misuse.

ChatGPT misuse results in detention of Chinese citizen

News of a Chinese citizen’s arrest for misusing ChatGPT to spread fake news has surfaced, barely half a day after the dangers of such AI-generated news texts were highlighted. The suspect, whose surname is Hong, has been detained for “using artificial intelligence technology to concoct false and untrue information”, according to a statement by the police on Sunday.

The cyber division of a county police bureau noticed a false news article claiming that nine people were killed in a train accident in the local area on April 25, as reported by the Hong Kong-based South China Morning Post on Monday. Hong is said to have used AI technology to create and post the false information on multiple accounts.

The police have not revealed how the man obtained access to the AI technology. This case raises concerns about the potential misuse of ChatGPT and other AI-based technologies for creating and spreading false information and propaganda.

The cyber division of the police bureau in Kongtong county became aware of this fraudulent activity when they spotted an article regarding a fictitious train accident, which was simultaneously posted by more than 20 accounts on Baijiahao, a blog-style platform operated by Chinese search engine giant Baidu. The article reportedly received more than 15,000 clicks before the authorities took notice of the situation.

According to reports, the suspect Hong was accused of violating the crime of “picking quarrels and provoking trouble”. The Gansu public security department confirmed that Hong was apprehended for “using artificial intelligence technology to concoct false and untrue information.” In China, this offence can result in a maximum sentence of five years, but for severe cases, additional penalties may be applied, resulting in up to 10 years of imprisonment.

This arrest is significant as it is the first time Chinese authorities have detained an individual for the misuse of ChatGPT technology. Furthermore, it is notable that this occurred after Beijing’s initial provisions to regulate the use of “deepfake” technology officially took effect in January.

According to the police, the origin of the fake news article claiming a train accident in northwestern Gansu province was traced to a company owned by the suspect Hong, who had registered personal media platforms in Shenzhen, Guangdong province. The police detained Hong 10 days after they discovered the article, and a search of his computer was conducted.

The police’s statement revealed that Hong had confessed to bypassing Baijiahao’s duplication check function to publish on multiple accounts that he had acquired. He used ChatGPT to generate different versions of the same fake story, inputting elements of trending social stories in China from past years to quickly create these stories. It was also noted that while ChatGPT is not directly accessible to Chinese IP addresses, users can still access the service through a reliable VPN connection.

In response to the misuse of ChatGPT, China’s top internet regulator has expressed concerns about the technology being used for criminal activities, such as scams or defamation. China closely monitors its social media through firewalls, particularly Sina Weibo, which has over 592 million users, to ensure that no critical content is posted against the ruling Communist Party of China (CPC).

As ChatGPT’s popularity has grown, China’s law enforcement agencies have voiced their suspicions and warnings about the technology. In February, police in Beijing specifically cautioned the public to be wary of rumours generated by ChatGPT.

SiGMA Group is proud to present a groundbreaking event in Asia

The AIBC Asia Summit is a must-attend conference for anyone passionate about the future of technology in the region. This summit provides a unique opportunity to network with industry leaders, attend KOL-led conferences, and participate in innovative expos. Stay informed about the latest news and updates by visiting our website and register early to secure your spot at this exciting event.