Article · AI translation
Thanks to IT Home reader Autumn_Dream for the tip!
IT Home reported on October 28 that, according to a Douyin bulletin, the spread of AI tools has further lowered the barrier to content creation, giving creators more convenient ways to create and express themselves. However, some “malicious marketing accounts” whose sole aim is to monetize traffic have used AI tools as a shortcut to build accounts rather than for legitimate creation. By generating content in bulk and copying popular templates to make money quickly, they continue to erode the platform’s ecosystem, severely squeeze the space available to quality creators, and undermine a fair environment for creative competition.
Recently, the platform found that some malicious marketing accounts were using AI tools to rapidly increase the volume of malicious marketing content. This has brought new challenges in content formats and distribution methods, and some accounts are also backed by illicit organizations that teach these practices. Their rule-breaking methods change quickly and are hard to detect. At present, AI-driven malicious marketing accounts mainly pose 3 problems: low-cost, homogeneous content produced on an industrial scale; behavior often accompanied by attempts to induce interaction; and, among some of the most egregious accounts, the spread of false information generated by AI. To maintain a healthy creative environment, the platform is continuing to take action against malicious marketing accounts that misuse AI tools in these ways.
IT Home provides the officially published cases below:
Case 1: Using AI to generate content that induces interaction, gain traffic through deception, and rapidly build accounts
This mainly involves using AI to generate blessing and gift-themed content, such as “miracle doctors bringing blessings,” “welcoming the God of Wealth,” and “paying tribute to soldiers,” to induce users to like, comment, and share, thereby gaining traffic through deception and rapidly building accounts.
For example, 450 accounts posted this type of content, with clear signs of templating, and attempted to monetize through product sales after gaining some traffic.
Other accounts continually posted AI-generated short skits about conflicts, focusing on family ethics, social tensions, and other gratifying drama plots. They often used scripted appeals at the end of videos to solicit likes and support, and rapidly built accounts in bulk by posting large amounts of similar content and copying one another’s posts. Of these, 807 accounts have been dealt with because their content was highly homogeneous.
Case 2: Using AI to impersonate public figures, capitalize on their popularity, and market products
This mainly involves using technologies such as AI-generated digital humans and synthesized voices to impersonate public figures or their relatives on camera, inventing facts to gain trust, and then directing viewers to buy products.
For example, some accounts used AI-generated digital humans to impersonate celebrities’ relatives, gaining trust with false health stories such as “claiming to have had cancer and recovering after drinking a certain product,” then directing users at the end of the video to place orders through their product storefronts. A total of 1617 accounts appeared in videos selling products this way. Multiple accounts submitted identical content and sold identical products, suggesting they may have received coordinated instruction from a group. They also falsely claimed to have authorization from “Douyin Variety Shows” to induce users to join an organization. All the accounts involved have been dealt with.
Other accounts produced content in bulk using AI to imitate celebrities’ voices for narration alongside rewritten scripts, and built multiple accounts around the same topics, impersonating the public figures themselves to spread related information. For example, 401 accounts used AI to synthesize public figures’ voices and mass-produce content featuring quotations or explanations of Chinese zodiac signs. Some accounts sold products related to traditional Chinese medicine, showing a clear marketing intent. The accounts involved have been dealt with.
Case 3: Using AI to fabricate false information in bulk to attract traffic
This mainly involves using AI to generate fake imagery, fabricated personas, or templated rumors to gain traffic and then market products or services.
Some accounts built networks of accounts in bulk, using AI-generated alarming scenes such as exploding household appliances to catch users’ attention, then directing them to storefronts selling related cleaning products. For example, 30 accounts used such scenes to drive traffic to product sales. All the accounts involved have been dealt with.
Other accounts used networks of new accounts to repeatedly post AI-generated false content about elderly pregnant women, disguising it as user experiences while actually directing traffic to IVF organizations. For example, 143 accounts posted AI-generated false “successful pregnancy” content in bulk to direct users to IVF organizations outside the platform. The vast majority of these accounts had been registered within the past 2 months, and all the accounts involved have been dealt with.
Still other accounts used AI to generate templated dating personas, then shifted to other forms of monetization after attracting traffic. For example, 66 accounts used AI-generated images of middle-aged and elderly men paired with audio love confessions to attract the attention of middle-aged and elderly women. All the accounts involved have been dealt with.
Case 4: Using AI-generated frightening, superstitious, and bizarre content to attract attention, direct traffic, and profit
This mainly involves using AI to generate bizarre content involving traditional superstitions and frightening depictions of the underworld, or creating a superstitious atmosphere to induce user interaction, then profiting by selling ritual supplies, directing users to paid divination services, or moving traffic to private channels. Such content can cause psychological discomfort and may also spread superstition and mislead users.
For example, 32 accounts used old-style Taoist names such as “Taoist,” “senior disciple,” and “master.” Under the guise of sharing folk customs, they posted AI-generated superstitious content in bulk about life in the underworld, the journey of departed souls, working in the underworld, and rituals to help the dead pass on. They created anxiety by suggesting that “without money in the underworld, you cannot get anywhere,” and directed users to buy ritual supplies called Jade Emperor money. Another 154 accounts used AI to generate images of white foxes, mythical beasts, and Guanyin Bodhisattva, adding audio that offered blessings and good fortune. They also used themes such as “visiting relatives in heaven” and “searching for relatives in the underworld” to induce interactions in the comments. After building up the accounts, they directed users to paid services outside the platform that claimed to “help check on the current state of deceased loved ones.” The platform has taken action against the accounts involved in these violations.
Case 5: Using AI tools to copy content cheaply, build accounts in bulk, and monetize them
This behavior is not confined to any particular content topic. Instead, the accounts treat AI tools as a shortcut to building an audience. Once a format becomes popular on the platform, large numbers of marketing accounts follow by imitating it with AI to build accounts in bulk. This has also given rise to an illicit ecosystem of paid instruction. Areas where traffic can be converted into revenue quickly have become especially prone to such violations.
For example, 304 accounts posted AI-generated comforting content in bulk, using emotional resonance and childhood nostalgia to attract users, then turned to obtaining traffic incentives for revenue after gaining followers. Another 732 accounts posted templated content in bulk under the banner of “popular science,” including biographies of historical figures, stories about medicinal herbs, and traditional Chinese culture, to lay the groundwork for later product sales or paid knowledge content. A further 108 accounts concentrated on folk stories, with strikingly similar plots across accounts, copying popular templates in bulk. In addition, 296 accounts used AI to generate product-selling videos in bulk with highly similar character backgrounds, dialogue scripts, and products for sale. All the accounts involved have been dealt with.
IT Home learned from the announcement that, over the past month, the Douyin platform has acted against a total of 1.135 million short videos that violated its rules and 507,000 offending accounts under its “Account Health Score Rules.” Another 302,000 accounts were guided toward compliant content creation through measures such as platform rules study and exams. Going forward, the platform will continue to step up its efforts against malicious marketing carried out through misuse of AI tools, strengthen its efforts to trace and combat illicit tools and organizations that teach these practices, and welcome users to provide leads on violations through its reporting channels to help maintain a healthy online environment.