Twitter has introduced a new anti-spam system called BotMaker and says that it is responsible for a 40 percent reduction in its key spam metrics.
The BotMaker anti-spam system was created with one low-latency sub-system (Scarecrow) which checks in real time the content posted on the site and decides whether the content should be approved or not. The second computationally-intense and learning sub-system (Sniper) checking in 'near real time' the user and content event logs that make it past the Scarecrow.
The BotMaker is constantly fed information by Scarecrow and Sniper, and issues commands to approve, deny or challenge posts. The micro-blogging site also runs periodic jobs on all the data compiled by the BotMaker system for routine checks to specific exercises by the engineering department.
Twitter's Raghav Jeyaraman stated in an official blog post the challenges faced by the team in creating BotMaker. He stated that due to Twitter's wide-ranging developer APIs, meant for third-parties to interact with the platform, spammers "know (almost) everything" about how the micro-blogging network functions, which makes it difficult to create an anti-spam system.
Keeping in mind these challenges, the team created a system that would prevent spam content from being created, lessen the amount of time spam is visible on Twitter, as well as reduce the reaction time to new spam attacks.
Jeyaraman wrote, "These operating conditions are a stark contrast to the constraints placed upon more traditional systems, like email, where data is private and adding latency of tens of seconds goes unnoticed. So, to fight spam on Twitter, we built BotMaker, a system that we designed and implemented from the ground up that forms a solid foundation for our principled defense against unsolicited content," he said. "The system handles billions of events every day in production, and we have seen a 40 percent reduction in key spam metrics since launching BotMaker."
A recent report by Twitter revealed that nearly 23 million of active users on the site are 'Bots'. However, the company says that these accounts are not necessarily spam accounts, which make up less than 5% of total MAUs. These spam accounts affect advertisers who are interested in reaching potential customers through the micro blogging site.
Source: Twitter