Can You Spot a Paid Product Review? Researchers Are Getting Better at It

How do you know if that glowing 5-star user review about the product you're researching wasn't just written by a paid professional? New Scientist reports on a paper in ArXiv, an e-print archive hosted by Cornell University, that has the inside scoop from a former member of China's "water army" -- paid posters that flooded the Internet with product reviews.

Cheng Chen and his colleagues conducted a study to "help distinguish a special group of online users, called hidden paid posters, or termed "Internet water army" in China, from the legitimate ones." In the paper's abstract, the researchers write that the water army is paid to "influence the opinion of other people towards certain social events or business markets."

The researchers evaluated the pattern of online posters, which New Scientist reports includes rules for posting so as not to look suspicious. New Scientist reports that the researchers looked at competing antivirus companies that had been commented about by more than 500 users on the Internet. Of these, the researchers believe that 70 could be from paid posters:

These suspected paid posters had a higher proportion of new comments (as opposed to replies), posted more often but for a shorter period of time, and were more likely to post similar comments several times than posters not suspected of being paid.

Sure enough, when the researchers applied these criteria to comments on a second news site, the suspected paid posters they flagged matched Chen's subjective classification with a false-positive rate of 1 percent and a false negative rate of 10 percent.

In the paper, the researchers state that while they've found clues as to the organizational structure of paid posters, more research should be conducted to improve detection techniques and potentially develop an algorithm for software that could detect paid comments.