Peer Awards Increase User Content Generation but Reduce Content Novelty

Abstract

Platforms that depend on user-generated content spend a great deal of effort crafting policies and
mechanisms that can yield a steady stream of engaging content. In this work, we consider the
effects of awards offered by peers, a feature that many platforms provide to enable users to
recognize the quality of their peers’ contributions. We conduct a large-scale field experiment on
Reddit, one of the largest social news aggregation and discussion platforms in the world and
evaluate the effect of peer awards on content generation in terms of both volume and novelty.
We leverage Reddit’s native peer award feature, the Gold Award, purchasing and randomly
assigning Gold to 905 posts, anonymously, over the course of two months. We collect and
analyze users’ behavioral trace data and posting content over the period leading up to and
following our treatments via Reddit’s API. We find that, on average, peer awards raise the
probability that treated subjects will make additional Reddit posts by 6.6%, in addition to
lengthening treated subjects’ posts by approximately 39.6%. Interestingly, however, we also
observe that the content users post under treatment exhibits greater similarity to past content
(particularly the intervention post), indicating a decline in novelty. Based on this result, we
conclude that peer awards are a double-edged sword. On the one hand, they foster increased
engagement and content production among recipients. On the other hand, the additional content
that awards elicit is less novel.

Keywords: Peer awards, user-generated content, novelty, Reddit, text-mining, field experiment

More information on Gordon Burtch can be found at: https://carlsonschool.umn.edu/faculty/gordon-burtch