Paper: pdf
Slides: pdf
Citation:
APA
Halfaker, A., Song, B. D., Stuart, D. A., Kittur, A., & Riedl, J. (2011). NICE: Social translucence through UI intervention, WikiSym'11 ACM, New York, NY, USA, (pp. 101--104).
bibtex
@inproceedings{halfaker11nice, author = {Halfaker, Aaron and Song, Bryan and Stuart, D. Alex and Kittur, Aniket and Riedl, John}, title = {NICE: Social translucence through UI intervention}, booktitle = {Proceedings of the 7th International Symposium on Wikis and Open Collaboration}, series = {WikiSym '11}, year = {2011}, isbn = {978-1-4503-0909-7}, location = {Mountain View, California}, pages = {101--104}, numpages = {4}, doi = {10.1145/2038558.2038575}, publisher = {ACM}, address = {New York, NY, USA}, }

NICE: Social translucence through UI intervention

Social production systems such as Wikipedia rely on attracting and motivating volunteer contributions to be successful. One strong demotivating factor can be when an editor's work is discarded, or "reverted", by others. In this paper we demonstrate evidence of this effect and design a novel interface aimed at improving communication between the reverting and reverted editors. We deployed the interface in a controlled experiment on the live Wikipedia site, and report on changes in the behavior of 487 contributors who were reverted by editors using our interface. Our results suggest that simple interface modifications (such as informing Wikipedians that the editor they are reverting is a newcomer) can have substantial positive effects in protecting against contribution loss in newcomers and improving the quality of work done by more experienced contributors.

Key finding

When experienced editors were reminded to be nice when reverting a newcomers work, reverted newcomers were significantly more likely to continue editing.

Summary

Social production systems such as Wikipedia rely on attracting and motivating volunteer contributions to be successful. One strong demotivating factor can be when an editor's work is discarded, or “reverted”, by others. In this research we demonstrate evidence of this demotivating effect and design a novel interface aimed at improving communication between the reverting and reverted editors. We deployed the interface in a controlled field study on the live Wikipedia site, and report on changes in the behavior of 487 contributors who were reverted by editors using our interface. Our results suggest that simple interface modifications (such as informing Wikipedians that the editor they are reverting is a newcomer) can have substantial positive effects in protecting against contribution loss in newcomers and improving the quality of work done by more experienced contributors.

Wikipedia is an interesting case of volunteer participation. The site maintains a large base of users who voluntarily do all of the dirty work of writing an encyclopedia. Despite the open and voluntary nature of the encyclopedia, it is of exceptional quality (Giles, 2005).

Recently, there has been press about a decline in the user base and the decreasing retention of new editors has been implicated as the primary cause (Editor Trends, 11) (Suh, 09).

Previous research by Zhang & Zhu (2007) showed that a decrease in editor activity could be measured after the articles they created were changed by other editors. They also found this effect to be more pronounced for newcomers. They concluded that the decrease in activity represented a decrease in motivation following the implied feedback of having an article changed by someone else.

We suspected that reverts, an edit to an article that discards the changes made by an editor, could represent a much more direct and effective feedback event and that the effects would be similarly deleterious for newcomers. Motivated by the recent decline in newcomer retention, we designed an experiment to determine the effects of negative feedback could be lessened.

Based off of previous work in Social Translucence (Erickson, 00) we hypothesized that editors would interact more effectively with newcomers if they were simply more aware of when they were interacting with one, so we tried something simple: we modified Wikipedia's interface to show editors when they were reverting newcomers for a group of editors who volunteered to participate.

Annotated screenshot of the NICE interface

Figure 1. The NICE interface.

After analyzing their activity and the activity of the reverted editors, we found a few interesting effects.

The sheer simplicity of the interface modification and the substantially effect that it had on retention suggests (1) that being reverted can be unnecessarily demotivating for newcomers and (2) improving the situation substantially can be simple and straightforward.

References