Notes on pending changes

August 13th, 2010 by

Back in June, I wrote about the then-almost-implemented pending changes system on Wikipedia. What’s it like two months on?

On the whole, I’m more than happy with its effects, and the feared imminent catastrophes haven’t materialised yet. Lag time to approve edits is pretty low; I haven’t dug up the turnaround times, but the page listing unchecked edits regularly changes completely in the few minutes between my first loading it and my remembering the tab is there and refreshing it. Indeed, it’s not uncommon to see the page empty entirely, and I’ve only rarely seen it listing more than half-a-dozen pages (out of a pool of ~2000). The lack of “pending pending changes” at any given moment also meant that spotting them via the watchlist, or casual browsing, was unlikely; to be aware of them, you usually needed to go to the central page. “Review conflicts” are quite common – perhaps a result of the noticeable slowness of the system on larger pages – but, then, so are rollback conflicts. This could definitely improve from speeding the page loading times up, I suspect; less time with the page pending is less time to have someone else come in.

The biggest problem I’ve found so far is, if anything, one of overenthusiasm. Whereas before we’d have a degree of “masterly inactivity” practiced on a lot of edits – someone would look at it, decide they don’t know enough to determine if it’s good or bad, and leave it be – the new system seems to have the effect of making people feel they ought to say one way or the other. Net result: more suboptimal approvals or rejections (ie, reverts), by people unfamiliar with what they’re dealing with, than we had before.

Why? Well, we have the central page, blinking at us, telling us there were four pages needing checked – four, just four! – and that there was a timer somewhere to note how long they took, and so on and so forth. There’s an impulse there, even if an unconscious one, to just do something so as to drive down the backlog.

Interestingly, this may be a problem that disappears as the system settles down, and becomes familiar and less excitingly novel. While there’s a small backlog – especially for a flagship new system – people will always feel the urge to just wipe the board clean, to keep it resolved, to have the satisfaction of having sorted it out. Once that backlog grows to a constant buffer of maybe twenty or fifty edits, the impulse to knock them all off while you make a cup of tea is sharply reduced, and so the likelihood of them being done for the sake of it is lowered; it becomes more likely that the edits will be picked up by someone who is intentionally watching the page, which is a good first approximation to “someone who knows what’s good”.

Assuming we have a fixed number of articles – protecting pages for the sake of protecting them is a bit odd – then the number of edits coming in will be constant; growing the buffer implies growing turnaround times, which is not the best thing. On the other hand, it’s probably inevitable – as the novelty wears off, and we stop thinking of it as an Important New Thing That Must Be Perfect, people are going to patrol the central page a bit less. It could well be that this inevitable decrease in responsiveness will actually have the unexpected benefit of improving the quality of reviewing.

Tags:

Leave a Reply