Is StackOverflow Losing Its Way?

I find myself rather surprised to be writing this post. I’ve long been a supporter of StackOverflow as a far more effective means of getting answers to programming questions than the extremely low signal-to-noise ratio you get get on the more traditional forum-based approach.

Gaming the System

StackOverflow, like many sites, uses a point-based system called reputation (and karma and other names elsewhere). You get points for having your posts voted up by other users. You lose points for having them voted down but unlike many other sites downvoting costs you a token amount and that token payment is important. It does a lot to stop spurious downvoting as you’ll witness elsewhere.

6 Simple Tips to Get Stackoverflow Reputation Fast was written by a user who managed to 25,000 reputation in ~4 months. That’s a little over 200 per day (the soft cap) so is noteworthy but not extraordinary.

As a long-time user with almost 50,000 rep I feel compelled to comment on some of the points and qualified to do so.

1. Be the First to Answer. Even at the cost of quality.

Being first is (or, rather, was but we’ll get to that) important but that’s not a bad thing. A user getting a quick answer is a good thing as long as it’s useful. If it’s wrong it’ll probably get downvoted. So you can’t just post anything or any benefit of being first will be irrelevant.

2. Use Downvotes and Comments Strategically

This is basically true. One negative behaviour this encourages is what’s usually termed “tactical downvoting” where you vote down another answer simply to improve the relative positioning of your own answer. Various fixes of this have been suggested like disallowing downvoting on something you don’t answer. I don’t agree with that because you often end up answering something because something is wrong.

My personal preference is either to not have anonymous downvoting or not to have it when you also answer in the interests of full disclosure.

3. Use obnoxious in-your-face formatting and lists.

I don’t know why the author chose the term “obnoxious”. It’s true that an answer with some combination of images, links, lists, block quotes and other content visually different from text that breaks up that text will, in general, be more pleasing to the eye and more easily noticed on casual inspection.

4. Be Aware of the 200 rep/day Limit

True enough if your sole goal is to increase your points. The only one negative thing about this (which again has met with stubborn resistance to fixing or changing) is that the point gain is inconsistent. Accepted answers, for example, can go above the 200/day limit but only if they happen when you’re at a score that will go over with the additional points. This means things like the order of upvotes and accepted answers can change the end number. There are lots of inconsistencies around this but they’ve all been written off as being “by design”, which in my view a failing.

5. Edit, But Don’t Edit Too Much

The author is correct about this. If you edit your own post ~6-7 times it becomes a form of wiki (“community wiki” is the official term) and not owned by anyone. Any future upvotes will gain you no points. This was done to stop people endlessly bumping their posts but, in this author’s opinion, it fails miserably at that. There are many ways of bumping your own post without editing it:

  • Add another answer as yourself or a so-called “sock puppet”;
  • Edit another post on that same question (requires 3,000+ reputation); and
  • Retag the question (500+ reputation).

I’m told that a sufficient number of any kind of edit will force wiki status on it anyway but I’ve never witnessed this personally.

My suggestion is to not have edits to your post after the 6th or so bump the question. People shouldn’t be disincentivized from maintaining content they wrote.

So basically the author is correct and, barring some qualification, I have no problem with the post itself, but this brings us to…

Interestingly, this thread prompted Jeff Atwood to respond on Meta with 6 Simple Tips to Get Stackoverflow Reputation Fast. But I’m interested to know what about this post gets the attention of the powers that be when these issues months old and cold in the ground having been declined or declared to be “by design”?

The Reddit Cesspool

The aforementioned post was posted to reddit as Why StackOverflow sucks, which to me is a gross misrepresentation of the author’s intent. Obviously the submitter simply took it as a case in point of why he thinks StackOverflow sucks but that’s his axe to grind.

The comments (over 300 at last count) make for a sad and in many ways predictable read. My favourite is from an urchin named relix:

I've answered exactly 1 question. It was the only correct answer to that question but it didn't get any votes and didn't get selected as the answer. Instead, the wrong answer was selected and got all the brownie points.

 

That was the moment I decided never to waste time on StackOverflow again. Why even bother.

The only sad part about it is that for some reason 27 people have seen fit to upvote this drivel. A neophyte picked a bad alternative answer. Get over it. It’s worth noting the answer in question currently has 49 upvotes (which is a lot in SO terms).

One has to wonder how many of the reddit commentors actually read the linked article or just took the title as invitation to grind their particular axes.

Yet this idea that the system is worthless if it isn’t perfect is unfortunately rather pervasive among many reddit regulars. My opinion is that this belies the likely age of frequent redditors as all or nothing black and white thinking is a common malaise of those under 25, particularly amongst programmers who have a tendency for that kind of thinking anyway.

But I Just Want To Talk!

The other common complaint about StackOverflow in these comments was the fact that it was hard to have a conversation with the chosen format. Yes it is. That’s the point.

Take the comments on reddit, slashdot or any other reasonably popular site. It’s not hard to get 500-1000+ comments on an article. What purpose do they serve? Much like the answers on Stackoverflow, you’ll find that people will scan the first few, quickly get bored and move on. There are probably a half dozen people on the planet that read every one of those comments.

These people are the East German judges of the internet.

To put it in statistical terms, they are outliers that will distort your sample. You simply throw them out as bad data and look at the rest.

In a programming context, it’s easy for forums (which are designed for idle chit-chat) to degenerate into noise. The example I used is the difference between able to ask a question about Haskell and getting an answer versus having that same forum topic degenerating into 15 pages of whether the Death Star uses Haskell or C#.

If you want to have a chat, Stackoverflow is not the place for you. This is a Good Thing.

Some see value in such endless dicussions (“evolve” was used on more than one occasion). You’re kidding yourselves if you think that. Only 5 other people will read them all and you’re all so closed-minded that you simply won’t convince anyone who disagrees with you anyway.

And yes that’s a generalization but where there’s smoke there’s fire.

I’ll Take Any Answer

A recent change was to display acceptance rates of users, being the percentage of questions they have asked that they have chosen an accepted answer for (with some caveats).

The idea is to encourage people to accept answers. Often you’ll get someone come to the site, ask their question, get an answer and that’s the last you’ll hear from them. If so, putting their acceptance rate up isn’t going to change anything.

So is the target more experienced users? If so, should we be encouraging people to pick answers for which there really is no satisfactory answer yet to avoid at least the appearance of not using the site right (by accepting answers)?

To me this is tinsel.

Fastest Gun In The West

Returning to the first point of codexon’s post: yes being first is useful from a pure points view. I consider this a good thing. Getting an answer within minutes is good. Waiting a day and hoping someone checks the forum for your post (let alone bothers to answer it) is not.

What has happened to the sorting of answers on stack overflow? has caused a bit of a stir. Previously the default view for answers was net votes first (highest to lowest), time of post second (oldest to newest). This has changed so posts with the same net votes will be randomly sorted on each view.

There are many reasons I hate this change.

  1. Until users figure this out, there will be a much greater tendency of picking a random answer even if it was a “me too” answer that came much later;
  2. It makes it easier to clone other answers because you now have to rely on people looking at timestamps to figure out who posted first rather than the natural order;
  3. It’s even less worthwhile putting extra effort into particularly low-vote questions because upvotes are a far more hit-and-miss proposition;
  4. The Fastest Gun In the West “problem” is not a problem; and
  5. This change is huge. To trade the devil you know for the devil you don’t for such small apparent (potential) upside is a huge risk.

This most definitely isn’t tinsel.

The old system clearly got questions answered. Isn’t that the most important thing? So what’s the problem?

Conclusion

What I’ve been seeing over the last month or two are changes that have more on from the cosmetic (and/or irrelevant) to the downright undesirable. Combine this with the obstinate refusal to even acknowledge the downsides of the current system (let alone do anything about them) is now bordering on the bewildering.

What’s more, I’m disappointed that it takes this kind of external scrutiny to grab attention when the Stackoverflow community—who have been vocal on these issues for a long time—is seemingly dismissed?

One hopes that the community is being listened to on the site created specifically for them to have a voice but seeing recent changes and that external scrutiny rather than internal dialogue is provoking a response, one has to question how much they’re being listened to.

13 comments:

Shog9 said...

Small correction: the user in question (#57757) gained only about 10K in aproximately 5 months - that's roughly my rate, and i *rarely* hit the daily rep cap (it comes into play with the occasional unusually-popular answer, but isn't something i ever really have to worry about).

As for the rest of the essay... This is the second time i can recall seeing a significant change made to the system without any real discussion, seemingly in response to shrill comments off-site. I don't blame Jeff&Co for wanting to avoid bad press, but it does kinda reinforce the idea that discussion on the blog or MSO is worth considerably less than grandstanding elsewhere.

William Shields said...

What's the other significant change?

Anonymous said...

it's -> its

hyperslug

Shog9 said...

The first to catch my attention was Vote-based closing

Codexon said...

Shog9, my estimated time to gain that much reputation is around 4 months with full participation.

The first couple of months I was fairly inactive, and during the summer I took many days off and received residual reputation during the days I was inactive. My rep/day averaged about 225 during these active periods.

I'd also like to point out that I was more of protecting my higher quality (imho) answers from copycats and fast answerers rather than trying to destroy Stackoverflow. I think I've pointed this out enough times but it hasn't stopped the flood of hate mail and calls to have me banned.

Eric Burke said...

Apostrophe Abuse in your title.

JohnFx said...

I had no idea about the wikki-ization of posts that you edit a lot. Thanks for pointing that out. I often respond quickly to a question (per the previous tip) then go back and edit it in response to comments or as the OP provides clarification.

I didn't realize it was costing me rep.

Reboltutorial said...

I'm amazed people have time to play this game and for what is it worth :)

Anonymous said...

Stackoverflow often helped me just by google searches, even if i'm not registered or anything.

So even though i think the reputation system is pointless, thank you for your time.

Anonymous said...

I could give a shit less about my reputation/points. I've saved hours and hours of my life by receiving answers from genuinely helpful users on SO - which is awesome. In my eyes it's a fast and helpful site, and I don't care about points. If I was that lame I would just play WoW.

Anonymous said...

Rep is so lame. I don't care about that. In fact it encourages people not to answer. The reason is: Some moron shoots of his mouth because of #1 "be the first to answer". Then no other morons will answer because of #1. Morons. If the question manages to catch some non-moron's attention, you are in luck!

At first, there were lots of good answers. Now there are lots of morons with lots of rep. Good luck SO!

The Dude said...

i couldn't care less about rep

stef said...

I agree,
The "high grade" people are playing gods !
I gave up and just using Stack as a guest !

Post a Comment