4

This may have been discussed previously, but my google fu isn't strong enough if it has been. The rather frustrating meta conversation about why this site sucks got me thinking about this. I guess this question may really belong on MSO, but I thought I'd vet it here first.

Should some people have a stronger voice or more trust based on a proven track record? SE already does this quite a lot. As you gain rep, you get more privileges, allowing users to get to near mod status. A user's rep is prominently displayed on any answer, lending an air of importance to their answers as their rep increases (if there's a lot of answers on a page, I definitely skip the string of zero vote questions from low rep users).

But the conversation of Subject Matter Experts (SME), valued users leaving the site (Darin's last visit was a month ago), and how to answer subjective questions got me thinking about another possible rep based ability. The site already awards badges based on proving as an SME (so many upvotes on a topic equals badge). Should those badges also bring about new abilities? Specifically, a silver badge for [tag] implies that your up-vote carries two votes, making your voice louder once the community has deemed you an SME. Gold badge could carry even more weight. You could also add more weight at a certain rep limit (i.e. 10k rep = x2 vote count).

This would allow the trusted voices of the community to have more say in what is and isn't a good answer or question. Because, lets be honest, not all users are created equal. It may also help assuage some of the evaporative cooling effect, as the trusted and valued users are treated differently than the untrusted users who are still proving their value.

Thoughts?

2

Should those badges also bring about new abilities?

They already do; editing a tag-based wiki requires that you hold the bronze tag badge, or are in the top (n) users for that tag.

2

You're associating score with recommendation, votes with endorsement. Which is easy to do - on some level, the site encourages this, as a single vote can cause one answer to rise above another. And with this in mind, it's natural to desire weight for expertise - if votes were removed entirely, and the only indication of a post's quality were the comments other users left after reading them, then users would naturally give more credence to the endorsements or criticisms of users they respected...

But score is not a recommendation. If one post has a score of five, that just means five users thought it was helpful or worthwhile reading. If two answers have scores of 5 and 8 respectively, that could indicate that the latter is a better answer... or merely that the former was posted later and hasn't been read as much.

And votes aren't endorsements. Imagine again that there are no votes, just people leaving comments. You read a reasonable post, an answer written after thorough testing, backing up its assertions with citations and personal experiences. Ten comments follow, from users like yourself who tried the advice and found it useful. And then you see a single word comment from a Subject Matter Expert: "No". How much weight do you give this "vote"?

If an expert wants to tout or damn a given answer, all he needs to do is leave a comment explaining why he thinks it's valid or invalid. If he makes a reasonable point, and other users trust him, this will likely swing the votes far more effectively than any artificial weight the system might give him. But more importantly, this adds information to the site, while weighted votes do not.

  • 1
    this is exactly what I was trying in my own inept way to say. Thanks. – justkt Oct 13 '10 at 17:02
10

This is something that was previously discussed on MSO for Stack Overflow:

The idea was posted very early on by someone who is now a high-ranking member of the community and actually a current moderator.

It has a score of -9, with a rebuttal answer of score +24.

Now, admittedly, your proposal is slightly different. You're suggesting that people receive this weighting for upvotes in a specific tag, and only in that tag, as opposed to a general upvote weighting based on reputation in general. However, many of the old arguments still apply:

  • People are still attracted to high numbers. This system would allow the categorized experts to bias the community vote, which is exactly what the SE design aims to avoid (voting is as fair and impartial as possible).

  • Upvotes are still not (necessarily) the same as expertise. Upvotes, even in a single tag, are partly a measure of quality but also a measure of time, and someone who is not a real expert but has consistently provided many "decent" answers could easily amass more upvotes than someone who is a real expert and has only provided a few answers (albeit amazing ones).

  • We'd still be telling new members that, essentially, their votes don't matter, and at our current stage new membership is extremely important.

  • If the weight of downvotes were increased, the idea takes on sinister overtones, allowing privileged users to game the system by burying other answers to questions they themselves have answered.

In addition, any proposed solution based on tags suffers from its own problems:

  • Tags aren't always reliable. Over time we've put a major dent in the awful [technique] tag, but haven't even gotten started on the equally awful [cooking-techniques] tag. A lot of people just don't use good tags, especially newbies, and right now our community isn't great about retags (that's OK - even Stack Overflow has its issues).

  • Tags can change, and can be changed by users with fairly low reputation for that matter. This is a good thing, it helps keep the site organized, but presents several difficult conundrums when you start introducing important privileges based on tags. Should people without "tag expertise" be allowed to retag, even if the tags were blatantly wrong? If "tag experts" have placed votes, and then that tag is removed, are the votes then recalculated? What if the retag happened months later and the damage has already been done (i.e. by biasing the other votes)?

I completely agree with the premise that we need to recognize subject-matter experts. I think one of the things our friend ignored is that we already do this with the interweaving of reputation and privileges; subject-matter experts have veto power on questions.

Perhaps that's not enough. Perhaps we need some form of veto power on answers as well. It's tricky, though, because we don't close questions merely because they're bad - they have to be a particular kind of bad, and most of those criteria don't apply to answers. So such a feature would have to be thought out rather carefully to make it meaningful and prevent abuse.

I'm open to discussing better ways to retain experts - and in fact I think the Stack Overflow team is already discussing this issue internally and at length - but unfortunately, I don't think this is a good way. It's just fraught with problems, several of which could undermine the very factors that make these sites so appealing in the first place.


Anyway. I am sure Darin has his reasons for departing. We don't know if it's temporary or permanent. My guess is that, as our site passed the "peak of inflated expectations" in the Hype cycle, new questions started to dwindle and he wasn't seeing that many new and interesting ones. This too shall pass, it's part of the life cycle of every new site or product. In any case, I'm quite certain that the community recognized the value of his contributions as evidenced by his average of 5 upvotes per answer.

I also want to mention that just because it appeared on somebody's blog does not mean that the "evaporative cooling effect" is actually a real or interesting phenomenon. There is turnover in every community, and many communities are occasionally graced with the likes of subject-matter celebrities just as many restaurants are occasionally graced with the likes of Hollywood celebrities; that does not necessarily make them a "hot spot" for such or prove that they are any more successful than other communities.

The biggest problem with that post is that the author, by defining success on his own terms, has rigged the debate. It's choice-supportive bias and we have it too. Insular communities that over-emphasize expert/celebrity status do have higher immunity to the brain drain, but they also have far less immunity to groupthink and general decay. I have been a member of a few communities now that were great with member retention, never really suffered from trolling or low-quality members, but eventually just died of "old age" - people got bored and that was that.

What's ironic is that no matter how open your system is, people will still complain that it over-rewards the elite. Capitalism, supposedly, rewards the big corporations and the super-rich. But if that were true, then why does it work any better than feudalism? It's because anybody can compete. They can start from nothing at all, and if they're good enough, work their way up to the top.

SE sites are, or at least try to be, like free markets. The rags-to-riches has happened on Stack Overflow a number of times. It happened to me; I had a mostly inactive account, one day I decided to start actively participating, and in about 6 months I was near the top of page 3. Then I got, well, bored, but while I was on my way up I found myself neck-and-neck with another guy who did keep at it and he's now in the top 20. That's what's great about these sites - that people who aren't necessarily recognized experts can become recognized.

Perhaps some feel that they aren't "recognized" enough, but it doesn't seem to have deterred a lot of those people from continuing to actively contribute. And even if we do want to be "better" at recognizing them, I don't think that giving them "super-votes" is the right way, because while that may make those experts feel better about sticking around, it ultimately hinders the ability of other members to grow.

Perhaps what we need is a form of in-game currency. We sort of have that with bounties. Somebody who's amassed a huge amount of reputation can, without really giving it a second thought, fire up a bounty and award up to 500 rep points for what they think is an amazing answer. The bounty award even shows up next to the post, and unlike votes, bounties are not anonymous - you can see which expert awarded it.

Personally, I think that's actually enough. The recent-ish improvements to the bounty system actually provide a really great and useful way for people who consider themselves experts (and whom the community has recognized as experts) to highlight other experts and help promote them to expert status.

But maybe we need a bounty-like system for votes as well, that might give experts the ability to bump answers up the tree - at some cost to them. Possibly worth investigating as well.

However, in my opinion, not really worth investigating yet. We're barely into the growth stage; what we especially need is to attract new experts and not necessarily worry extensively about retaining the ones we have (unless they are seriously dissatisfied, obviously - but I don't think that's the case at the moment).

  • Thanks for the well thought out and detailed answer. I'll have to give it a think, but I think I'm convinced. – yossarian Oct 13 '10 at 15:29
0

I think that a trusted SME will automatically have a louder voice. In the meta thread you mentioned, Aaronut referenced Alan Kay's single answer and single question on Stack Overflow. Alan Kay's definitely an SME in the areas referenced (his own quotes and new inventions in computing), and boy did the community recognize that. He got 295 upvotes for saying essentially "I don't remember." If Harold McGee or [your favorite name in your favorite culinary subject] showed up here, people would recognize it. If I see that one of our resident professionals has left a comment indicating an answer is good, I'm way more likely to upvote it.

In short, I think the system already works.

  • 1
    That's not really my point. The community already rewards answers from people with good rep (either internal to the SE system or external like Alan Kay). What I'm suggesting is that votes from those people also carry more weight. If Roux, Darin, Michael, and Hobodave all upvote one answer and 10 users with <100 rep upvote another, the current system says the second answer is better. Since votes are anonymous, there's no way to gauge who is judging an answer. My suggestion would allow the anonymity while also giving trusted users a louder voice in voting. – yossarian Oct 13 '10 at 14:08
  • @yossarian - while it doesn't look like it from my current answer, I did understand that. This was supposed to be the meat of my answer 'If I see that one of our resident professionals has left a comment indicating an answer is good, I'm way more likely to upvote it.' I'll be interested to hear what others think, though! – justkt Oct 13 '10 at 14:36
  • 1
    So you did. I'm not sure how often the trusted people leave "+1" type comments. It also requires an extra step and association from other users for something that could be automated. I'm not convinced my suggestion is a good idea, but it was an idea, so I thought I'd see what the community thought. – yossarian Oct 13 '10 at 14:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .