In Addition to Post Count - Power Ranking?

xrunner

  • Posts: 192
In Addition to Post Count - Power Ranking?
« on June 5th, 2013, 01:36 AM »
Has anyone ever looked into, what I'll call for the time being, a "power" ranking for each member. I hate for people to use the post count to determine who's a valuable/high contributing member.

Let's say John has 50 posts, but he doesn't really contribute much for each post, and he hasn't started any topics. Suppose he mainly makes replies such as "Good points in that post" and the like. If I added all the individual characters he's posted it comes to 2500.

Then we have Mary who has 20 posts, but they are really good and consist of several paragraphs each, and she's started 5 topics that generated, respectively, 10, 15, 20, 10, and 5 responses from other members. Lets say her character count was 10,000.

Mary provides much more content than John, generatates more traffic (topic replies) but has fewer posts. How to show this?

So a power ranking might be characters posted * total topics started (if greater than zero) * total replies to topics started (if greater than zero).

So if you take John, you have 2500 characters and no topics started and (therefore) no replies to your topics, the total is: 50 * N/A * N/A = 50.

Take Mary - If you have 10,000 char. and 5 topics started and 10+15+20+10+5 replies to each topic, the total is: 10,000 * 5 topic starts * 60 replies from other members = 3,000,000.

Now to avoid large numbers you scale them by, say, 1000.

So the power rankings would be -

John: 0.05
Mary: 3000


That's just a brainstorm I had, the values chosen for the calculations could probably be improved. Is that an idea (or something like it) worth consideration as a built-in feature?

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: In Addition to Post Count - Power Ranking?
« Reply #1, on June 5th, 2013, 01:38 AM »
It's basically called reputation for which mods already exist in SMF to do pretty much all this.

I'm reluctant to put them in the core however because it's a fairly niche thing and prone to abuse.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

MultiformeIngegno

  • Posts: 1,337

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: In Addition to Post Count - Power Ranking?
« Reply #3, on June 5th, 2013, 02:03 AM »
While Hermeticism is cool and all, the average community wouldn't really function best out of it.

Even the most well intentioned reputation mechanic gets abused by egomaniacs. For anyone who thinks I have an ego problem, hang around StackOverflow for a while and look at the attitude of the highest-reputation people.

MultiformeIngegno

  • Posts: 1,337
Re: In Addition to Post Count - Power Ranking?
« Reply #4, on June 5th, 2013, 02:08 AM »
True. What I was saying is that ponderating post counts with char count could be wrong. It's quantity vs quality. Everyone has his style. Maybe one user makes a more qualified reply with less chars.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

MultiformeIngegno

  • Posts: 1,337

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

xrunner

  • Posts: 192
Re: In Addition to Post Count - Power Ranking?
« Reply #8, on June 5th, 2013, 02:16 AM »
Quote from MultiformeIngegno on June 5th, 2013, 02:08 AM
True. What I was saying is that ponderating post counts with char count could be wrong. It's quantity vs quality. Everyone has his style. Maybe one user makes a more qualified reply with less chars.
Of course it could be wrong, it's just a starter idea.

But it's also wrong to look at a post count and determine how "valuable" a member is from that number, yet many people think it's meaningful. Any idiot can post 100 times, but the post count is the weakest indicator of meaningful activity there is.

For example, there is some measure of value if a post generates 15 responses, even if the post itself is bad, because it generates valuable corrections and discussion, so to that extent the person who posted the bad post should get more credit for generating forum activity - because the very activity is valuable.

Surely we can come up with something better than the simple integer called "post count".

MultiformeIngegno

  • Posts: 1,337

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: In Addition to Post Count - Power Ranking?
« Reply #10, on June 5th, 2013, 02:26 AM »
That's the point, though, you're assigning a metric that doesn't necessarily hold up either.

Let's take something akin to a real world example: sm.org. Nao started the Aeva thread, it has something like 6000 replies to it which if I remember rightly is actually more than his entire post count there.

Then you have me, I didn't make that many threads (and it's screwed up because some of them aren't properly attributed to me, and I won't even get into the debacle of mod threads that are no longer attributed to me)... but I have 9x the post count, and of the topics I made, I suspect I probably merited a similar amount of replies *in total* to them. Hard to say but let's take it as a reasonable hypothesis. I don't know how many topics I started but including the private ones in the mod reviews board, let's say 100 threads in total. Vs the... 10? threads Nao made.

By that reckoning, my contributions to sm.org are 51k * 6k * 100 vs Nao's 6k * 6k * 10 = 30.6bn vs 360m - several orders of magnitude even with a linear scaling factor - and that's without distorting it based on size (since I wrote some pretty large posts too)

And that's being conservative in some areas. I like the idea in principle but the practice just doesn't quite work out. There are just too many variables to bring into a common field of statistical reference and too many ways to screw up such an algorithm.

xrunner

  • Posts: 192
Re: In Addition to Post Count - Power Ranking?
« Reply #11, on June 5th, 2013, 02:33 AM »
Quote from Arantor on June 5th, 2013, 02:26 AM
By that reckoning, my contributions to sm.org are 51k * 6k * 100 vs Nao's 6k * 6k * 10 = 30.6bn vs 360m - several orders of magnitude even with a linear scaling factor - and that's without distorting it based on size (since I wrote some pretty large posts too)
Well you're kicking ass there! That would be a great Power Ranking! :cool:
Quote
And that's being conservative in some areas. I like the idea in principle but the practice just doesn't quite work out. There are just too many variables to bring into a common field of statistical reference and too many ways to screw up such an algorithm.
Well I knew the original equation I started with wasn't going to fly, I wanted to read what you all thought and try to take into consideration for possible refinement. I just hate looking at a member's post count on new forums I go to and trying to make any kind of determination as to their "value". Of course since I just said that, then I don't do it myself, but I think a lot of other people just look at that number and cower in fear when sometimes the person with 5,000 posts is no better at whatever the subject of the forum is than they are. >:(

That's what I'm trying to make better, somehow. :)

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: In Addition to Post Count - Power Ranking?
« Reply #12, on June 5th, 2013, 02:38 AM »
Oh, I know where you're going and it's fundamentally a good place. Unfortunately until a machine can objectively rank something in terms of quality[1] it's just not going to fly.

There is a certain forum I can think of where pretty much everyone doesn't really know the subject in question. Imagine that: a forum for beginners... by beginners. It doesn't work well.

Post count is a crappy metric. Topic count is a similarly crappy metric. Likes is better, so is reputation but both will to a point scale with post count - the more posts one makes, the greater likelihood someone will like them or upvote them or whatever.
 1. IBM Watson is getting there... it managed to algorithmically judge an analyst's report as 'bullshit' because someone thought it would be great to feed the Urban Dictionary into the finest AI we've ever built

xrunner

  • Posts: 192
Re: In Addition to Post Count - Power Ranking?
« Reply #13, on June 5th, 2013, 02:50 AM »
Quote from Arantor on June 5th, 2013, 02:38 AM
There is a certain forum I can think of where pretty much everyone doesn't really know the subject in question. Imagine that: a forum for beginners... by beginners. It doesn't work well.
Oh boy - that's got to be kindof entertaining to visit.
Quote
Post count is a crappy metric. Topic count is a similarly crappy metric. Likes is better, so is reputation but both will to a point scale with post count - the more posts one makes, the greater likelihood someone will like them or upvote them or whatever.
And then there's karma, which I hate. There, you get into karma wars and it turns into a pile of muck - with little meaning. That's why, because my members want it, I refused to allow it unless they fessed up to the karma action, so I use the karma log mod. At least I can see who's making B.S. karma and if the reason isn't to my satisfaction it will be removed.

But - I do know the post count can be hidden. Perhaps if we all had it to do over again, if forums were just being invented, we'd keep post count away from prying eyes like PMs. People would be judged as they are, day by day, by the quality of their contributions.

I know, I'm a hopeless dreamer. :-/


Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: In Addition to Post Count - Power Ranking?
« Reply #14, on June 5th, 2013, 02:56 AM »
Quote from xrunner on June 5th, 2013, 02:50 AM
Oh boy - that's got to be kindof entertaining to visit.
It's not entertaining, it's kind of sad really.
Quote from xrunner on June 5th, 2013, 02:50 AM
And then there's karma, which I hate. There, you get into karma wars and it turns into a pile of muck - with little meaning. That's why, because my members want it, I refused to allow it unless they fessed up to the karma action, so I use the karma log mod. At least I can see who's making B.S. karma and if the reason isn't to my satisfaction it will be removed.
Karma's abusable because it's for the *person* not for the *post* so it's inherently fecked anyway. The only way it works is if it's per-post, but even then it still needs careful watching.
Quote from xrunner on June 5th, 2013, 02:50 AM
But - I do know the post count can be hidden. Perhaps if we all had it to do over again, if forums were just being invented, we'd keep post count away from prying eyes like PMs. People would be judged as they are, day by day, by the quality of their contributions.

I know, I'm a hopeless dreamer. :-/
Nah. We'd just want some other metric to judge on. Numbers, for their ills, are easily digestible - and a member with an obscenely high post count is usually not likely to be a troll or otherwise a pain because if they were, they'd have been dealt with and summarily ejected.

It's the middle ground, the 0-5k post ground to watch out for, by the time they're past that, they're almost certainly in it for the long haul.