Nao

  • Dadman with a boy
  • Posts: 16,080
Bot logging
« on October 16th, 2011, 12:42 AM »
Hey guys.

Do you think it's all right to have smf take bots into account when calculating most online and similar stats?

I think it's unfair to take them into account. And misleading. But I fear that smf users switching to wedge would think that they lost all their visitors overnight and switch back :P

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Bot logging
« Reply #1, on October 16th, 2011, 12:45 AM »
It's not all right, however it has several interesting side effects.

Firstly, it would decrease queries on the server quite a margin (and queries that have complications, at that).

Secondly, for those folks who use Google Analytics, it would bring the number of hits reported much closer between SMF and GA, because spider hits aren't counted with GA.

What you could do is to record spider hits separately to visitor hits. You wouldn't get the saving of the first one, and the accuracy of the second would be preserved - and you'd be able to neutralise the effects of SMF users thinking they'd lost hits.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Nao

  • Dadman with a boy
  • Posts: 16,080
Re: Bot logging
« Reply #2, on October 16th, 2011, 01:20 AM »
Or we could simply have an option in the admin area, disabled by default, to skip bots from these logs...
Considering that there's no 100% sure way of determining whether an IP is a bot, it'd make sense to leave it up to the admin to decide what they wanna do. But personally, I would certainly enable such an option. As long as one is aware that less hits doesn't translate to less visitors...

IIRC, when I used to use GA on noisen.com, my stats over there were MUCH lower, something like 5 times less than SMF's own account.

Separating spider hits from the rest would be suitable as long as we have an option to disable spider stat logging. But I wouldn't feel comfortable implementing that myself...
Posted: October 16th, 2011, 01:19 AM

(Although I seem to remember I'd separated spiders from visitors in Noisen/Wedge.org code, but I don't remember what it was about, since the stats themselves do take bots into account...)

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Bot logging
« Reply #3, on October 16th, 2011, 01:29 AM »
Quote
But personally, I would certainly enable such an option. As long as one is aware that less hits doesn't translate to less visitors...
And by the same token, more hits doesn't mean more visitors either.
Quote
IIRC, when I used to use GA on noisen.com, my stats over there were MUCH lower, something like 5 times less than SMF's own account.
Yup, that's about right.
Quote
Separating spider hits from the rest would be suitable as long as we have an option to disable spider stat logging. But I wouldn't feel comfortable implementing that myself...
Wouldn't we put it into the spider options?

Aaron

  • Posts: 356
Re: Bot logging
« Reply #4, on October 16th, 2011, 11:25 PM »
Quote from Nao on October 16th, 2011, 01:20 AM
Or we could simply have an option in the admin area, disabled by default, to skip bots from these logs...
Considering that there's no 100% sure way of determining whether an IP is a bot, it'd make sense to leave it up to the admin to decide what they wanna do.
That sounds like the right way to do it, yeah. :)
"The entire British Empire was built on cups of tea … and if you think I'm going to war without one, mate, you're mistaken."