« Humor
Baidu bloody loves me.
An Ultimate Forum »

Pandos

  • Living on the edge of Wedge
  • Posts: 635
Re: Baidu bloody loves me.
« Reply #15, on September 9th, 2011, 11:00 AM »
Why not stopping Baidu by iptables?

 iptables -I INPUT -s  119.63.192.0/24  -j DROP
# dpkg-reconfigure brain
error: brain is not installed or configured

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Baidu bloody loves me.
« Reply #16, on September 9th, 2011, 11:01 AM »
Because not everyone has control of iptables (though I think Nao has access to it here)

But you can definitely exclude them by examining the HTTP request.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Baidu bloody loves me.
« Reply #17, on September 9th, 2011, 01:54 PM »
I may have access to iptables, but... Why exactly would I stop Baidu? As long as it's not killing the bandwidth, I could care less about it...

If you don't like seeing it in the who's online area, I can remove it from the list of recognized bots... :P

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Baidu bloody loves me.
« Reply #18, on September 9th, 2011, 02:07 PM »
That's the thing: it consumes 8-10x more bandwidth than any other search engine, and it's going to keep going.

Norodo

  • Oh you Baidu, so randumb. (60 sites being indexed at once? Jeez)
  • Posts: 469
Re: Baidu bloody loves me.
« Reply #19, on September 9th, 2011, 02:18 PM »
That and for people outside China it's just not a bot that is needed. We have -much- better search engines.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Baidu bloody loves me.
« Reply #20, on September 14th, 2011, 02:19 PM »
In other news, now it's viewing my profile 3 times over.

Dismal Shadow

  • Madman in a Box
  • Me: Who is Arantor? Cleverbot: It stands for time and relative dimensions in space.
  • Posts: 1,185
“I will stand on my ground as an atheist until your god shows up...If my irreligious bothers you much, and if you think everything I do is heresy to your god I don't care. Heresy is for those who believe, I don't. So, it isn't heresy at all!


   Jack in, Wedge,
   EXECUTE!

PantsManUK

  • [me=PantsManUK]would dearly love to dump SMF 1.X at this juncture...[/me]
  • Posts: 174
Re: Baidu bloody loves me.
« Reply #22, on September 15th, 2011, 02:23 PM »
Maybe they're getting better; I told Baidu to bugger off in my robots.txt and (so far) they've not returned (finger crossed, touch wood...)
« What is this thing you hoomans call "Facebook"? »

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Baidu bloody loves me.
« Reply #23, on September 15th, 2011, 02:25 PM »
Quote from PantsManUK on September 15th, 2011, 02:23 PM
Maybe they're getting better; I told Baidu to bugger off in my robots.txt and (so far) they've not returned (finger crossed, touch wood...)
Wait, Baidu LISTENED TO YOU? Colour me surprised.

busterone

  • Posts: 12
Re: Baidu bloody loves me.
« Reply #24, on September 16th, 2011, 04:26 AM »
They have never honored my robots.txt.  I was sometimes getting 40 to 50 of them at a time. The damn little roaches were everywhere at once.  I don't have access to iptables, so I just banned them in .htaccess about 6 months or so ago.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Baidu bloody loves me.
« Reply #25, on September 16th, 2011, 07:45 AM »
Maybe it requires a specific format in the robots.txt...? Like, Baidu has to be written in hanzi? :P
« Humor
An Ultimate Forum »