txcas

  • Bug Zapper
  • Posts: 202

Nao

  • Dadman with a boy
  • Posts: 16,079

txcas

  • Bug Zapper
  • Posts: 202

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #3, on March 15th, 2014, 09:35 AM »
Ideally, to fix this, we should, in order:

- Be able to reproduce the error message... I went to your post (and even to the attachment download link) with my Chrome Mobile, and saw no such error.
- And then disable CloudFlare to see if it's a proxy problem.

It's all about the 'Accept' header not being in the request, but I don't know if it's ACTUALLY due to a bot or not.

Also, I noticed in your accesss log that many apparently valid Bing requests are blocked. Not that I give much of a damn about Bing (AFAIK, only Windows Mobile users have to deal with this), but it's still something of a worry...

http://www.bing.com/webmaster/help/how-to-verify-bingbot-3905dc26

I may have to disable that rule entirely. Thanks to Microsoft's laziness, it looks like I may have to let some bots go through the process. -_-
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #4, on March 15th, 2014, 09:46 AM »
It's interesting to note that...

- These errors ONLY come from kalindark,
- And they're ONLY triggered on image requests: any avatars, and any media thumbnails, too. (So, Aeva Media and the attachment system are out of suspicion; it's really related to the MIME type itself.)

Did you do anything special to your server configuration for image files..?

And I just tested with Android's stock browser, also on Android 4.1.2 like kalindark, and have no such error, really.
Ask kalindark if he's using a customized version of Android, and if he's using an anti-virus on it.

txcas

  • Bug Zapper
  • Posts: 202
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #6, on March 18th, 2014, 03:42 PM »
I will ask the use and get you more info.
Posted: March 18th, 2014, 03:30 PM
Quote from Nao on March 15th, 2014, 09:35 AM
Also, I noticed in your accesss log that many apparently valid Bing requests are blocked. Not that I give much of a damn about Bing (AFAIK, only Windows Mobile users have to deal with this), but it's still something of a worry...

http://www.bing.com/webmaster/help/how-to-verify-bingbot-3905dc26

I may have to disable that rule entirely. Thanks to Microsoft's laziness, it looks like I may have to let some bots go through the process. -_-
Yes I noticed that Google can parse everything but Bing can only see 2 pages.  It looks like Baidu and Yahoo are also getting blocked.

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #8, on March 18th, 2014, 06:30 PM »
Weird. The wikipedia page for that device says it runs Android 2.3, not 4.1.2... :-/

Anyway, it's a very obscure phone. I'd be tempted to say it's badly customized.
Still, dunno what to do... Dropping the Accept rule could let more spammers go.
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #9, on March 18th, 2014, 06:44 PM »
'kay, I've found this...
http://honeybeenet.altervista.org/beefree/?id=111101
Although this is obviously written by a lunatic, he makes a good point: the HTTP protocol standards do say that the Accept header is not mandatory.
Thus, any software could technically remove them from your browser's requests. And block you out of a website without you getting a clue as to why.

I'm tempted to remove all of the Accept test code, then...

txcas

  • Bug Zapper
  • Posts: 202

MultiformeIngegno

  • Posts: 1,337

Nao

  • Dadman with a boy
  • Posts: 16,079

txcas

  • Bug Zapper
  • Posts: 202

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Intrusion log: Request did not contain 'Accept' header.
« Reply #14, on March 22nd, 2014, 05:15 PM »
I am not a specialist. I don't know why Pete chose to include Bad Behavior into Wedge, and I don't know why he chose to include it inside our code, rather than as a pluggable library with a switch. Anyway...

Anyone else? I need opinions on this.