Wedge

Public area => The Pub => Bug reports => Topic started by: txcas on March 14th, 2014, 03:54 PM

Title: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 14th, 2014, 03:54 PM
I have about 40 pages worth of these errors for just one user that uploaded an avatar to the forum.

(http://www.carloss.us/pictures/kalindark.png)
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 15th, 2014, 12:20 AM
Is it always for the same link..? Attachment #497?
Can it be reproduced with the Android stock browser?
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 15th, 2014, 03:48 AM
Always /do/dlattach/?attach=492;type=avatar.  I don't have an Android device to test.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 15th, 2014, 09:35 AM
Ideally, to fix this, we should, in order:

- Be able to reproduce the error message... I went to your post (and even to the attachment download link) with my Chrome Mobile, and saw no such error.
- And then disable CloudFlare to see if it's a proxy problem.

It's all about the 'Accept' header not being in the request, but I don't know if it's ACTUALLY due to a bot or not.

Also, I noticed in your accesss log that many apparently valid Bing requests are blocked. Not that I give much of a damn about Bing (AFAIK, only Windows Mobile users have to deal with this), but it's still something of a worry...

http://www.bing.com/webmaster/help/how-to-verify-bingbot-3905dc26

I may have to disable that rule entirely. Thanks to Microsoft's laziness, it looks like I may have to let some bots go through the process. -_-
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 15th, 2014, 09:46 AM
It's interesting to note that...

- These errors ONLY come from kalindark,
- And they're ONLY triggered on image requests: any avatars, and any media thumbnails, too. (So, Aeva Media and the attachment system are out of suspicion; it's really related to the MIME type itself.)

Did you do anything special to your server configuration for image files..?

And I just tested with Android's stock browser, also on Android 4.1.2 like kalindark, and have no such error, really.
Ask kalindark if he's using a customized version of Android, and if he's using an anti-virus on it.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 17th, 2014, 07:15 PM
Bump...
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 18th, 2014, 03:42 PM
I will ask the use and get you more info.
Posted: March 18th, 2014, 03:30 PM
Quote from Nao on March 15th, 2014, 09:35 AM
Also, I noticed in your accesss log that many apparently valid Bing requests are blocked. Not that I give much of a damn about Bing (AFAIK, only Windows Mobile users have to deal with this), but it's still something of a worry...

http://www.bing.com/webmaster/help/how-to-verify-bingbot-3905dc26

I may have to disable that rule entirely. Thanks to Microsoft's laziness, it looks like I may have to let some bots go through the process. -_-
Yes I noticed that Google can parse everything but Bing can only see 2 pages.  It looks like Baidu and Yahoo are also getting blocked.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 18th, 2014, 05:07 PM
This is what I for back from the user:

I have a Casio GZ one, droid.  My avatar was downloaded from my old droid incredible.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 18th, 2014, 06:30 PM
Weird. The wikipedia page for that device says it runs Android 2.3, not 4.1.2... :-/

Anyway, it's a very obscure phone. I'd be tempted to say it's badly customized.
Still, dunno what to do... Dropping the Accept rule could let more spammers go.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 18th, 2014, 06:44 PM
'kay, I've found this...
http://honeybeenet.altervista.org/beefree/?id=111101
Although this is obviously written by a lunatic, he makes a good point: the HTTP protocol standards do say that the Accept header is not mandatory.
Thus, any software could technically remove them from your browser's requests. And block you out of a website without you getting a clue as to why.

I'm tempted to remove all of the Accept test code, then...
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 18th, 2014, 08:06 PM
From the information on that link it looks like Bad Behavior causes more problems than the ones it prevents.  Is there an option for the forum admin to turn it off?
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: MultiformeIngegno on March 19th, 2014, 09:09 AM
Quote from Nao on March 18th, 2014, 06:44 PM
I'm tempted to remove all of the Accept test code, then...
If it's not mandatory shouldn't be a rule
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 21st, 2014, 03:20 PM
Guys, what do you think..? Do you prefer to block a few legitimate users with strange browsers, or let spam bots & script kiddies visit your forum...?
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: txcas on March 22nd, 2014, 01:11 AM
I am sure we all want the bots and spammers blocked, but in a reliable fashion. I am not sure Bad Behavior is that good, but at the same time I don't think there are many choices.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 22nd, 2014, 05:15 PM
I am not a specialist. I don't know why Pete chose to include Bad Behavior into Wedge, and I don't know why he chose to include it inside our code, rather than as a pluggable library with a switch. Anyway...

Anyone else? I need opinions on this.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: godboko71 on March 23rd, 2014, 05:38 AM
Switches are good. That said I wonder how many "real" users are affected verses spammers.
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on March 26th, 2014, 07:43 PM
Just FYI, the 'Accept' rule accounts for 70% of my BadBehavior log here at Wedge.
Most of them seem to be from self-declared bots.

I really don't know what to do about that...
Title: Re: Intrusion log: Request did not contain 'Accept' header.
Post by: Nao on June 15th, 2014, 01:06 AM
Bump!