Pretty URL remarks

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Pretty URL remarks
« Reply #45, on April 27th, 2012, 04:48 PM »
Quote from Arantor on April 27th, 2012, 04:40 PM
I'm sorry, I'm not prepared to accept being insecure by default just because people are too lazy to do some work themselves occasionally.
Why would your site be insecure just because there's an option to update the files...?

I mean, if we have such a system in, we'd simply warn any admin that an update is ready, then they go to the admin area, click to update, we pass along the session IDs etc, and the script goes on to import the file, extract it in a temp folder (all of this is very safe...), and then either we ask the user to use FTP to move the folder to the root (i.e. automatically replace all files), or we can offer to do it for them... (Which you wouldn't do, I guess.)
Quote
Here's the thing: other platforms don't have one-click upgrades, and they're absolutely happy with this for exactly the same reason I am.
And Wedge isn't exactly a platform that's like all others... :P
I'd like for it to require as little attention as possible from the admin point of view. SMF upgrades are so painful to me that I simply don't do the upgrades and leave my installs opened to security holes (well, I do try and fix them manually though...)

I'd really like for Wedge to be able to say that "our installed base has zero security holes"...

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #46, on April 27th, 2012, 04:55 PM »
Quote
Why would your site be insecure just because there's an option to update the files...?
Because just about every system ever designed to do that has to make the files writable to the webserver. Which means making everything 777. Or you fuck around with creating a manual S/FTP connection, unpacking the update file by file and uploading it via S/FTP that way. While the latter is feasible for plugins, it simply isn't feasible for large-scale updates. And that still assumes the system is actually running FTP or SFTP, systems like IIS won't be. (And believe me, getting it running properly on there is a nightmare)

So if you don't fuck about with S/FTP, you tell people how to chmod or chown (and just look at how many support topics there are on SMF for this) their files, and once it works, they're not going to put it back to secure afterwards.
Quote
I'd really like for Wedge to be able to say that "our installed base has zero security holes"...
Isn't going to happen. You'll always have people who won't upgrade because they've made custom changes they won't upgrade for, or because they think that their plugins won't work.

But, it's interesting to note that the systems that have one-click updates are more routinely victims of server abuse through files being infiltrated (because of poor permissions setup) than those that don't.

Even putting aside these factors, you're left with an update system that makes the files insecure by default, or an update system that is secure by default but requires users to do some work to keep it up to date. I'd certainly prefer the latter, especially as updates should just be a case of uploading a new set of files, nothing more.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Pretty URL remarks
« Reply #47, on April 27th, 2012, 05:20 PM »
<sigh>

Then no automatic updates for me...

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #48, on April 27th, 2012, 05:28 PM »
Did you ever do automatic updates for SMF?

How many people are screwed because automatic updates don't work for them? (The number of 1.1.11 updates that failed was pretty vicious, btw)


Note that if someone can show me a robust method for doing it that doesn't require making things insecure by default, I'm more than willing to entertain the idea. But the sheer mechanics of doing it considering the usual mess that is hosts + permissions means I'd rather avoid it entirely, just because that's actually better for the user.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Pretty URL remarks
« Reply #49, on April 27th, 2012, 05:31 PM »
Nope, never did...
But I did do automatic updates for AeMe sitelists, and guess what, never had a single problem with 'em... :P

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #50, on April 27th, 2012, 05:31 PM »
Do you know why you never had a problem with them?

Nao

  • Dadman with a boy
  • Posts: 16,082

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #52, on April 27th, 2012, 05:41 PM »
Sadly, it wasn't because you rock, it's because SMF was creating the file by the webserver user itself, meaning that it was implicitly writable by the webserver, and indeed by any other user on the server, which is really no better than the file being owned by the proper account holder but being 777.

The whole point of faffing with FTP/SFTP the way I mention is so that the webserver owner explicitly does NOT own the files, but the FTP user account does (i.e. the user's own account). That way you leave them at the standard 644 and they're not writable, and thus not at risk from tainting.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Pretty URL remarks
« Reply #53, on April 27th, 2012, 05:44 PM »
Well... I'm not 100% sure it's true.
Aeva-Sites.php is indeed generated through SMF, but Subs-Aeva-Sites.php or whatever was included in the package so it wouldn..... Oh, wait, the package is usually installed by SMF. Which would explain...

Well, how 'bout we make Wedge into a package, too? :lol:
I mean -- we ship with a gzip file and an index.php file, and we run the index file to decompress the gzip and install the files with a webserver user ID...

(Of course it's also opening the door to issues when people start FTPing updated files. Hmm.)

Well, I guess FTP'ing the whole stuff is the only solution then. What's the issue with security? Why not use your SFTP component if available?

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #54, on April 27th, 2012, 05:53 PM »
Quote
Well, I guess FTP'ing the whole stuff is the only solution then. What's the issue with security? Why not use your SFTP component if available?
Because it's still the same basic problem, regardless of whether you use FTP or SFTP.

Scenario 1: the www-data user owns the files. They're writable from Wedge. They're also writable by definition from ANYTHING ELSE that's web-based on the server. That means any rogue process can infect any Wedge file or plugin file with code. This, really, is the risk vector that we're talking about.

Scenario 2: the proper user account owns the files. Then they make everything 777 (and experience says they won't put them back to 644), so Wedge can write the files. Same problem: they're still writable by any other code that's executed via PHP from web requests, which will all be run as www-data.

Scenario 3: we do it as I'm thinking, with FTP. That means we upload it to the server or otherwise obtain it, it'll be in a temporary folder somewhere. Exactly where is irrelevant, because however we get it, the same thing has to occur: it goes to a temporary place. Here's where it gets complicated: whatever we do, we really then have to get into unzipping that file. [1] Whatever we do, we've unpacked the file and we then have to upload it file by file over FTP. For any sizeable package, this is not going to be a quick process, especially if we're talking about a major upgrade. It's likely to then fall foul of the 30 second timeout, amongst other things.


These are the alternatives as I see them, and I don't like any of them that much :/
 1. We end up dumping it all into system-wide temporary folder, where it's also at risk, though there's no guarantee the files will actually be left available over any period of time. It's really a side concern though.

nend

  • When is a theme, no longer what it was when installed?
  • Posts: 165
Re: Pretty URL remarks
« Reply #55, on April 27th, 2012, 07:19 PM »
Quote from Nao on April 26th, 2012, 05:25 PM
Quote from nend on April 17th, 2012, 08:27 PM
On SMF caching I cache 1 file per page, it would be too much of a waste 1 file per URL. I figured there are only going to be a few pages hit often and allot of pages not so often, so I believe it will even itself out using this method.
I don't know. I'm not sure this is for the best... We're talking about a feature that takes on average a hundredth of a second to execute (it's about half the time of the entire ob_sessrewrite execution time IIRC), perhaps two at the most, and caching this would logically save even less time... Then again, most of the cache calls in Wedge/SMF are done to save a few milliseconds only... But they usually don't have to target a wide range of URLs or anything.
So your saying caching the data takes more time then if I processed all the urls in a page?

I wasn't too sure which would be faster when I made the modification to do file caching instead of db. I know the file caching has to be faster than the db, not only faster but you don't have to worry about the DB being filled with junk that never purges. Both can be turned off easily anyways.

However I know you and Arantor, always bickering over performance and slight gains, which isn't a bad thing. So I am pretty sure your information is correct. Thanks though for the information, I think I am going to disable caching completely in that area.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Pretty URL remarks
« Reply #56, on April 27th, 2012, 07:22 PM »
It's not a simple x is faster than y argument, either. If you have stuff in the DB that's being called frequently, assuming you don't have a stupid DB layout and bad queries, there's a reasonable chance that you might be able to get some of that stuff out of the query cache, and if the table isn't too huge, you might even get it entirely into memory anyway, and RAM is faster than HD for things like that ;)

nend

  • When is a theme, no longer what it was when installed?
  • Posts: 165
Re: Pretty URL remarks
« Reply #57, on April 28th, 2012, 05:21 PM »
Quote from Arantor on April 27th, 2012, 07:22 PM
It's not a simple x is faster than y argument, either. If you have stuff in the DB that's being called frequently, assuming you don't have a stupid DB layout and bad queries, there's a reasonable chance that you might be able to get some of that stuff out of the query cache, and if the table isn't too huge, you might even get it entirely into memory anyway, and RAM is faster than HD for things like that ;)
The method in the VB's script is not DB friendly. The URL DB cache for most people out grows everything else in the DB. Before I noticed it, the URL DB cache was taking up at least 3/4 or 75% of my DB. That is when I suggested to VB to use SMF's cache and also give the option to disable, because that is better than the DB option IMHO. Plus who wants 75% of their DB just used for Pretty URL's.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Pretty URL remarks
« Reply #58, on May 1st, 2012, 12:55 PM »
Quote from nend on April 28th, 2012, 05:21 PM
The method in the VB's script is not DB friendly.
Just wanna say... Again, it's not vb's script. It's Dannii's. vb did pretty much nothing on it. All he was trying to do is accumulate mods in his ownership and use them to advertise his paid-for mods. Bad karma.
Quote
The URL DB cache for most people out grows everything else in the DB. Before I noticed it, the URL DB cache was taking up at least 3/4 or 75% of my DB. That is when I suggested to VB to use SMF's cache and also give the option to disable, because that is better than the DB option IMHO. Plus who wants 75% of their DB just used for Pretty URL's.
No one.
Any volunteers to benchmark Wedge in DB/non-DB Pretty URLs modes?
Shouldn't be too complicated to do... But I have no time myself, I'm afraid. I'm busy IRL (most today, tomorrow should be better for me), and even then I'm struggling to finish my newest commit which is going to be huge because I finished many of my 'works in progress' yesterday night (maybe after all, it's a good thing I had my Internet down for so long because at least I was able to focus :lol:), anyway, it's going to take some time to document it and everything...

Either way, I think there's a 90% chance that I'll be removing the DB feature and replacing it with nothing ;)
Re: Pretty URL remarks
« Reply #59, on May 1st, 2012, 01:17 PM »
Re package auto update -- I really dont think it's unfeasable. But it'll require some thinking Indeed.