Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Nao
5446
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 17th, 2012, 07:11 PM »
Quote from nend on April 17th, 2012, 06:12 AM
How about convert the string to something that can't possibly be a collision risk and file name safe. I just thought of this while writing this post and ran a test in the middle of the post, bin2hex. From test I am getting real fast speeds from this function, the only problem I forsee though is filename limits.
Hmm, I like that... bin2hex seems to be fast indeed.
I was also looking into sha1(), crypt('sha256') and pack() (there are plenty of options to convert these), or even using base_convert() to automatically use alphanumeric characters only...
But bin2hex could be just as simple :P
Although not as secure as a md5() on boardurl.filemtime...

Oh, Pete -- how about we calculate that md5 hash only ONCE per page...? It seems stupid not to do it. We could simply store it in $context['cache_hash'] or something... If empty, just fill it in...
5447
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 17th, 2012, 07:03 PM »
Quote from Arantor on April 16th, 2012, 11:33 PM
There is, $settings['settings_updated']. But the settings are also cached...
My, you always have an answer to everything... :lol:
So, in order to retrieve the cache hash, I would need to first get the cache hash and then retrieve the cache based on that cache hash... THEN I can know what the cache hash is :^^;:
Quote
I'm not saying md5 vs base64, I'm saying something vs nothing where the result is not trivially guessable by a hacker.
Hmm, I was thinking... md5() is a 128-bit number, so it's something like at least a billion billion billion billion... Thinking about it, since we can barely have more than 2000 files in a folder, it's very, very unlikely to get a collision -- ever... No?
Quote
But the base64 can still generate characters that are invalid.
I forgot about the slash being a valid base64 char. Oh, my... They should have chosen something else! It's not like there are so few valid characters for an URI...
5448
Plugins / Re: Light URL Plugin Maybe?
« on April 17th, 2012, 06:56 PM »
Quote from 0x on April 17th, 2012, 04:37 AM
WIll the shortlink can be linked to specific post? maybe something like example.com/go/thash#postnumber
/go/mHASH would redirect to /?msg=unhashed, which in turns redirects to /msgXXX/#msgXXX... (Of course I'd be skipping the ?msg step, no need to redirect twice...)
Posted: April 17th, 2012, 06:53 PM
Quote from Arantor on April 17th, 2012, 02:20 PM
domain.com/go/b#1 is shorter than domain.com/index.php?board=1 yes (but it requires rewrites and you can't get around that)
Of course, there's no reason not to provide a /?go=tHASH alternative if you have problems with htaccess files... It's just one extra char ;)
Quote
But you've all missed the point I went at great lengths to try and explain. I got the fact that it would be an automatic thing, but if you make it a plugin, you have to either side-swipe other plugins and rewrite all their URLs (on top of the code in Wedge itself), or you have to expect the plugins to support it through choice. The former requires a lot of work, and the latter just isn't going to happen.
I'd really like to have this as core... (But disabled by default, probably. I can see the interest in doing that. I just don't know if it's useful everywhere.)
Quote
Now, let's say for the sake of argument that you make it core, presumably in addition to Pretty URLs. Not only, then, do you have to spend more time testing it, it makes it harder for plugin authors to work with, because there's three schemes instead of two that they have to contend with. I don't even see authors making real use of PURLs as it is, let alone having to navigate multiple schemes of URL routing.
I don't see where there's a problem with it...?
5449
Bug reports / Re: Pretty URL remarks
« on April 17th, 2012, 06:52 PM »
Quote from nend on April 17th, 2012, 04:31 PM
So no caching?
Wedge already does topic/board name caching (I need to add category name caching as well...), which represents the single biggest performance savings you could get. There's also a mini-cache inside the replacement callback (i.e. if a URL was already converted earlier in the page, we won't even bother to re-calculate what little data needs to be re-calculated).

What I'm not convinced with, is the global cache that stores every single URL to the cache... And I already added some warnings in the admin to discourage its use. And I'm considering dropping it entirely.
Quote
On Pretty URL's by vb I have done allot of modifications to the code.
Just wanted to say... Pretty URLs is NOT vb's. Definitely not! Dannii wrote it (and I wrote some small parts of it, or more precisely, I suggested improvements to Dannii because at the time I had no experience with collaborative programming, and I never used the SVN access he gave me on Google Code). He gave it to vb after he decided to retire from it, and forgot to offer it to me first (for which he apologized, so that's okay, it wasn't against me.) vb did absolutely nothing for PURLs, save for a few minor bug fixes. In fact, the mod is available in a BSD license so he couldn't do the usual things he does with mods he inherits (i.e. resell them!)

The PURLs in Wedge is based upon the version I rewrote for Noisen many years ago. All in all, it has very little to do with the original mod, except that I kept its name for simplicity, and it still uses the same basic techniques (going through all links to store them, extracting topic IDs, querying their names, and going through all links against to replace them.)
Quote
I have reported some of what I have done to his code back to him hoping it will get implemented. Some does and some doesn't. However I figured it easier to cache url's with SMF caching instead of using the DB.
But how did you do that...? One file per URL? That would incur a lot of disc writing/reading... One file per page?
Quote
I have also removed index.php from the urls with the mod. Added one little extra rule to the source.
Can even be done with a str_replace... :)
$buffer = str_replace('"' . $scripturl . '", '"' . $boardurl . '"', $buffer);
Of course it's not the exact not I had in my code... And I'm doing it differently these days anyway (stripping the index.php directly when initializing $scripturl, which might create issues with plugins if they're not careful, so I'm still unsure I'll be doing that forever...)
Quote
Separately though I have made all old urls illegal and display a 404 page. I have found through access logs that allot of bots look for say action=, topic= or board= to identify SMF.
Hmm... But if you used to have a purl-free forum, you're basically telling Google to go away...
5450
Plugins / Re: Light URL Plugin Maybe?
« on April 16th, 2012, 11:30 PM »
Quote from Arantor on April 16th, 2012, 11:19 PM
It's an interesting concept but I'm not sure plugins will behave cleanly with it, I suspect unless it's managed pretty much automatically they won't use it.
It doesn't matter that much, does it..?
Quote
Mind you, most plugins will only use PURLs features that are managed automatically (very few plugins for SMF ever cared, and those that did, were pretty much portals and/or PURLs type plugins in the first place), so if it can be done transparently and 'just work', plugins can use it but otherwise they probably won't.
They don't have to. They simply can...
5451
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 16th, 2012, 11:28 PM »
Instead of the filemtime, I'd really (really) rather we have a $settings variable that we increment when we ask for a cache flush...
I know that most file operations are cached in PHP for some time, so a filemtime shouldn't be expensive at all, but I'd rather not count on it. I'm still considering adding an option to disable file date checking for CSS and JS files, which I'm sure would save more than a few milliseconds on each page load...

I don't really see security being any better if using md5() over base64. What can happen? Someone trying to repeatedly run a check on a URL by decreasing the time from now to a week ago or something...? It's tantamount to brute-forcing a md5() as well, if you ask me...

In any case, I don't see the need for storing $boardurl in the file name. And the base64 encoding should replace the strtr, at least that is proven to be as fast, or faster.
5452
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 16th, 2012, 09:23 PM »
So, just keeping an unencoded filemtime at the start, the rest should be fine...?
5453
Plugins / Re: Light URL Plugin Maybe?
« on April 16th, 2012, 09:21 PM »
Pete what's your opinion on the go thing then?
5454
Plugins / Re: Light URL Plugin Maybe?
« on April 16th, 2012, 08:31 PM »
Keep cool, Pete :P
5455
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 16th, 2012, 07:57 PM »
Quote from Dragooon on April 16th, 2012, 07:22 PM
I'm fairly sure SMG had no cache when I made it...or did it?
What I know is that I had absolutely no knowledge of SMF2 caching until I started work on Wedge ;)
Well, I knew it existed (and was my prime reason to use SMF2), but I'd never looked into the code, it was too esoteric for me.

One could say, that basically when you were 13, you were way more knowledgeable about SMF programming than I was. And I'm 20 years older than you :P
5456
Features / Re: New revs
« on April 16th, 2012, 07:53 PM »
rev 1550
(7 files, 7kb -- and PLENTY of potential bugs for you guys to find :P)

* Extensively rewrote Pretty URLs to be easier to tweak (not that you can actually easily add new keywords to transform right now, but it will come...) Basically, you no longer need to provide two pattern strings, the search pattern will be reused for replacements as well. The regex is easier to grasp (even though it uses a lookahead assertion... Learn to love it.) There are WAY too many changes to document them all, just trust me that overall my optimizations made the entire process about twice faster. Your server will thank you (and me) for that. Hopefully didn't break any of the patterns with my rewrite. (Subs-Template.php, Subs.php, PrettyUrls-Filters.php, Ajax.php, Feed.php, Aeva-Foxy.php)

* A few speed optimizations for the topic filter, and general optimizations based on the use of references. I know, one is not supposed to use references for optimizations... Except when it makes it three times faster, right? (PrettyUrls-Filters.php)

* Moved session ID injection code a bit further down the output buffer rewriter, to catch as many URLs as possible. Untested. (Subs-Template.php)

* Moved category URL replacement to the board filter, rather than the action filter -- makes more sense. (PrettyUrls-Filters.php)

+ Added support for /do/action even inside a topic or board. (PrettyUrls-Filters.php)

! Feeds were sending session IDs to feed fetchers. Baaaaad! (Feed.php, Subs-Template.php)

! Last-minute typo in error 404 logging... A classic of mine. (QueryString.php)
5457
Bug reports / Re: Pretty URL remarks
« on April 16th, 2012, 06:47 PM »
Done here as you can see.

Adding new URL schemes through hooks is no longer possible with my current code, though... But hopefully it'll be (nearly) doable once I commit. I'd like to clean up my code now, and commit something for a change... ;)
5458
Archived fixes / Re: SMF bug 4956 (slash in cache key causes cache to fail)
« on April 16th, 2012, 06:45 PM »
As I didn't read the original report, I didn't even know there was a problem with AeMe...
Blame any Aeva cache issues on Dragooon, he did it all :niark:

There's also Subs-Sounds.php that clearly uses slashes in the key...

I'll still cast my vote on base64_encode (alone), based on what was discussed.

Now... I'm looking into the actual caching code, and here's the thing.

Code: [Select]
$key = md5($boardurl . filemtime($sourcedir . '/Collapse.php')) . '-Wedge-' . strtr($key, ':', '-');

It's ALREADY doing both a md5 (on the prefix though!), and on the key (to replace ":").
So it can easily be turned into strtr($key, ':/', '--') for speed reasons...
However, it would make more sense to 'simply' put everything into the md5() call. I see no reason not to do so...
Also, I'm not exactly sure why it absolutely needs to retrieve filemtime here, is it for security reasons? (i.e. not allowing a user to find the correct URL for the cache...)
But we *already* have an htaccess file that prevents directly viewing php files...

(Plus, I've always been bothered by this 'touch' on Collapse.php -- previously on Load.php in SMF which is a VERY bad idea, because that file is critical and often modified...)

Suggesting this as replacement, then...

Code: [Select]
$key = base64_encode($key);

And voilà, problem solved... And supercharged cache ;)
5459
The Pub / Re: The Cookie Law (in the UK at least)
« on April 16th, 2012, 05:15 PM »
Quote from PantsManUK on April 16th, 2012, 05:12 PM
Well, you Frenchies have strange data protection laws as it is :eheh:
But they're not applied, as such... So it's just there for everyone to laugh.
5460
The Pub / Re: The Cookie Law (in the UK at least)
« on April 16th, 2012, 03:13 PM »
Quote from Arantor on April 16th, 2012, 01:29 PM
You mean I'd have to run SMF instead of running Wedge on my sites (since I'd be banned from using it)?
Yes! Because the British have always loved nonsensical humour haven't they? :P
Posted: April 16th, 2012, 03:12 PM
Quote from PantsManUK on April 16th, 2012, 01:51 PM
Except it's an EU directive, so all of y'all will be coerced into enacting it eventually, the UK just happened to have done it "early".
I guess it makes sense that it is -- except that I've never even heard about it being planned to be done in France...