Wedge
Public area => Features => The Pub => Features: Theming => Topic started by: Nao on May 10th, 2011, 06:43 PM
-
Feature: JavaScript caching
Developer: Nao
Target: everyone
Status: 100% (believed to be complete.)
Comment:
Right after minifying your CSS and JS files, Wedge will cache them and gzip them. The good point is that it's smart enough to cache multiple files together, relieving the server a bit.
I'll use my script.js example again. At the time of writing, the regular script.js/theme.js duo is about 50KB in SMF2 (split over two files), less than 30KB in Wedge (with many extra functions in them), and the actual stuff you download is a single 6KB file, after minification and gzipping. Meanwhile in the wasteland, SMF2 still sends you 50KB of data. Because everyone knows all end-users absolutely need to have comments in the JavaScript code they execute.
-
Why not minify and cache the single built file, rather than minifying separately and then combining and caching? You'll probably get better perf.
Honestly, this is why I didn't use comments in the js files of olde, I figured it was way too much to run something on them to remove comments. But, it's getting more popular now and making sense.
-[Unknown]
-
link=msg=262640 date=1308472459]
Why not minify and cache the single built file, rather than minifying separately and then combining and caching? You'll probably get better perf.
Also, performance is not very important at the minification stage. Indeed, since everything is cached forever as long as the css files aren't modified, after a few hundred visits (encompassing all possible browser setups), the minifier will no longer be called at all.Honestly, this is why I didn't use comments in the js files of olde, I figured it was way too much to run something on them to remove comments. But, it's getting more popular now and making sense.
Commenting JS files should be mandatory. If only to help future developers get into the groove :)
-
I was just going over some old notes and saw this: http://www.simplemachines.org/community/index.php?topic=391132.msg2981869#msg2981869 and then I went here and saw this.
-
Doesn't show up to guests (banned, remember? I tend not to go over there too often these days......)
Please copy & paste?
-
Well ... I am not banned there (yet :eheh: ) but I cannot see it either.
-
An Error Has Occurred!
The topic or board you are looking for appears to be either missing or off limits to you.
-
I can't see it either, but I would probably have to re-activate my account to do so.
-
You killed your account?
-
Why would I keep it? It's only another tie to the past I can do without and it means I find myself spending much less time browsing over there for stuff to laugh at - and I don't get dragged into stupid arguments with narrow-minded fools who don't really understand what's going on, and I was fed up of being labelled as a bringer of drama without due cause.
I don't need to go there, overall, except to browse for ideas for things to build for Wedge in the future, because I can. Oh, and it means that they're left to their own devices as far as security is concerned, after I called them out on a mod that allows for arbitrary iframe injection. I burned far far too much time there.
-
Why would I keep it? It's only another tie to the past I can do without and it means I find myself spending much less time browsing over there for stuff to laugh at - and I don't get dragged into stupid arguments with narrow-minded fools who don't really understand what's going on, and I was fed up of being labelled as a bringer of drama without due cause.
Of course I haven't been (much) to sm.org after that, myself... So I could hardly have noticed anything.
Still, it's a shock to see you aren't in the memberlist.
(And as always, the SMF team has yet to take your request into account. Your account can still be browsed.)I don't need to go there, overall, except to browse for ideas for things to build for Wedge in the future, because I can. Oh, and it means that they're left to their own devices as far as security is concerned,
after I called them out on a mod that allows for arbitrary iframe injection. I burned far far too much time there.
-
The usual rule is 30 days, so it's no surprise. Though I thought they'd be glad to see the back of me to be honest.
-
They're seeing your back in every sense of the term!
-
What does that have to do with this function?
It's used for the following to remove bad URLs from the indexers view. Well, if we just changed them to real, good URLs and then worked with those instead, that would make more sense and more understandable for everyone IMO.
It's not even in use everywhere. I took a look at the message index for this board:
result:
<span style="font-weight: bold;" onclick="expandPages(this, 'http://www.simplemachines.org/community/index.php?board=74.%1$d', 150, 2100, 50);" onmouseover="this.style.cursor='pointer';"> ... </span>
So, if it isn't being used there with an odd URL, shouldn't it be presenting errors or odd accesses in the logs?
It's used several times on the infocenter toggle object on the board index to add quotes around the $txt and session vars.
The following is a shining example from the display template. It is so much harder to read because of the escaping:
sTemplateBodyEdit: '\n\t\t\t\t\t\t\t\t<div id="quick_edit_body_container" style="width: 90%">\n\t\t\t\t\t\t\t\t\t<div id="error_box" style="padding: 4px;" class="error"><' + '/div>\n\t\t\t\t\t\t\t\t\t<textarea class="editor" name="message" rows="12" style="width: 100%; margin-bottom: 10px;" tabindex="5">%body%<' + '/textarea><br />\n\t\t\t\t\t\t\t\t\t<input type="hidden" name="af00a1ba031" value="c5e7963677f5827d8476125917ac76b2" />\n\t\t\t\t\t\t\t\t\t<input type="hidden" name="topic" value="382883" />\n\t\t\t\t\t\t\t\t\t<input type="hidden" name="msg" value="%msg_id%" />\n\t\t\t\t\t\t\t\t\t<div class="righttext">\n\t\t\t\t\t\t\t\t\t\t<input type="submit" name="post" value="Save" tabindex="6" onclick="return oQuickModify.modifySave(\'c5e7963677f5827d8476125917ac76b2\', \'af00a1ba031\');" accesskey="s" class="button_submit" /> <input type="button" value="Spell Check" tabindex="7" onclick="spellCheck(\'quickModForm\', \'message\');" class="button_submit" /> <input type="submit" name="cancel" value="Cancel" tabindex="8" onclick="return oQuickModify.modifyCancel();" class="button_submit" />\n\t\t\t\t\t\t\t\t\t<' + '/div>\n\t\t\t\t\t\t\t\t<' + '/div>',
I finally found one that supports what everyone is saying:
sTemplateSubjectNormal: ', JavaScriptEscape('<a href="' . $scripturl . '?topic=' . $context['current_topic'] . '.msg%msg_id%#msg%msg_id%" rel="nofollow">%subject%</a>'), ',
sTemplateSubjectNormal: '<a hr'+'ef="http://www.simplemachines.org/community/index.php'+'?topic=382883.msg%msg_id%#msg%msg_id%" rel="nofollow">%subject%<' + '/a>'
How, you might ask, do I propose we fix that? Well, there are two (or even three) options.
One means that a normal, simple link is no longer created in the template. That makes the non-cached JavaScript a lot smaller.
Modify QuickModify() returning that link with the correct information. Makes a whole lot more sense that way.
Second would be to use the JavaScript var: smf_scripturl.
sTemplateSubjectNormal: \'<a href="\' + smf_scripturl \'?topic=', $context['current_topic'], '.msg%msg_id%#msg%msg_id%" rel="nofollow">%subject%</a>\'',
The last would be a URL creator function (pseudo since I'm using the browser to write all of this):
function url(base_uri = smf_scripturl, query_string, args)
{
// see sprintf(): http://www.diveintojavascript.com/projects/javascript-sprintf
return sprintf();
// there are also easier replace().
}
Any way you look at that, it actually makes it easier for you to rewrite the URLs. It could still be possible to use a JS escaping function, but we should really figure out what we want to do with it and how it makes the user safer.
-
Oh, I remember that discussion now. It was about what JavaScriptEscape did, and why it was necessary to call it for URLs - because there were all kinds of bad URLs getting into Google (namely the one indicated with %msg_id% in it)
-
I just realised that can get be banned.
-
Dragoon, although I wish you would have asked me before quoting that, it is my post and I don't mind (for that one).
Arantor, yeah, that one.
-
I originally minified everything together, but performance was not any better. Probably a bit worse IIRC. Plus, it had other issues linked to it. Once I started doing files separately, it went better.
Also, performance is not very important at the minification stage.
Commenting JS files should be mandatory. If only to help future developers get into the groove :)
Oh, I remember that discussion now. It was about what JavaScriptEscape did, and why it was necessary to call it for URLs - because there were all kinds of bad URLs getting into Google (namely the one indicated with %msg_id% in it)
Also AFAIK Google appears to often aggressively detect URLs, so even escaping the href probably does no good (at least in my usages it didn't.)
Last of all, \t can be cleanly represented in js and \n can be escaped as in:
var x = '\
<div>\
Hi\
</div>';
So if JavaScriptEscape is kept, it's probably better to just escape the newline rather than making it \n, if concerned about readability. Probably gzips better.
Of possible interest to this topic, I've been briefly experimenting with a plugin to TOX-G that adds minification to inline js/css (as long as it doesn't have any ifs/etc. in it) and minifies some parts of the HTML (all of this done in the cached template, so runtime speed is not affected.) Initial impressions seem like it's mostly a waste of time (at least in my inline script tags, which are not common), but offers a small benefit. I haven't been messing with it much since.
-[Unknown]
-
I think minification is more about saving space in the cache. Especially for mobile devices.
-
I think minification is more about saving space in the cache. Especially for mobile devices.
One of the "good" reasons for using inline script is when it's only used on the one page, it's an uncommon page, and you want to reduce the HTTPs even further, especially if the script is small. But lowering the network consumption is a good thing, because if I can reduce the whole page by 20% (which I can't, more like 8% it seems from a standard HTML page even with some embedded script), without any additional CPU cost, that can potentially be a good savings both for user and the network operator who has to be $5/megabit or whatever.
I've become more concerned about these issues as more and more US companies have been limiting their users' bandwidth, which I hate.
-[Unknown]
-
link=msg=263516 date=1309749092]
Hmm, this is not the same as my experience, odd. I just combine with newlines between the files, which generally solves all the problems I can think of (some people don't add newlines, which means Windows-style files with a comment at the end or a missing semicolon can have issues.)
Sure, unless you're debugging/developing.
If you're specifically testing the minification process, then you probably won't need very high performance -- if it works, no need to re-parse.Well, with minification, yes. Although, I see the JS has changed markedly since I last touched it. I've mentioned before my dislike of hungarian type notation.
Also, sometimes it can be helpful to know what the *intended* purpose of a variable is. I know it has helped me... Don't forget that our code needs to be readable by future devs!Ah, I've seen this too. I think the best way (and the way I use) is to just use a common base url variable.
I think it's better to json encode values (and not have to deal with the quotes and escaping and such) than this JavaScriptEscape business.
I don't even remember seeing a ready-made function for that in ECMAScript...Note that json_encode escapes / as \/ to avoid the </script> issue, which is cleaner and more obvious than '<' + '/'.
Also AFAIK Google appears to often aggressively detect URLs, so even escaping the href probably does no good (at least in my usages it didn't.)
I can find a middle point by turning href into hr\ef... (Seems to work, at least.)Last of all, \t can be cleanly represented in js
and \n can be escaped as in:
var x = '\
<div>\
Hi\
</div>';
So if JavaScriptEscape is kept, it's probably better to just escape the newline rather than making it \n, if concerned about readability. Probably gzips better.
Ah, would be swell if Apache added support for 7z or even RAR... :lol:Of possible interest to this topic, I've been briefly experimenting with a plugin to TOX-G that adds minification to inline js/css (as long as it doesn't have any ifs/etc. in it) and minifies some parts of the HTML (all of this done in the cached template, so runtime speed is not affected.) Initial impressions seem like it's mostly a waste of time (at least in my inline script tags, which are not common), but offers a small benefit. I haven't been messing with it much since.
hacked into fixed yesterday.)
-
I haven't got a lot to add except...
But it would mean having to decode after that, wouldn't it...?
I don't even remember seeing a ready-made function for that in ECMAScript...
-
Ah... Yes indeed. That makes sense :lol:
I just gave it a try. It's interesting, but I don't know if it's worth replacing JSE with it.
For instance it'll always escape "/", even when we don't need it. It'll also turn tabs and newlines into \t and \n with no way to disable these. And re-deescaping them could prove complicated (I don't remember if regexp in JS supports assertions... Because without an assertion, you can hardly turn \\t into a litteral \t, it'll just turn it into \ followed with a tab...)
-
With PHP 5.3 json_encode has parameters you can pass. One of them, JSON_UNESCAPED_SLASHES, might be relevant, but it's undocumented (yay)
It should be faster, though.
-
It's actually documented but not in the json_encode help page... You have to go look into the JSON module and search for 'resources' or something like that. I found it originally via Google (duh)...
I posted *after* looking into these. At that point I already knew that none of the params (including the 5.4.0 params) could help in this situation.
-
It's actually documented but not in the json_encode help page... You have to go look into the JSON module and search for 'resources' or something like that. I found it originally via Google (duh)...
-
Just Google it ;)
I suppose strtr() isn't that slow anyway...
Tss, another day of frustration in my code. Among many other things, I wanted to set the action buttons straight in the post list. I ended up settling for the 'easy' solution -- display:table. Unfortunately I don't have Safari installed right now. Since it's not very table-friendly I'm afraid it'll fail. I tested under Firefox 5 and it doesn't work very well already... Only Opera 11 loves it, meh. And, strangely, IE9... Although it doesn't set .quickbuttons to full height (it should fill the table... Like any NORMAL table cell...!)
I'm getting very annoyed by the constant changes in my CSS. I'd really like to call it quits and commit it, but I'm scared most people will think it doesn't look as good as a stock SMF2, ah ah...
Heck, I made many changes to the visuals this week really. Even the windowbg divs have finally lost their border... (I'm not kidding. I was adamant this border would be in the final design... Until I decided it wasn't needed.)
-
I'd really like to call it quits and commit it, but I'm scared most people will think it doesn't look as good as a stock SMF2, ah ah...
-
I know, but it's there, so... I just use it. It's a matter of being consistent with the existing codebase. There would be too much to change otherwise.
Also, sometimes it can be helpful to know what the *intended* purpose of a variable is. I know it has helped me... Don't forget that our code needs to be readable by future devs!
But, even for a strictly typed language, type-based hungarian notation seems silly and wrong to me. All the more for a loosely typed one.
But, just IMHO. I know there's tons of programmers out there with different opinions.You mean like <base href>...?
No, just a:
var smf_base_url = "http://wedge.org/";
In my experience, Google doesn't read bare links after that, e.g.:
var smf_theme_url = smf_base_url + "/Themes/default";
Obviously, it becomes more complex when those urls aren't always in the base - as with SMF, and as with some of my stuff. I think I tried separating with + without any luck, and ended up using a htaccess forbidden rule or something (because I didn't want the separate content url indexed.)
There are issues with base href in some browsers iirc. It generally works and I've used it before, but I know sometimes it doesn't work like you'd expect.I don't even remember seeing a ready-made function for that in ECMAScript...
var x = <tpl:json value="{$x}" />;
Note that you have to wrap that in CDATA (e.g. around the whole script, as often done in SMF for XHTML compat) or use <tpl:container doctype="html5"> if you want it to be escaped right in <script> (which has text semantics.) In attributes (e.g. onclick="blah(<tpl:json value="{$x}" />);") it will escape the html entities as well, which is necessary to avoid security holes.It seems to be the case for me... In which case, why did JSE ever begin to escape them...?
Gzip doesn't really matter here, you'd be surprised with the strange results I've been getting. (There's a recent discussion somewhere about the effort I went into converting some variable names to be one char instead of long stuff, and in the end the resulting gzipped file was actually longer by a few bytes... I just couldn't believe it.)
function show()
{
var should_i_really_show_it = !this.is_showing();
if (should_i_really_show_it)
this.really_show_it();
}
function hide()
{
var maybe_really_hide_it = this.is_showing();
if (maybe_really_hide_it)
this.really_hide_it();
}
It might seem like the variable names are way too long, and they are (from a this-code-is-ugly perspective.) But really, the biggest problem for compression is that it isn't very repetitive. Consider instead:
function show()
{
var is_it_showing_now = this.is_showing();
if (!is_it_showing_now)
this.really_show_it();
}
function hide()
{
var is_it_showing_now = this.is_showing();
if (is_it_showing_now)
this.really_hide_it();
}
Yes, the variables are shorter. But, the more common pattern may compress better.
One of the important things minifiers do is create patterns. For example, doing "if (x) {} else y();" may actually be better than "if (!x) y();" in a long document that uses "if (x) {" a lot and uses "} else" a lot.
Potentially, but not necessarily. Without actually testing, the cases can always be strange, and small scale experiments are rarely accurate.
Anyway, I was guessing that using \ns might gzip better, at least because the pattern after the \n and including it may be more likely to match (such as "\n <div".) Obviously, the \ does muck this up and so it might not really compress better after all. Certainly it's the same number of bytes whether a newline or n is escaped.Ah, would be swell if Apache added support for 7z or even RAR... :lol:
Anyway:
https://bugzilla.mozilla.org/show_bug.cgi?id=173044
https://bugzilla.mozilla.org/show_bug.cgi?id=366559
AFAIK rar is encumbered so I wouldn't hold my breath for it.Just Google it ;)
I suppose strtr() isn't that slow anyway...
I think that in that PHP fork that was posted here, the guy sped up strtr. Clearly there's room for improvement.
By the way, json_encode, on my setup, is in fact even faster than addcslashes (which by the way, is usually faster than addslashes, even with the same character list.) Haven't checked its memory impact, though, would probably need to xhprof that.I'm getting very annoyed by the constant changes in my CSS. I'd really like to call it quits and commit it, but I'm scared most people will think it doesn't look as good as a stock SMF2, ah ah...
Heck, I made many changes to the visuals this week really. Even the windowbg divs have finally lost their border... (I'm not kidding. I was adamant this border would be in the final design... Until I decided it wasn't needed.)
-[Unknown]
-
I need to answer that post at some point..... :^^;:
Just a quick example on the issues of gzip compression.
index.css, background image for the selected menu item.
h4:hover extends .arrows, h4.hove extends .arrows
background-color: #5a6c85
background-position: 98% -28px
Given that .arrows is defined as such:
.arrows
background-image: url($images/menu.gif)
background-repeat: no-repeat
Now, this css file weighs in at 74.548 bytes.
Replace the first block with this:
h4:hover, h4.hove
background: #5a6c85 url($images/menu.gif) no-repeat 98% -28px
This will duplicate menu.gif (we're talking about base64 encoding, so we get the actual data twice). That particular file is 150 bytes.
The resulting index.css is 74.721 bytes, i.e. 173 bytes more, which makes sense.
Now, let's gzip both files...
Results:
First file: 74.548 -> 30.432 bytes
Second file: 74.721 -> 30.430 bytes
So, the question would be: what's best, a short CSS file, or a short gzipped CSS file?
Obviously it depends on what your users are receiving. If their client can't receive gzipped data, they're better off with the first file. Otherwise, the second file is best.
As it happens, the difference is too small (2 bytes) to justify thinking too much about it. It's a very slightly noticeable improvement for a very, very small share of the community, and a negligible negative difference for the larger majority. I'll take the first version because, really, I like my inheritance system.
But it's a real annoyance that I can't find a way to improve gzipping on the first test case...
-
The resulting index.css is 74.721 bytes, i.e. 173 bytes more, which makes sense.
Now, let's gzip both files...
Results:
First file: 74.548 -> 30.432 bytes
Second file: 74.721 -> 30.430 bytes
So even though it's less bytes, it may not fit a global pattern. How often in the entire CSS file do you use the shorthand versions? Probably not often.
Maybe you use:
"background-repeat: no-repeat;
background-color: #5a6c85;
background-position:"
A lot. But you rarely if ever use:
"background: #5a6c85 url($images/menu.gif) no-repeat"
That sounds likely. So, with the first one, gzip is able to build a larger dictionary entry, which gets used more often, saving more bytes. It's not like gzip compresses x% always, it just compresses patterns like any other compressor.
That said, less bytes may be better for the client than less gzipped bytes, in some cases. Especially for the browser and RAM usage. But generally, less gzipped bytes is probably better. In this case, yes, obviously, doesn't matter. They probably both fit into the same number of packets anyway.
Are you aware of any clients (aside from poorly written HTTP clients in PHP and some robots) that can't handle gzip?
-[Unknown]
-
This particular problem is fixed now because I didn't actually need to include menu.gif again because it was already registered as the background for h4 tags (a slight oversight.)
I'm still of the opinion that micro-optimizations aren't a waste of time, more like a way to train oneself to determine what's important to optimize, and what's not.
I don't really know what can't handle gzip, honestly, but I'm sure there are clients that can handle gzip content, but can't handle files with a gz extension. I mean, Safari itself doesn't, I had to do this annoying htaccess/MIME trick for it. (Uh.)