« Login/Register Feature
These two bytes may not matter to you...
Need some help on a UI matter »

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #135, on October 4th, 2011, 06:45 PM »
Quote from Nao on October 4th, 2011, 04:35 PM
Well... Readability is important -- when you're planning to maintain the thing!
But do you think you're going to?
Therein lies the issue. I don't really expect to do much maintenance, even less than with BB.
Quote
I was looking for similar software, thanks for pointing to it. I manually passed all of the getid3 files to it, stripping comments and whitespace (but not doc comments because getid3 uses this in its parser... Geeky for sure, but a horrible way of doing things!), and keeping just the header comment.
The comments aren't needed for phpseclib and with the header that retains the copyright and so on, I spent 15 minutes on it and got it down to 280KB or so, then just spent time faffing about cleaning the code, didn't really change the size but it did make it tidier. It'll zip fairly well, I think, haven't tested it.
Quote
Not bad...! Although I'll have to test it, I guess... It's an interesting alternative, and it only took 15 minutes or so to convert all of the files. Something I *can* do on each major release for sure...
How important is it to keep the download size small vs readable? I'm not actually that fussed myself.
Quote
- doing on-the-fly minification (i.e. not cached...) on inline JS and CSS. Do you think it's realistic? I'm not sure myself, but I could cook up a simplified minification function, although I wouldn't know how to deal with strings. Hmm. Still, just as an example, if I manually minify the noi_resize() code, I save something like 60 to 80 bytes off the page, and a reasonable ~20 bytes when gzipped. Considered it's to be found on every single page...
If it's on every single page, why is it inlined? Or, at least, not minified by hand?
Quote
- media cache: I'm always tempted to add a htaccess for the media folder, just so I don't get the pagespeed warning when it can't find a far-future expiration date. Problem is -- if you replace the files with a newer version, if the files have the same name, then the generated name has the same name as well. And that's only for thumbnails... Previews and full pics are accessed through PHP, so I can't "regenerate" a new filename for them, it won't help at all.
Aren't the timestamps included in the filename? They are for avatars.
Quote
Need I worry about caching media items at all...?
If the timestamp's part of it, yes, we can cache them. If not, don't worry about it.
Quote
- considering using YepNope to load jQuery and subsequent code. YepNope is 1.6kb minified and gzipped, so it's not exactly super-light, but at least it would trick pagespeed into thinking no other JS is executed at page load time, so I'd be likelier to get it to a perfect score... Main problem is, I doubt I'll be saving any time here. The only 'big' script we have to load is jQuery, the rest is negligible (except maybe for script+theme). At best, jQuery will load in parallel while YN is loading other JS files, and thus will be able to execute everything at the same time instead of waiting to load script+theme and other scripts after jQ is loaded. Still, if jQ is integrated into the main local file, it's even more pointless. At worst, YN wastes time executing itself when both jQ and local scripts are already cached... :-/
I wouldn't worry about YepNope, because you're still doing more work that doesn't - ultimately - benefit the user. It just fools a score.
Quote
- Considering using "FileETag none" in all htaccess files. I'm not a specialist though... But as soon as we have an expiration date, ETags aren't useful at all. I'm not sure if it should be done for all files.
If you have an expiration date, the ETag is still good until that date.
Quote
- IP packets carry the cookie, as everyone knows. My question would be, does it carry it only in the first packet of the page, or on every single freakin' packet? In which case it'd be in our interest to make cookie size as small as possible...
It's embedded into the HTTP request, which is only sent once for the request, even if the header is split across multiple packets because of lots of extra headers. It's still worth keeping it as small as possible (though, frankly, I don't see how else we can!) simply to hope we can keep it into as few packets as possible.
Quote
- I noticed that by default, avatars are loaded into the attachment folder. I *know* AeMe will change the entire system, but since I still don't know when I'll deal with this, I'd like us to consider creating a 'custom' folder in /avatars/ and upload all avatars there by default -- and obviously, set Wedge by default to show clear URLs for these avatar images. We save *by default* as many HTTP requests (to PHP FFS!) as there are avatars on the page.
http://tracker.wedge.org/?sa=ticket;ticket=28 ;) Yes, we save multiple PHP/DB requests *per page load*.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Dragooon

  • I can code! Really!
  • polygon.com has to be one of the best sites I've seen recently.
  • Posts: 1,841
Re: These two bytes may not matter to you...
« Reply #136, on October 4th, 2011, 09:14 PM »
I want to know, why are you guys putting in so much effort for a server side script? I can understand for a JS library but why for a script in which reducing the size doesn't actually matter in terms of speed or anything else?
The way it's meant to be

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #137, on October 4th, 2011, 09:19 PM »
I can't speak for getID3, but a script that's 600KB (including comments) that will all be loaded at once, because that 600KB is all interdependent. That, and the fact that I'm not too thrilled about the size increase that will necessarily result from it (but that, at the same time, I can't really make it a plugin either)

Nao

  • Dadman with a boy
  • Posts: 16,082

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

Nao

  • Dadman with a boy
  • Posts: 16,082

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278

Nao

  • Dadman with a boy
  • Posts: 16,082

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #143, on October 4th, 2011, 09:50 PM »
Nope. It's all pure PHP. Lots of big numbers expressed in 0x0000 format, but that's about it. Though, I suspect I'd rather not mess about to keep reminifying if possible.

AngelinaBelle

  • Still thinking...
  • Posts: 92
Re: These two bytes may not matter to you...
« Reply #144, on October 6th, 2011, 03:08 PM »
In the code tags above, The "\" in "\n" seems to be supressed on display.  The serialize line looks like this:
Code: [Select]
file_put_contents($cachedir.'/test4.php', "<?phpnreturn '" . serialize($myDataArray) . "'; ?>");
I'm an SMF doc writer.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #145, on October 6th, 2011, 03:14 PM »
How quaint, an old SMF bug back to strike again. I thought that had long since been fixed (because it is an SMF bug originally)

Nao

  • Dadman with a boy
  • Posts: 16,082

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #147, on October 6th, 2011, 04:15 PM »
Quote from Nao on October 6th, 2011, 04:08 PM
Is this on Wedge, or Wedge.org...?
Wedge.org is running SMF2 RC4, not Wedge.
It's here. Back when you posted your code sample testing var_export vs serialize a page or so ago, there's a string with <?php in it which should be followed by \n but isn't, it just becomes <?phpn in the layout.

It's originally an SMF bug around how it does the post split to push things to the highlighter but I thought it was long since (pre RC2!) fixed.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: These two bytes may not matter to you...
« Reply #148, on October 6th, 2011, 06:48 PM »
Maybe I just didn't port that bug fix into my copy of Noisen/Wedge.org...? I mean, originally Noisen runs on SMF 2.0 Beta 1 (the September 2007 release), and I applied all subsequent updates manually...
Quote from Arantor on October 4th, 2011, 06:45 PM
Therein lies the issue. I don't really expect to do much maintenance, even less than with BB.
Then, you may easily compress the files.
Anyone can re-download them from the relevant website (getid3 or libsecsomething) and replace their minified files with the originals...

The comments aren't needed for phpseclib and with the header that retains the copyright and so on, I spent 15 minutes on it and got it down to 280KB or so, then just spent time faffing about cleaning the code, didn't really change the size but it did make it tidier. It'll zip fairly well, I think, haven't tested it.

Oh, speaking about zipping... I made a few tests with a Sep. 15 revision of Wedge.
I didn't test with WinRK (it's not free), but I think it wouldn't be (much) better than PAQ...
Also, please remember that the Wedge package is currently 50% bigger than SMF's -- add 1.5MB for AeMe, 500KB for /other/stupid-sub-folders/*, and the rest is mainly new features, new code etc...

ZIP: 3653kb (by comparison, SMF = 2610kb)
ZIP (best): 3642kb
tar.gz (best): 3132kb (85% of the zip file) (by comparison, SMF = 1941kb, i.e. 74% of the zip file)
RAR (best): 3006kb
tar.bz2 (best): 2734kb (75% of the zip file) (by comparison, SMF = 1598kb, i.e. 61% of the zip file)
7z SFX LZMA (ultra): 2688kb
RAR (solid best): 2464kb
7z SFX PPMd (ultra): 2450kb (67% of the zip file)
ZPAQ: 1983kb
PAQ8o: 1828kb (50% of the zip file)

So... PAQ does twice better than ZIP, but it's not very realistic to use it (too long to compress, too much Ram needed....), so 7z SFX PPMd (which doesn't require any decompression program per se) would be a good alternative for offering a downloading to non-techies. (Okay, Mac users may be non-techies as well but I'm catering only to Windows users for now...)
I'm not exactly sure, though, why bz2 is less efficient on Wedge than on SMF... Do we have that many extra image files in it, for instance...?

Oh, and where are you now, regarding the SMF 2.0.1 patch?
Quote
How important is it to keep the download size small vs readable? I'm not actually that fussed myself.
Anything we maintain should be readable. Anything else we use, the smaller the better. We don't want a big download size, not because it's "big", but because it costs money to cover for the bandwidth -- although we can probably share the files via Mediafire and so on... Or just extra mirrors... And because I'm not sure people would like to download and reupload a big file 10 times a year just because we release early and often. I know I wouldn't...
(Then maybe we should have some kind of system that automatically sends the updated files to the server.)
Quote
Quote
- doing on-the-fly minification (i.e. not cached...) on inline JS and CSS. Do you think it's realistic? I'm not sure myself, but I could cook up a simplified minification function, although I wouldn't know how to deal with strings. Hmm. Still, just as an example, if I manually minify the noi_resize() code, I save something like 60 to 80 bytes off the page, and a reasonable ~20 bytes when gzipped. Considered it's to be found on every single page...
If it's on every single page, why is it inlined? Or, at least, not minified by hand?
It's inlined because I need to execute it ASAP. Waiting for jQuery to run is out of the question. It would save probably a hundred bytes, but would create a very noticeable delay when loading a page that doesn't match the expected viewport width. Then again, onresize() doesn't execute when using the Back button (at least in Opera), so it's not exactly a 'perfect' solution anyway. Doing my best...

Can be compressed manually, yeah. But I figured I'd rather have it done logically -- if one part of the inline code is compressed, then all of it should be compressed. And I'm not sure what to use to minify JS when it comes to something that has to be done on-the-fly for everyone, on every page, instead of done once and cached...
Quote
Aren't the timestamps included in the filename? They are for avatars.
No, I don't think they are (for media files). If you update the file with the same file, it gets the same filename.
Which is a bit of a bummer...

Oh, and I'd love us to consider showing preview & full sizes in plain view as well. As long as the filenames can't be 'found out' by analyzing the thumbnail URL...
Quote
I wouldn't worry about YepNope, because you're still doing more work that doesn't - ultimately - benefit the user. It just fools a score.
Yeah, I suppose so... And even YSlow doesn't care about jQuery being there. Only Google PageSpeed does. And it would only save at BEST 2% because I get a maximum score of 98%, even with jQuery loaded normally.
Quote
Quote
- Considering using "FileETag none" in all htaccess files. I'm not a specialist though... But as soon as we have an expiration date, ETags aren't useful at all. I'm not sure if it should be done for all files.
If you have an expiration date, the ETag is still good until that date.
But technically it's not used at all because the browser will no longer request the resource (as long as it caches it, of course.)
Hmm, seems like YSlow doesn't recognize my FileETag query. Maybe I should also do 'Header unset ETag'... And any removed header means more space for pure data...
Does Chrome have an extension to help view request headers?
Quote
It's embedded into the HTTP request, which is only sent once for the request,
Phew...
Quote
even if the header is split across multiple packets because of lots of extra headers. It's still worth keeping it as small as possible (though, frankly, I don't see how else we can!) simply to hope we can keep it into as few packets as possible.
Yeah, well, I suppose it's not as scary if the cookie is loaded sent once. The first packet will be shortened a lot, but not the rest.
Quote
http://tracker.wedge.org/?sa=ticket;ticket=28 ;) Yes, we save multiple PHP/DB requests *per page load*.
Ah, I forgot about that ticket...
Then I guess we agree on this, eheh.
Do you want to implement the change?

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: These two bytes may not matter to you...
« Reply #149, on October 6th, 2011, 08:17 PM »
Quote from Nao on October 6th, 2011, 06:48 PM
Maybe I just didn't port that bug fix into my copy of Noisen/Wedge.org...? I mean, originally Noisen runs on SMF 2.0 Beta 1 (the September 2007 release), and I applied all subsequent updates manually...
I don't know where the bug was, just that there *was* a bug in code elements with it occasionally eating backslashes, back before RC2 and that I couldn't reproduce it.
Quote
Then, you may easily compress the files.
Anyone can re-download them from the relevant website (getid3 or libsecsomething) and replace their minified files with the originals...
Though I was planning on making it a single file to avoid having to fix the pathing; the paths are all working under the assumption that the include path has been set to include the parent working directory (much as you would in anything using PEAR libraries, as there is a PEAR library included)

If I were to leave it as multiple files, I would at least have to fix the source loading, but that's a very small price to pay.
Quote
Oh, speaking about zipping... I made a few tests with a Sep. 15 revision of Wedge.
I didn't test with WinRK (it's not free), but I think it wouldn't be (much) better than PAQ...
Also, please remember that the Wedge package is currently 50% bigger than SMF's -- add 1.5MB for AeMe, 500KB for /other/stupid-sub-folders/*, and the rest is mainly new features, new code etc...
And some of it is going to get bigger, some smaller, but it's interesting to note.
Quote
So... PAQ does twice better than ZIP, but it's not very realistic to use it (too long to compress, too much Ram needed....), so 7z SFX PPMd (which doesn't require any decompression program per se) would be a good alternative for offering a downloading to non-techies. (Okay, Mac users may be non-techies as well but I'm catering only to Windows users for now...)
Compression isn't really a problem, because it's not like we'll be doing it daily or weekly. The other thing is that we don't need to offer 3 types of package, but offering zip and tar.gz is advantageous, when I deploy a new site, I just wget the .tar.gz to my server, tar -zxf the file and I'm about ready to install.

I'm really not sure that offering self-extracting archives is really that useful.
Quote
I'm not exactly sure, though, why bz2 is less efficient on Wedge than on SMF... Do we have that many extra image files in it, for instance...?
No, but spriting images makes a difference. Both gz and bz2 start by making a single file out of everything before compressing it (while zip doesn't) - and if you have less repeating patterns, like multiple GIF headers condensed into a single PNG file, or the patterns are physically smaller, like shortening file headers, the saving is smaller.
Quote
Oh, and where are you now, regarding the SMF 2.0.1 patch?
Not yet done anything with it.
Quote
Anything we maintain should be readable. Anything else we use, the smaller the better. We don't want a big download size, not because it's "big", but because it costs money to cover for the bandwidth -- although we can probably share the files via Mediafire and so on... Or just extra mirrors... And because I'm not sure people would like to download and reupload a big file 10 times a year just because we release early and often. I know I wouldn't...
(Then maybe we should have some kind of system that automatically sends the updated files to the server.)
In which case it might be worth comparing zipping stuff vs zipping stuff without comments and seeing what difference it makes. But I don't really have an issue with bundling phpseclib as is, just I didn't want to bulk out with 600KB of code if I didn't have to.
Quote
It's inlined because I need to execute it ASAP. Waiting for jQuery to run is out of the question. It would save probably a hundred bytes, but would create a very noticeable delay when loading a page that doesn't match the expected viewport width. Then again, onresize() doesn't execute when using the Back button (at least in Opera), so it's not exactly a 'perfect' solution anyway. Doing my best...
Just wanted to check. A bunch of code, potentially, can be inlined into the page, both actual-inline and delayed-inline, and it wouldn't hurt to be able to minify both - provided that the generation time isn't huge. It wouldn't be Packer, I'd argue the benefit there would be outweighed by the execution cost. But it's certainly worth some experimentation.
Quote
No, I don't think they are (for media files). If you update the file with the same file, it gets the same filename.
Which is a bit of a bummer...
The filename should really include the timestamp.
Quote
Oh, and I'd love us to consider showing preview & full sizes in plain view as well. As long as the filenames can't be 'found out' by analyzing the thumbnail URL...
Hence putting the timestamp in, making one more unknown for an assailant. Though we could get around that by making any folder other than avatars totally off limits (but with the cost of making thumbnails have to go through PHP)
Quote
Yeah, I suppose so... And even YSlow doesn't care about jQuery being there. Only Google PageSpeed does. And it would only save at BEST 2% because I get a maximum score of 98%, even with jQuery loaded normally.
*nods* It has a use, but not here, I suspect.
Quote
But technically it's not used at all because the browser will no longer request the resource (as long as it caches it, of course.)
Hmm, seems like YSlow doesn't recognize my FileETag query. Maybe I should also do 'Header unset ETag'... And any removed header means more space for pure data...
Unless we use ETags, unset them.
Quote
Does Chrome have an extension to help view request headers?
Use the Inspector. Switch to the Network tab, and just wait for the request to be made - you get to see the request and response headers in full.
Quote
Yeah, well, I suppose it's not as scary if the cookie is loaded sent once. The first packet will be shortened a lot, but not the rest.
The cookie isn't actually that big, and it's not like we can readily shorten it either.
Quote
Ah, I forgot about that ticket...
Then I guess we agree on this, eheh.
Do you want to implement the change?
Yes, fully to put avatars in their own folder and have them requested directly rather than through PHP.

Considering that SMF can do this already, I was only originally planning to make use of the existing code, but strip out the other stuff of serving avatar files through dlattach. I hadn't factored in working with Aeva at that point.
« Login/Register Feature
Need some help on a UI matter »