« Getting ready for an alpha release: CSS fixes
Getting ready for an alpha release...
Controversial idea: post moderation on by default »

Nao

  • Dadman with a boy
  • Posts: 16,082
Getting ready for an alpha release...
« on August 18th, 2012, 12:24 PM »Last edited on August 31st, 2012, 01:55 PM
So... I'm looking into making this August 25 date a reality, right...? :edit: Obviously at a later date, now...

In order to achieve such a close release date, I needed to decide that some of the features I want for 1.0 will have to go through an upgrade script of my own. I hate the idea, but I'd rather postpone 20% of my features (and a stable release), than have Wedge remain vaporware forever... Plus, the interest we might generate could help get more developers onboard.

So, there are places in the code where "I'm not too sure" about what I've been doing, and here's what I'll do: I'll simply post patches and ask for feedback... This is better than withholding commits until I'm sure of what I'm actually doing.

Could you guys review the first patch? It's a simple one really -- it's something we discussed before, the default lengths for various database columns. I tend to forget what we talked about, especially when it comes to MySQL, so I don't know what is "right" and what isn't. And because we're going to have an upgrade script eventually, it also means we can safely change these field lengths later in the process, so I'm not even sure this patch needs applying at all.

Yet, there might be a few fields that warrant shortening or lengthening. You tell me. Thanks.

:edit: Removed attachment to make it possible to have the topic in public...

live627

  • Should five per cent appear too small / Be thankful I don't take it all / 'Cause I'm the taxman, yeah I'm the taxman
  • Posts: 1,670
A confident man keeps quiet.whereas a frightened man keeps talking, hiding his fear.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Getting ready for an alpha release...
« Reply #2, on August 18th, 2012, 08:40 PM »
OK, let's recap something.

smallint ranges from -32768 to 32767 (0 to 65535 when unsigned)
mediumint has a range of +/- 8 million (16m when unsigned)
int has a range of +/- 2 billion (4 billion when unsigned)

You cannot assign that many groups to users, because if you have that many, the additional_groups will overflow IIRC as it only has so much space.

And, also, I'm still not comfortable expanding ids just for the sake of it - it WILL have a performance penalty on indexes on larger forums.

I can see the use for millions of groups - user created contact lists. (Though the group column MUST remain signed in most places e.g. with permissions, where permissions are required to apply to guests which are group -1. I can't consider the need for literally billions of groups, though even with a signed mediumint, that's still a column running into the millions, which I can imagine could be the case.)
Re: Getting ready for an alpha release...
« Reply #3, on August 19th, 2012, 02:13 AM »
Oh, that reminds me, I need to remove the ban system as it stands so people don't get any funny ideas about using it because it'll be completely fucked up.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Pandos

  • Living on the edge of Wedge
  • Posts: 635
Re: Getting ready for an alpha release...
« Reply #4, on August 20th, 2012, 09:59 AM »

Just take a short look at your install.sql and i have a view remarks:


boards table:

In boards table there is no field named id_draft where you define: PRIMARY KEY (id_draft).
Perhaps change PRIMARY to id_board?


What about the messages table?
Do not need to have an PRIMARY KEY on id_msg?
Also the topics table. No PRIMARAY KEY?


board_members, group_moderators, log_boards, log_mark_read,  moderators and subscriptions_groups:
ENGINE=MyISAM ?



This is my first view. Will take a deeper look later.


THX for all your work!!!


Sven
# dpkg-reconfigure brain
error: brain is not installed or configured

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Getting ready for an alpha release...
« Reply #5, on August 20th, 2012, 02:37 PM »
It's a patch, not the full install.sql file. You're only seeing the differences between what's in the SVN trunk and what Nao's suggesting, which means you're seeing a tiny tiny tiny fraction of it.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Getting ready for an alpha release...
« Reply #6, on August 23rd, 2012, 02:36 PM »
@John> Are you implying that Wedge will never be installed on a successful forum? :P
Quote from Arantor on August 18th, 2012, 08:40 PM
smallint ranges from -32768 to 32767 (0 to 65535 when unsigned)
mediumint has a range of +/- 8 million (16m when unsigned)
int has a range of +/- 2 billion (4 billion when unsigned)
Additionally there are some fuck-ups...
For instance, let's take the group ID.
By design, SMF considers guests as group '-1'. Meaning that everytime the guest group has no reason to be stored in the database (simple example: the members table, they can't belong to the guest group...), we can safely use int(10) unsigned, but when it needs to be taken into account (example: permission tables), we have to use int(10) signed. I'm saying int(10) and not int(11) because if the negative number is never going to be < -10, there's no reason to account for the minus sign, but it doesn't matter as it's only for representation purposes in the command line as you said.
So, I don't even know if it's worth storing unsigned at all, because there are going to be situations where id_group > maxint/2 and thus will be stored correctly in some places (members), and badly in others (perms).
Quote
You cannot assign that many groups to users, because if you have that many, the additional_groups will overflow IIRC as it only has so much space.
I thought we'd discussed that it'd be best to get rid of additional_groups and instead use a subselect..?
An alternative would simply be to limit the numbers of groups you can create, number of people you can put in them, and number of groups you can belong to. i.e. have to find some technical way of allowing you to join any group, but disallowing people from adding you to their groups after a while.
Of course, in a realistic environment, it's unlikely that more than 0.0000001% of all Wedge installs will ever need to have more than a few hundred spaces 'available' for storing groups and so on.
Quote
And, also, I'm still not comfortable expanding ids just for the sake of it - it WILL have a performance penalty on indexes on larger forums.
And these forums are precisely the ones that need the expanding...
And if they're big, they can afford the extra hardware to handle the extra weight :P

(And you can retort that they can also afford to hire us to augment their database field sizes and optimize everything even more. :lol:)

Also, it's worth noting that Dragooon's original database schema for SMG was... A little bit skewed towards integer fields. Many fields have an int(11) definition even for storing small numbers -- e.g. the width/height fields in media_files are integers when they could just as well be smallint(4), i.e. < 65536... I don't see any realistic scenario where an online uploaded image would be wider/taller than that.
Quote
I can see the use for millions of groups - user created contact lists. (Though the group column MUST remain signed in most places e.g. with permissions, where permissions are required to apply to guests which are group -1. I can't consider the need for literally billions of groups, though even with a signed mediumint, that's still a column running into the millions, which I can imagine could be the case.)
My point exactly. :)
Re: Getting ready for an alpha release...
« Reply #7, on August 23rd, 2012, 02:51 PM »
- I'm not sure whether mass database changes are ideal right now. But I also don't see myself making an upgrade script for all of that... Maybe we can write a script that would automatically parse install.sql, and compare it to your current one (maybe storing the latest used install.sql somewhere in the database in compressed form..?). It would just be a matter of doing a virtual diff of both files, finding the areas where the changes are done, and trying to do them in the database automatically.
But you wanna know...? I don't feel like I'm ever going to write that kind of script. This is where I'm getting crazy.

- I've added a 'fun' little experiment on the Stats page (go see it)... Since there was an empty area, I added a 'Top liked authors' box which has the benefit of showing more names than just Pete and I, ah ah! Well, it's funny because in the end, after all this time with likes enabled etc, it looks like we both have the same amount of posts (approx.), as well as the same number of likes! But I'm wondering whether it's fine to show this box. It's more of a philosophical matter. Do we want to encourage competition for people to get more likes..?

- could someone do tests on wedge.org's test board, posting various embeddable material and seeing if everyone's going all right? (i.e. no double-encoding of links, no broken stuff, works everywhere, works when posting in quick reply, quick modify, reply and modify...) Because wedge.org has been running a custom Aeva-Embed.php and Subs-BBC.php for quite some time now (including the removal and addition of an AWFUL lot of code), and I think it's going to be time to commit it...

- shall I count theme users per skin or per theme in the Skin Selector page? It only involves replacing "id_theme" with "id_theme, id_skin" in the SQL query for that -- but it might not help when it comes to performance...

- shall the board icons be set to 'New' status if there are new topics in the corresponding board, or new topics in the corresponding board *and* its children...? Currently it's the latter, but I think the former would be best, if only because sub-boards have their own 'New' label...

- just for the record: I'm currently reworking (quite heavily) the layouts for the Search page and Media page (homepage mostly). And I'm totally not happy, and I totally need to commit before we go alpha... :-/

Aaaaand, that's all for now.
And I haven't even had a look into my to-do-list for months, really... What fun to be had!

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Getting ready for an alpha release...
« Reply #8, on August 23rd, 2012, 03:24 PM »
I'd leave database changes for now, ultimately. You're right that it's very complicated.

As far as upgrading goes, I really wouldn't bother worrying about it for the alpha. Anyone that uses an alpha should be prepared for anything and everything, including manual DB changes, and shouldn't be using it on a production site anyway.

I'd suggest just committing the changes to Aeva and Subs-BBC. Most things work as expected at this point in time. (Though it would be nice if we can get Aeva to use the no-cookies version of the YT embed code, and ideally using the iframe method rather than a Flash object)
Quote
Do we want to encourage competition for people to get more likes..?
That is going to be dependent on the community. There was a thread on AAF discussing the fact that I had a lot less posts than Shawn and yet more likes. If the community notices enough to care, they'll figure out whether they want to make it competitive.
Quote
- shall I count theme users per skin or per theme in the Skin Selector page? It only involves replacing "id_theme" with "id_theme, id_skin" in the SQL query for that -- but it might not help when it comes to performance...
Change it, that's cool. We can worry about performance later.
Quote
- shall the board icons be set to 'New' status if there are new topics in the corresponding board, or new topics in the corresponding board *and* its children...? Currently it's the latter, but I think the former would be best, if only because sub-boards have their own 'New' label...
That makes sense. It would solve a lot of troubles.
Quote
- just for the record: I'm currently reworking (quite heavily) the layouts for the Search page and Media page (homepage mostly). And I'm totally not happy, and I totally need to commit before we go alpha...
That's the beauty of being an alpha, it doesn't have to be ready in any fashion.

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Getting ready for an alpha release...
« Reply #9, on August 23rd, 2012, 07:45 PM »
Quote from Arantor on August 23rd, 2012, 03:24 PM
I'd leave database changes for now, ultimately. You're right that it's very complicated.
Couldn't we at least update the most crucial ones, like the id_group stuff..?
Quote
As far as upgrading goes, I really wouldn't bother worrying about it for the alpha. Anyone that uses an alpha should be prepared for anything and everything, including manual DB changes, and shouldn't be using it on a production site anyway.
Well... We're using it on wedge.org, and will definitely have to update our tables one way or another... :^^;:

What could be done, sorta, is getting the new install.sql file, and doing a pseudo installation, i.e. running it with a setting of db_prefix = db_prefix + 'temp_install_whatever', and then compare the tables... (Or whatever.)
Quote
I'd suggest just committing the changes to Aeva and Subs-BBC.
There are a lot, still ;)
That's why I want to be sure that it works as is, right now...
If anything, I know that footnotes are still buggy in some respect. I mean, when you accumulate multiple quotes with multiple footnotes in them.. It starts to fail. Kind of thing I haven't worked on lately. Was too busy on CSS crap...
Quote
Most things work as expected at this point in time. (Though it would be nice if we can get Aeva to use the no-cookies version of the YT embed code, and ideally using the iframe method rather than a Flash object)
Hmmm... That's odd. I was pretty sure YT as an iframe was already implemented?! I remember working it into AeMe months ago... But it's nowhere to be seen in my current code (?).
Can you confirm this?
It's really easy to implement the new YT code, it's pretty much the same as in Google Maps (set type to html and use an iframe tag), only simpler :P
Quote
That is going to be dependent on the community. There was a thread on AAF discussing the fact that I had a lot less posts than Shawn and yet more likes. If the community notices enough to care, they'll figure out whether they want to make it competitive.
I've committed it... Feel free, and I mean it, feel free to revert it anytime!
Quote
Change it, that's cool. We can worry about performance later.
(Done.)
Quote
Quote
- shall the board icons be set to 'New' status if there are new topics in the corresponding board, or new topics in the corresponding board *and* its children...? Currently it's the latter, but I think the former would be best, if only because sub-boards have their own 'New' label...
That makes sense. It would solve a lot of troubles.
Good then :)
Quote
That's the beauty of being an alpha, it doesn't have to be ready in any fashion.
Our first users will hate us for it... :lol:
Re: Getting ready for an alpha release...
« Reply #10, on August 24th, 2012, 12:38 AM »
Just had a silly idea, but well...
Adding a 'mobile' variable to $context['browser'], similar to 'webkit', but being based upon $user_info['is_mobile'] instead. This would allow skinners to create index.mobile.css files for mobile devices.
However, all devices go through the mobile path, and thus are usually redirected to the Wireless skin, so it might be a bit overkill to additionally offer this keyword...
Anyway...

godboko71

  • Fence accomplished!
  • Hello
  • Posts: 361
Re: Getting ready for an alpha release...
« Reply #12, on August 25th, 2012, 06:33 PM »
Well the keyword might encourage skinners to make wireless versions of there theme which is always welcome. Not that they can't do it with wireless
Thank you,
Boko

Nao

  • Dadman with a boy
  • Posts: 16,082
Re: Getting ready for an alpha release...
« Reply #13, on August 25th, 2012, 11:59 PM »
(A small bump for Pete, still! :lol:)

Issue with using a index.mobile.css as opposed to <options><mobile>1</mobile></options> in skin.xml, is that mobile users get the extra CSS whatever their choice is. One of the features I like in Wedge, is that I keep track of both desktop and mobile use, and serve a different skin, but you can also choose to ignore that and use the desktop skin in your smartphone, or the mobile skin in your desktop browser. If you add an index.mobile.css file, it will always (always!) execute on mobile phones/tablets, and never on desktop.

Which might very well be as intended, of course! So I'm not sure it's actually a drawback, or instead another proof that Wedge is freaking flexible. 8-)

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Getting ready for an alpha release...
« Reply #14, on August 26th, 2012, 12:23 AM »
Quote
Couldn't we at least update the most crucial ones, like the id_group stuff..?
I really would leave it alone at this point if you're still planning on doing some kind of release this weekend.
Quote
What could be done, sorta, is getting the new install.sql file, and doing a pseudo installation, i.e. running it with a setting of db_prefix = db_prefix + 'temp_install_whatever', and then compare the tables... (Or whatever.)
For everyone else I figure that alpha releases won't really be upgraded, and it's not like we can't just diff the install file to figure out what's changed - and convert that into something else that's easily usable (even here)
Quote
That's why I want to be sure that it works as is, right now...
It's committed, just needs more testing really. I haven't noticed any funny problems but I haven't been trying to embed. (Unfortunately I'm just in the process of figuring out how to pay for my grandmother's healthcare, is complicated)
Quote
This would allow skinners to create index.mobile.css files for mobile devices.
However, all devices go through the mobile path, and thus are usually redirected to the Wireless skin, so it might be a bit overkill to additionally offer this keyword...
I didn't initially understand the consequences but I think the result is better for themers to do so, so go for it :)
« Getting ready for an alpha release: CSS fixes
Controversial idea: post moderation on by default »