This is where all of the announcements about Wedge will be posted... But mostly, our updates about the development process.
Hey everyone! Long time no blog. Nope, none of us are dead... We just had 'things' happen in real life, which slowed everything down, sometimes to a halt. I made sure to have at least a couple of commits out every week, just so I never lose focus on what I'm planning to do, but it's true that I've had a tendency to focus on smaller details rather than the big picture.
The main point is that we've often gone through the same story: deciding to implement a minor feature that would be cool, and then realizing, weeks later, that we didn't realize that 'minor' doesn't mean 'quick to implement'... And it went on, and on. As a result, I myself became wary of starting the heavy-duty work on a number of 'heavy' features that I'd like to have in Wedge from day one, and I think if we keep it that way, we'll always have a good excuse not to release Wedge. Not that we're in a hurry to do it -- but we've got to keep it real, and part of it means we have to keep it real for users.
So, the reason why I'm so reluctant to go public is mainly the fact that it's hard to update the database once it's installed somewhere. If you don't tell people to use the upgrade script etc, then you're likely to get annoying support questions...
So here's what I'm suggesting: storing a $settings['db_version'] variable (or a variation if there's already an equivalent), and set it to the SVN revision number corresponding to the lastest database structure change. So, at runtime we can simply, as soon as we load $settings, check for the variable, and if it's lower than the current required database version, load an extra file that will automatically adjust the database structure. e.g. if the db_version is 1652, and user installs version 1815, Wedge will execute all database (and file structure) modifier functions located between those rev numbers -- db_update_1664, db_update_1727, etc... It should be easy to calculate, really, and we can safely call them, and then update db_version to the latest database version available. Then it wouldn't be (so) hard to change the database, whether for us or for users.
What do you think...?
PS: bugger, another issue I just discovered... The selection is off by a few bytes when I click the Bold button in the bbc box, perhaps related to the Opera hack that I wrote for editor.js in both SMF and Wedge... Maybe they FINALLY, after years, fixed that bug...? :edit: Yes, they did, but they didn't fix all of it, so I'll have to keep an eye open...
Aftermath of the Great Deployment
So, yeah, we're about 5 days in after finally moving wedge.org to actually running Wedge, and I'd like to take a bit of time to just go over how I (at least) see it.
First of all, thank you to everyone who's stopped by, especially if you've stopped by and really loved what Nao's done with the default theme. I had the joy of seeing it ahead of time, so the surprise wasn't quite as big on me, but it was really great seeing it, especially being able to watch it grow.
Also, a big thanks to everyone that's put up with the bugs we've had. As you can probably imagine, Wedge is still a living breathing work, it's still growing, still changing, as you can see from the New Revs thread every time we commit.
Because of that, and because we'd never deployed it in a public way beforehand, there was a lot of stuff that had never been thoroughly tested. So it was a matter of a little disappointment for us that things were buggy - and the list has seemed to be pretty huge in some respects - but overall, I think it's turned out pretty well. The bulk of the bugs are more annoying than serious, and I don't believe any of them are privacy or security related, either, so that's something to be quite happy about, really.
The funny thing is that there's a lot of things that I doubt people have really noticed yet. The theme's the most obvious change, and the way there are icons and animations and things, but I would encourage people to dig a little deeper. There are all sorts of less obvious changes afoot. You might just be surprised at what you find ;)
As ever, if there's anything you like or don't like, tell us! We can't fix things if we don't know they're broken, and if there's a better way of doing something than we're currently doing, tell us that too - the end of it, we want to turn out something truly awesome, and while Wedge is a long way down that road, there's still plenty more to do yet.
Sadly, we have not yet turned the corner that marks the end of 'new features' which would otherwise put us on the road to public releases. Things are still a bit too raw for that, still too many things not yet polished, or finished or tested to destruction. But we've certainly passed the biggest milestone: we've finally moved our own site to our own platform. With that, the single greatest hurdle has been passed; we get to test things in a real environment, we got to thoroughly test the importer, and we get to properly share with you what we've been doing all this time.
So, really, thank you all for bearing with us while we got here, and while we deal with the immediate things we've seen from it - and you'll be able to see the rough edges get polished, things get tweaked, things get added or removed. It's been a long road thus far, and it's not nearly over yet - but you're certainly welcome to join us for the ride :)
Regular readers will likely have noticed that I've been slacking for a while... life's been like that, a royal pain. But in order to celebrate getting back into the swing of things, I tackled something I've been wanting to do in SMF for years (and did begin to plan it back in 2010) - some serious automated moderation rules.
The idea is that you should be able to do things easily and conveniently and awesomely. It's still very much WIP, and those who've been following one of the other topics will get a better idea of the innards but that's not why I'm writing here.
I'm writing here because I have screenshots to show as well :D And this way it'll get a bit more visibility than it might elsewhere, which means hopefully a few more people might add their comments to this. Those who follow us on Facebook will have already had some comments from me on this.
The idea is that the system should be able to do some of the work for you, not just moderating posts based on some rules (like post counts and boards) but also on content, and I fully intend to make this even *more* powerful than it already is. But already it's powerful.
The first rule says, if the post begins with /lock and the user has the permission to lock any topic, well, lock the topic.
The second rule says, if the post contains a rude word and the user isn't an administrator, moderate the topic. That's the power we're talking about, you can do things that don't implicitly require permissions everywhere.
The last rule says, simply, if the post is made in General Discussion, moderate it. That's a very broad rule, not likely to be used in real life, but illustrative more than anything.
Notice that we have two moderate rules - this is a logical rather than a technical divide. This way, we can create many different combinations quite easily - here, it's (user isn't admin + says a rude word) OR (posting in a specific board).
I couldn't think of any better way to display it but if you have any ideas, I'd love to hear them. Meanwhile, I'm going to finish up this code (it's barely enough to actually display this!) and then get onto the create-rule UI.
Oh, and people who've read the FB post will also know that I've removed Core Features. Nao's r1304 removed the Media Gallery from being in Core Features leaving Post Moderation as the only thing there, and now, it's not there. Locally, not yet committed, I've removed most of the traces of post moderation from the system (I just need to strip out the permissions stuff where it's buried deep in the permissions system)
But that means I get to finally purge one of the most illogical parts of SMF that we inherited, and I'm glad it's finally gone. I like trimming useless code at the best of times, but this was annoying to boot :D
Anyway, this is me saying "Hello!" and getting back in the saddle to make more awesomeness :)
One year and a half... In a couple of weeks, it will be one year and a half since Pete and I decided to work on our own SMF fork. Talk about commitment!
About a year later, thanks to the SMF license change, over half a dozen new forks went into production. Somehow, I felt a bit relieved that we weren't going to be 'the new SMF' and that maybe we could simply focus on being the smarter one, instead of the one everyone wants to use (including morons. We hate morons. But don't tell them, or they'll come.)
Some of them were officially stopped after a few days or weeks. Others lasted a bit longer, but are clearly never going to happen. Nightwish's fork is clearly, and definitely, the only 'serious' competition we ever had, and given that I was the one that revealed its existence, I feel somehow responsible for pushing him into working hard on it. After several months of daily commits, however, it went off the radar and it would seem that none of the other forks made it into 2012.
Well, I'm sad about it. Once again, everyone's looking at Wedge again and expecting a release. I was kind of enjoying being able to simply work on it without any pressure. Hmm, still no pressure, but I'm starting to feel uncomfortable that nearly a year after SMF 2 went gold, no single fork has been made available to the public. So, it's time for me to look into what's ready and what's not.
If you'll remember, back when I published the first screenshots, I said I'd start using Wedge on Wedge.org itself once we'd reach 200 Likes on Facebook. While it was more of a joke than anything, it just so happens that we're getting close to that decent number of friends, and that in the same time, I'm working hard on making the transition as smooth as possible. So it's very likely that Wedge.org will actually be running Wedge by the time we reach 200 Likes. I'm currently running a 'beta-test' of the Wedge.org website on our friends group. The feedback is very positive, thankfully, so it appears I won't have too much to implement or fix before I make the final switch.
What else? Well, it's become customary to give a quick review of what we did since the last blog post... Pete worked on removing the calendar to turn it into a plugin, but as it turned out, it was harder than expected, and it pretty much hurt his motivation, and he has yet to dive back into the game. Still, 11 commits is better than all of the other forks together, so I guess I can still hope. I really don't want to keep working on Wedge alone, that's enough to drive a man crazy. Plus, well, I love working with Pete. He's such a great guy. I could say the same of the rest of our non-dev team. It's always been such a pleasure to share ideas with you!
And now for the customary changelog summary...
As for myself, I made 125 commits, which is pretty much on par with my regular output. I'm not going to keep up with this in 2012, well I'm hoping I'll take some time off because I never stopped to take a moment, and maybe I should go see a doctor or something. Or at least get paid for working on Wedge. I spent most of November working on improving the template skeleton system (it's now pretty much flawless), and finishing the thought system. In December I rewrote the BBCode implementation to be smarter about fixing mismatched tags and even giving proper error messages to the user.
Which brings us to January, where I kept working on the improved select box. I most notably added a custom scrollbar. After I was done, needless to say I was pretty burnt out, so I decided to focus on a small and easy task instead -- getting rid of the Wysiwyg editor's iframe. Well, that's what I thought... It turned into an absolute nightmare. Oddly, the only browser that liked doing without the iframe was Internet Explorer. Of all browsers. Including IE6. Well. Eventually, I had to admit defeat and went back to the original code. Which, I hate to say, was just as broken. I just don't use Wysiwyg that much, you know. So I spent another couple of weeks working on fixing it. It was pretty much an extensive rewrite, and I can now safely say that our Wysiwyg editor is more solid than SMF's. And shorter. And smarter. As I pointed out in the changelog (you know where to find it), the most annoying bug was due to Wedge using sprited images (inside a div) instead of regular img tags to show buttons and smileys. It turned out that IE loses focus when you click a div outside the iframe, but it doesn't when you click an img. Don't shoot the messenger. No, fear not, I didn't come back to a series of image files, it would be ridiculous. I simply added a dummy transparent image in front of the buttons if using IE. Sometimes, the simple solution is what is best.
After I was done with "small tasks that end up taking weeks to complete", I went the safe route and implemented some additions I'd already written for Noisen.com and Wedge.org -- such as indicating the number of new posts next to a New icon. Or adding support for transparent avatars. (Well, this one's from Aeva Media 2.x, technically.) Then I fixed a few broken areas of WeCSS (which live627 declared his love for, oh I sooo needed that!), and I renamed the 'sticky' feature to 'pinned', which took longer than I'd hoped, but will definitely be an improvement for everyone -- pinning a topic makes more sense than 'setting it sticky'. What does that mean anyway?
Now, what's in store for February? Well, as I said at the beginning, my plans are to finally use Wedge on Wedge.org once and for all, as soon as possible, and that does mean this month. So I've quickly cooked up a new skin for Wedge, which I'll hopefully be using here. My current keyword for Wedge is 'simplicity', so I'm trying to remove as much crap from the templates as possible. The current feedback is very positive, so I think I'm on the right track here.
And if I happen to fall asleep and not post for another month -- go, go, The Artist! The French touch is alive and kicking!
Ticking away the moments that make up a dull day
OK, so what's been happening in the last month? It's been... pretty busy, actually. 71 commits since the last blog post, including:
* OpenID got removed
* some purging of $scripturl from the system in favour of a cleaner method of handling it
* links to profiles now get coloured by membergroup colour through the code (if the current user is not able to visit other profiles, the link is removed, saving bandwidth and preventing search engines from repeatedly trying links that won't go anywhere)
* improved replacement of fetch_web_data, that allows for using cURL on hosts that support it (which is cleaner), and for images on remote servers, attempt to only get as little information as possible for the image rather than the entire image itself, which is great for massive photos, for example
* quite a few new hooks
* Noisen-like thoughts system added
Not to mention various clean-up and overhauling of structures internally.
Today, though, I'm going to talk a little about what I've been working on... the calendar.
If you were to go back through the threads, you'd probably find the earliest discussions on it a year ago trying to find reasons to keep it or lose it, arguing both in favour of keeping it core and turning it into a plugin. At the time the conclusion was that we could keep it and see if we could find reasons to make it worth keeping.
But interestingly, a year later, I'm still having trouble finding reasons to keep it core, when I can make it a plugin and give it better treatment that way.
So, that's what I'm doing, and I'm actually more surprised than I thought I'd be when I realised how intertwined the calendar really was in SMF's core (and, thus by extension, Wedge's core)
If you're curious, just do a search for 'calendar' on the SMF 2.0 codebase, even before you get into all the places where it's just a cal_ prefix, it's hundreds. Far more than I certainly expected, but it's interesting to note that it's prompting new hooks and new ways to extend things that I'd never have thought of.
For example, SimpleDesk came with its own maintenance/find-repair routines. Had I been thinking about it, I'd probably have integrated that into the general find/repair routines, if there were a convenient way of doing so. Which there wasn't, but now there is, so that's good.
It does also suggest one thought to me; the SMF team have long since indicated that it is a desire of theirs to do much the same thing. The one lesson I would take from this, and it is quite important to understand this, is that doing it in the SMF methodology of file edits... this would be beyond a nightmare.
I haven't even gotten to the display or posting pages yet and I've still modified:
* index.php (to remove the calendar action)
* Admin.php (for the admin menu)
* Class-DBPackages.php (to remove the calendar tables from the list of reserved tables)
* ManageMaintenance.php (to remove the calendar tables from the list of tables considered by the old converter)
* MoveTopic.php (to remove functions that update calendar entries when moving topics plus adding a hook for the same)
* RemoveTopic.php (to remove the code called when deleting a topic, so that the calendar is updated at the same time)
* RepairBoards.php (to remove the code from find/repair for the calendar, and adding a hook for the same)
* Subs-Boards.php (to remove the code deleting calendar items when deleting a board, and adding a hook for the same)
* Who.php (to remove the entry regarding permission to view the calendar)
* various language files, notably Admin, Help, ManageMaintenance, Who
Naturally all the removed code is converted to hooked functions, and Calendar.php, Calendar.template.php, ManageCalendar.php, ManageCalendar.template.php and Subs-Calendar.php aren't in the trunk any more
That's also not all of it, either, there's plenty more. I still have 323 mentions of 'calendar' to clean up in the source, and some of those - just for fun - are calendar_month and calendar_day language strings that aren't calendar uses (paid subs, if you're wondering)
But that should indicate how big and complicated the calendar was, and if it's just converted to a mod, it's going to be pretty vile to maintain because of all the edits it necessarily has to make to make everything else work. Doing it without a proper plugin setup is just madness if support is to be a realistic goal, IMO.
I'm actively adding support for things as I go to make it possible to make the conversion, and I hope that if the SMF team are seriously considering it, that they either read this and do something about it, or more likely that they come up with their own game plan as to making extensions fundamentally cleaner to write, because allowing it to go on to be a classic mod is simply a very bad idea.
Plugging in the Plugin Manager
OK, so what's been going on lately? Nao's been busy with a variety of tweaks and gouges to the source, but I've been busy working away on plugins.
(Yes, careful readers will note that we used to call them add-ons, but they're officially called plugins now after the big conversation we invited everyone into. It's a small detail but an important one and since it affects usability, we wanted to make sure you were all involved.)
Why am I writing about it now? Apart from the fact it's been weeks since our last blog post, I really wanted to pimp out the fact that plugins now exist for it - you see, last month when I talked about it, it was still just fresh off the design board, had a few rough edges but worked. Now, some of those rough edges have been smoothed out - you can, for example, actually delete plugins now once they're disabled.
But that's not really what I wanted to talk about. I wanted to talk about the plugins themselves, and I've attached a screenshot from the plugin manager as it is on my PC right now.
First up, you might be wondering why WedgeDesk is broken, when it would be a plugin I'd be more inclined to keep updated. Well, it's only because I very recently changed the hook mechanics in the display page, phasing out the old display_buttons hook and I haven't yet updated WD for the new hook, but that's something I'll do soon enough. (It's also, incidentally, how I realised it wasn't possible to delete a plugin that couldn't be enabled for whatever reason)
All of the other plugins were discussed in http://wedge.org/pub/6929/personal-plugin-showcase-yes-i-m-an-attention-grabbing-git/ - a somewhat aptly named thread, if I may say so myself. ;) If you're curious about any of the plugins I would encourage you to go read that thread, as it has pictures and more details about them.
It's actually listing them alphabetically, though I think I need to change that to include natural case sorting (r in reCAPTCHA comes after Z in the ASCII table, and it probably should discount the case) and I suspect the interface may need some changes going forward to cope with lots of plugins at once - but that's why I'm opening the floor now, so that if you have any thoughts about that, that there's a venue to share them now that you can actually see what having multiple plugins at once would look like.
Oh, and I suppose to validate that it's not some kind of random hyperbole - these are real plugins that can be used :)
It's been a fun period these last few days, kicking out small plugins - I've spent maybe an hour and a half per plugin, and in that time I've been able to establish that the hooks are present for the plugins to work with, and if they weren't present, to make them available; until I wrote the reCAPTCHA plugin, the hooks to extend the verification system didn't exist, for example.
It's also interesting in other ways, I get to test other systems in Wedge, and find ways to enhance them; one of the screenshots I've posted for 'Flitter' in the above-mentioned thread shows a 'Your settings have been saved' message - this is an enhancement I only added recently as I couldn't be sure whether saving had been done without going into the DB. It's a usability enhancement, and it doesn't require plugins having to manually support it; it's covered totally automatically for standard settings pages :) And I managed to find a small bug in the admin pages because of doing plugins, there's really no better way to test how things work than by trying to break them!
My plugins repo is not publicly available (though a few people do have access to it) and I certainly wouldn't call these release-quality, in almost every case it's to prove the system is up to the job, rather than to make sure there's plugins for everyone when we get round to doing some kind of launch (and I can't see my releasing the plugins before Wedge 1.0 gold anyway)
Sadly, I'm going to have to leave it here for now, it's late, I have a few things to do before bed, and even more sadly I couldn't find a good musical reference to leave in this post :( But as the philosopher Jagger once said, you can't always get what you want. But if you try sometimes, you might just find you get what you need.
But there you are, that's where I am right now :)
The saga of the Add-on Manager
It's been a while since we've posted anything on the blog, so I figured I'd do just that, heh.
Of recent times I've been playing with the replacement for the package manager, which while still rather incomplete, is taking form in the code - it even enables and disables simple add-ons right now (meaning that simple add-ons can actually be used in the spirit in which they are intended)
For those who haven't seen the pictures already posted on the subject, I present a screenshot that I took at the weekend.
It's close enough to how things are currently, with the difference that it actually works for both enabling and disabling simple add-ons (ones that don't create or modify the existing DB tables, that is)
I know it was left very much at a disjoint at the end of the last time I wrote about the replacement for the package manager, so I just want to cover off everything that's happened, because there's an awful lot of it, and it's why there haven't been any commits from me lately despite there being a lot of coding going on.
A lot of the stuff I talked about before as being almost whimsically answered is now pretty much finalised, so without further ado... I should point out that there were a surprising number of "Oh, that hadn't occurred to me" moments in the design, but I'm pleased to say that I think they've all been ironed out now.
If you're not interested in technobabble, here's the point where you probably want to stop, with the understanding that it's being designed in a way that promotes add-ons that don't need file edits and should thus survive updates and so on a bit better.
There's an Addons/ folder, right up there next to Packages/, and each add-on gets its own folder in there. Doesn't matter how big or small it is, that's what happens. Everything that happens, happens in that folder. Sources, templates, language files, CSS, JS, images, etc. It's all in there.
I will write a proper tutorial on this sometime but here's the rundown of what the system does.
Here's where it gets really funky. You know as well as I do that users are unpredictable, and often prone to doing odd things, so I took care of that. It doesn't matter what folder an add-on is created with, it could be myaddon/ or my_addon_1.1/, it doesn't actually matter, because you never directly refer to it.
When you create the add-on, you give it an ID, that has the author and add-on name, e.g. Arantor:WedgeDesk, and it's *this* that's used. Everything that you do with an add-on will make use of this ID.
Let's say you have a big add-on that has multiple source files and multiple template files, arranged in src/ and tpl/ respectively, and just live in your add-on's folder. I'll use WedgeDesk as an example.
I want to load the main source: (The .php is added automatically.)
It doesn't matter what folder it's in, you never have to worry about it. Similar deal with loading templates and language files:
No more dumping files throughout the different folders - everything's in one place.
There is a caveat at this point: it does mean that an add-on can't magically have different templates for different themes and just drop them into the relevant folders. That's not to say that it can't be done, it just means having the multiple templates stored inside the add-on and making sure that the theme is checked before calling loadAddonTemplate.
Lastly, you may be wondering how you insert images, if you're not using $settings['images_url'] or $settings['default_images_url'], well, there's a method for that too.
Specifically, if you ever need to refer to an add-on by URL, the base URL (i.e. www.example.com/Addons/youraddon) is available in $context['addons_url']['addonid'], so I can always call WedgeDesk images through $context['addons_url']['Arantor:WedgeDesk'].
There's more, too. If an add-on uses only hooked files (the recommended method), when an error occurs, it's possible to attribute that error to that add-on in the error log, so you'll know if an add-on is acting up.
There's plenty more to do, of course, like provide facilities in the language editor for editing the language files in add-ons, but for now at least, there's an awful lot of work done to facilitate strong add-on support. There's also file editing to provide, plus quite a bit of the DB support, but it is coming along nicely now.
(Oh, and one last thing. If you rename an add-on's folder to something different, or just delete it, it should automatically disable. If it uses file edits, all bets are off, of course, but that's discouraged anyway.)
A year ago, on August 25, 2010, the first commit to the Wedge SVN server was made. I was hoping to get to rev 1000 before the first anniversary, but I tend to commit small bug fixes in a large batch so we 'only' reached the 965 commit milestone.
Well, a year later, what can be said of the project...? It already had a name back then, it still has the same. It had goals and a vision, it still does. Some adjustments were made to our schedule (personally, I decided to give up on a lot of the Noisen.com improvements I was planning to integrate, and will rewrite them later), and other things were pushed into the codebase as we went -- the product of being mainly fueled by passion, instead of money. I know, I know, it's a classic, but it's just the plain truth -- you can't work full time on an unpaid project for a year without some kind of solid passion and commitment. Back when we started the project, I wasn't 100% sure Pete would remain interested in the project in the long run. I was totally wrong, so I'd like to take the opportunity to give a big thumb up to him for being one of the driving forces (the other being me, but I may not say :P) behind the Wedge project, a shining beacon of courage and determination, a herdsman for the nerds, a vanquisher of spaghetti code and... What, I fully expect him to give me the same treatment in return! :niark:
(Oh, and in other news, apparently I'm back in the 'Friends' group at SM.org. Can anyone tell me when this kind of thing happens? :P)
What better way to celebrate the first anniversary than to release a public alpha version! Yes!
Okay, I'm playing with your nerves... There is nothing (yet), but it won't be too long now. As the commit process is now a public one, you've all seen we're still as busy as ever to fix bugs and add new features. I can't honestly see myself release Wedge without having first integrated Aeva into the avatar and attachment system, and I can't work on that before I can fully focus on it. And this month has been doing a very good work at distracting me: Nightwish's fork (interesting ideas, it's definitely going to be the worthiest competitor to Wedge if Nightwish can keep up for another year :D), playing a little gem of a game played Xenoblade (okay, so I passed on other RPGs and JRPGs these last few months because they're time eaters, but the guy made one of my all-time top 3 games so I have to play his titles, even if unrelated to Xenogears), and more generally a personal burnout and declining health (I'm not one to complain, though. These things happen in cycles.)
In short: I don't remember my original schedule, but right now my personal target is for a demo in early September, and an alpha in late September or October. I'm still kind of wary about releasing my code to a general audience. It's our baby you know... We want Harvard for it. Hopefully it'll spawn into Facebook. Or something else.