The saga of the Add-on Manager

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
The saga of the Add-on Manager
« on September 13th, 2011, 03:31 PM »
It's been a while since we've posted anything on the blog, so I figured I'd do just that, heh.

Of recent times I've been playing with the replacement for the package manager, which while still rather incomplete, is taking form in the code - it even enables and disables simple add-ons right now (meaning that simple add-ons can actually be used in the spirit in which they are intended)

For those who haven't seen the pictures already posted on the subject, I present a screenshot that I took at the weekend.


It's close enough to how things are currently, with the difference that it actually works for both enabling and disabling simple add-ons (ones that don't create or modify the existing DB tables, that is)

I know it was left very much at a disjoint at the end of the last time I wrote about the replacement for the package manager, so I just want to cover off everything that's happened, because there's an awful lot of it, and it's why there haven't been any commits from me lately despite there being a lot of coding going on.

A lot of the stuff I talked about before as being almost whimsically answered is now pretty much finalised, so without further ado... I should point out that there were a surprising number of "Oh, that hadn't occurred to me" moments in the design, but I'm pleased to say that I think they've all been ironed out now.[1]

If you're not interested in technobabble, here's the point where you probably want to stop, with the understanding that it's being designed in a way that promotes add-ons that don't need file edits and should thus survive updates and so on a bit better.



There's an Addons/ folder, right up there next to Packages/, and each add-on gets its own folder in there. Doesn't matter how big or small it is, that's what happens. Everything that happens, happens in that folder. Sources, templates, language files, CSS, JS, images, etc. It's all in there.

I will write a proper tutorial on this sometime but here's the rundown of what the system does.

Here's where it gets really funky. You know as well as I do that users are unpredictable, and often prone to doing odd things, so I took care of that. It doesn't matter what folder an add-on is created with, it could be myaddon/ or my_addon_1.1/, it doesn't actually matter, because you never directly refer to it.

When you create the add-on, you give it an ID, that has the author and add-on name, e.g. Arantor:WedgeDesk, and it's *this* that's used. Everything that you do with an add-on will make use of this ID.

Let's say you have a big add-on that has multiple source files and multiple template files, arranged in src/ and tpl/ respectively, and just live in your add-on's folder. I'll use WedgeDesk as an example.

I want to load the main source: (The .php is added automatically.)
Code: [Select]
loadAddonSource('Arantor:WedgeDesk', 'src/WedgeDesk');

It doesn't matter what folder it's in, you never have to worry about it. Similar deal with loading templates and language files:
Code: [Select]
loadAddonTemplate('Arantor:WedgeDesk', 'tpl/WedgeDesk'); // loads tpl/WedgeDesk.template.php
loadAddonLanguage('Arantor:WedgeDesk', 'lang/WedgeDesk'); // loads the right file from lang/WedgeDesk.*.php

No more dumping files throughout the different folders - everything's in one place.

There is a caveat at this point: it does mean that an add-on can't magically have different templates for different themes and just drop them into the relevant folders. That's not to say that it can't be done, it just means having the multiple templates stored inside the add-on and making sure that the theme is checked before calling loadAddonTemplate.

There is more, since there's functions for loading CSS files from your add-on, plus JavaScript files, and in both cases they must be called so that the proper caching and substitution comes in.

Lastly, you may be wondering how you insert images, if you're not using $settings['images_url'] or $settings['default_images_url'], well, there's a method for that too.

Specifically, if you ever need to refer to an add-on by URL, the base URL (i.e. www.example.com/Addons/youraddon) is available in $context['addons_url']['addonid'], so I can always call WedgeDesk images through $context['addons_url']['Arantor:WedgeDesk'].

There's more, too. If an add-on uses only hooked files (the recommended method), when an error occurs, it's possible to attribute that error to that add-on in the error log, so you'll know if an add-on is acting up.

There's plenty more to do, of course, like provide facilities in the language editor for editing the language files in add-ons, but for now at least, there's an awful lot of work done to facilitate strong add-on support. There's also file editing to provide, plus quite a bit of the DB support, but it is coming along nicely now.

(Oh, and one last thing. If you rename an add-on's folder to something different, or just delete it, it should automatically disable. If it uses file edits, all bets are off, of course, but that's discouraged anyway.)
 1. This is why I've spent a lot of the last week converting SimpleDesk to run on WedgeDesk, wherein it has been renamed WedgeDesk. A good deal of this stuff only came to light when actually trying to use it in a real and practical sense. I've now done that, and altered the design accordingly.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial


Re: The saga of the Add-on Manager
« Reply #2, on September 13th, 2011, 07:16 PM »
Freudian slip :P But if that's the only critique, I did really well :D[1]
 1. Though it's not on SVN yet, might get more critique then... :whistle:

Re: The saga of the Add-on Manager
« Reply #3, on September 13th, 2011, 07:24 PM »
Quote
Though it's not on SVN yet, might get more critique then... :whistle:
Depends on if I actually try it out, which I probably will, if only to see what happens when I try to port my mods and break Wedge. ;) I have a feeling that your next commit'll introduce some new hooks to play around with :D

Re: The saga of the Add-on Manager
« Reply #4, on September 13th, 2011, 07:30 PM »
Yes, it does add a few new hooks, but the fact that a hook can actually load a file automagically will help.

But I'll be interested to know also what hooks will be desirable to add as well.


Re: The saga of the Add-on Manager
« Reply #7, on September 19th, 2011, 01:34 AM »
OK, as a heads-up in case you didn't see it in the revision log: the first version of this has been committed. It's capable of enabling any add-on in the addons folder, provided that it only does simple DB changes (new tables of its own, updating those tables only, and not adding columns/indexes to tables that aren't its own), hooks, scheduled tasks and settings.

As of right now I'm about to test enable/disable scripts (much like SMF's code and database blocks in the package-info.xml file, with the difference that they should only be for very specialised jobs, like inserting rows into tables, or doing complex install-time logic)

After that, I'm going to work on the ability to add arbitrary columns and rows to existing tables (like a new column in the members table, for example)

I am, I have to say, quite proud of the updating code here; when you call create_table for a table that isn't one of Wedge's own (i.e. from an add-on) and the table already exists, it tries to manipulate the table to add new columns and indexes[1] so that the table is updated. The cool part is that it does it all in a single statement per table.

While that might not seem important, it can be if there's multiple columns to add: the way MySQL handles an ALTER TABLE to add columns is that it will perform a temporary table creation, with the new schema, and copy the existing data into it as it goes - then switch the tables around after it's done.

On top of that, consider also that the setup may not only require 2x the space (for the old and new tables) but potentially a third copy, and while it's doing all that file I/O, it is hampering the possibly DB queries it could be serving instead, so doing it once if possible is much more efficient in the long run.

Anyway, that's where I'm up to with it so far, and I'm planning on building a few very simple demo plugins, not so much for download and distribution (after all, they won't be any use to you without Wedge!) but for those with access to the source to see how they work and all the stuff that goes on, because there is a lot of it behind the scenes.
 1. It won't add or change primary or unique keys. If you ever need to do that for any reason, apart from the fact it generally means your design was flawed... do it in the enable script, that's partly what it's there for.
Re: The saga of the Add-on Manager
« Reply #8, on September 23rd, 2011, 11:37 PM »
So, a progress report.

Enable/disable-time scripts now work, meaning that it's possible to install big and scary add-ons, and even work-around (badly) certain limitations in the add-on manager that will be phased out in due course.[1]

So, I already shared these behind closed doors, but I think I should share them here too.



Read the descriptions, there's more discussion there on the meat of the images.

Right now, there are three add-ons that I'm aware of for Wedge, one is a simple demo that proves what it does, one is WedgeDesk and there's a third, that was posted today as a WIP. I'm excited because it shows the potential of the system very neatly, but also because it's the first add-on that wasn't written by me.

It makes me feel warm inside because it's proof in some way that I'm not just living in a fantasy land of how wonderful writing Wedge add-ons will be,[2] and I have made some UI changes based on things that came out of that add-on, namely reminding me that certain things weren't present and making it easier for authors to provide contact details for their add-ons.

(Even though I have a certain amount of reputation for being an immovable force when it comes to what I think is best, I'm really not. Just have to give me a good enough reason for implementing something.)
 1. I plan to offer adding columns and indexes from the add-on manager itself so it can clean that up on uninstall automatically. Right now that can't be done, because adding new columns has to be done manually - so too would removing them.
 2. That's something I do regret a little about SimpleDesk. I loved the fact that it had a usable plugin system, but that it was built with the relative certainty that I'd be the only person writing for it, which always made me feel a little sad inside.

Re: The saga of the Add-on Manager
« Reply #9, on September 24th, 2011, 12:27 AM »
Dont worry Pete. Your system is perfection and worth the commitment. Well, it would be if it were called a plugin system :lol: just kidding.

Re: The saga of the Add-on Manager
« Reply #10, on September 24th, 2011, 12:29 AM »
It's not perfection - but it is getting there :P

And seriously, when it's done, if you want to go and rename it, I won't mind. Much. :P

I have to admit, I'm quite pleased with the feel of the remove page, even if it's a bit bald, but to be honest, if it were any prettier, it would neutralise some of the gravity of the page.

Re: The saga of the Add-on Manager
« Reply #11, on September 24th, 2011, 01:09 AM »
I wouldn't remind it. The name was popular amongst English speakers. Meaning it should be used in the English version. I use extension in French and that's nice enough. Other languages might use plugin...

Re: The saga of the Add-on Manager
« Reply #12, on September 24th, 2011, 06:33 AM »
So I had a silly little idea come to mind about the name of Add-On in Wedge. The idea was that there could be some pun on the name Wedge that can be turned into a form of Widget. Like Wedget or Wedgit (as a pun on Wedge-it).

Sure, you get the problem of having to have people figure out what a Wedget represents as opposed to the more universal title of Add-On, but I thought that it was rather amusing.

Re: The saga of the Add-on Manager
« Reply #13, on September 24th, 2011, 10:37 AM »
I already suggested wedget as a joke some time ago. Someone was enthusiastic about it -- but I wasn't ;)

My main issue with add-on is that one is supposed to use the hyphen but in theory it's not always used.