Wedge

This is mostly a descriptive post, of the mechanics of the add-on manager. If you're not a programmer, it won't interest you at all, but if you are, particularly if you ever plan on writing an plugin for Wedge, it will be of use to you.

NOTE: Some of the mechanics are not yet implemented, but this should serve as the reference for its mechanics.

Also note: plugin authors are not required to understand the inner workings of ManagePlugins.php, in order to write add-ons.


So, what is a plugin? How do they work?

At its simplest, a plugin is a folder within the /plugins/ folder. Inside that folder is a file named plugin-info.xml, and one or more files that relate to that plugin. Some of them will be code, some of them perhaps images or other resources, but at least one should be code to be loaded and executed at appropriate points in the runtime of Wedge.

This forms the underpinning of Wedge's plugin system: hooks. There are points in the code, given named references, which routines in your plugins can hook onto, and state that they want to be executed when that code is.

For example, there is a hook in the board index code, where the list of boards is prepared. Any function that wants to be called at that point will be called, and the list of boards that the user should see will be made available to those functions, and can be modified there. There are many more hooks, scattered throughout the source of Wedge, and likely more will emerge over time.

At its simplest, a plugin need only contain a single file containing a single function, and the appropriate hook to call. (The hook will not only run the function, but make sure to load the file first)


An example
Here's about the simplest possible plugin that can be written. It contains two files, the plugin-info.xml and a PHP file, demo_plugin.php.

plugin-info.xml looks like this:

And demo_plugin.php looks like this:

When unpacked, both files live in /plugins/demo_plugin/.

Now, the Plugin Manager will show this as an add-on to be enabled; it'll provide the name and description to the user, with an enable button, to allow them to enable it.

When enabled, the code is made aware that when handling the main menu (wherein there is a hook called menu_items), it should load $plugindir/demo_plugin.php (note the .php is added automatically), and run demo_function().

Exactly what a given hook will make available to a plugin will be hook specific, but more on this once the list of hooks is completed.

That's really the foundation of how to build an add-on for Wedge: the plugin-info.xml states what should be done when, and one or more extra files contains the rest of the functions to run.

Vastly more powerful plugins can be constructed out of this basic framework, and even just with this toolset, a lot is possible, but if I were to leave it just at that stage, I would be remiss in my developer duties.


More advanced hook setup

Firstly, an extract of the <hooks> block from WedgeDesk's installer:


I haven't quite finished renaming all the files (though I have done most of them). In this case, when we display a topic, there's a hook called display_buttons, which is for modifying the list of buttons that apply to a given topic (e.g. reply, print page, and SD/WD add one for moving topics to the helpdesk), which is what's happening there. There's no file to load, because I know that Subs-WedgeDesk.php will have already been loaded, and that it contains that function.

Then, we have this odd type of hook: a language hook. Normally, you'll define functions to be called, but in certain cases, you might simply just need to load a language file, which is exactly what the language hooks do. Most importantly, they're in the help pop-up and Who's Online pages, where add-ons very typically need to add their contents.

Note: There is no more Modifications.english.php file to modify. Things you'd normally add to Who's Online there, or to Who.language.php directly, just dump into a new file and call through the language hook.

Then there's this really odd thing: provides. This is for when plugins actually declare their *own* hooks. You may have noticed there isn't a version check, and that's because the design generally doesn't need one.

The vast majority of plugins are expected to use the hooks provided, and in which case, they don't generally have to care what version of Wedge they're running with. All they need to know is if the hook is available or not. If it's not, generally you won't be able to install it (because it checks the list of hooks you want to use, against the list it knows it has)

The point of provided hooks like shd_hook_init is that plugins might want to extend other add-ons. I fully envisage writing WedgeDesk plugins and making use of those hooks to do so.

There is one other thing I haven't mentioned: optional hooks. Simply add optional="yes" to the <function> item, and if the hook isn't available, it won't prevent installation, but it will still set it up as if it were. The point of this is if you create a plugin that can work with another, but doesn't *need* it to be installed.


Settings

A fair number of mods don't need much but they do depend on settings existing and having default values. Well behaved mods will check on use with empty(), and some others[1] will go away and create the settings in the database if they don't already exist.

Well, in the setup you can declare the settings your plugin uses. It also means that when we get on to clean-up and uninstallation, the settings are cleaned up to the user's request automatically.


Very simple structure: you just create a <settings> block that contains all the settings you want to use, and then a <setting> per actual setting. Note that it won't create the setting if it already exists in the system, but if it does already exist it will use the stated default. (You do need to state a default value.)


Readmes

Everyone likes readme files. The setup in the Plugin Manager allows you to provide readmes, and do so in a fashion that allows for multiple languages.

Simply add a block like so:

As long as you put $plugindir at the front, and the file actually exists, it'll be provided to the user when they click on it.

More specifically, it looks through the <readme> items for languages installed by the user, so if English and French are installed and an plugin supports more, only English and French will be shown, and there will be a little flag icon for them to click on. Remember: this is all pretty much automatically handled on your behalf: you just need to provide the readme files in the add-on and link to them in the plugin-info.xml file.


More hokey magic: scheduled tasks

I don't think this is something that's going to come up that often, but it's convoluted enough to do manually that I wanted to make it easier.

Creating a scheduled task in the system is as simple as adding this block:

One task per <task> block, and it should be fairly obvious that again, you're indicating how often the task should run, what function to call and a file to load that contains that function.

(NB: Right now the task won't receive a name in the admin panel properly, I haven't yet quite decided how I want to fix that, but rest assured, I'll provide examples once I figure it out.)


Lastly, database changes

This is the big kahuna. If you've used SMF's $smcFunc['db_create_table'] structure, you'll recognise this because it's mostly the same.

Before those who tell me they hate XML or whatever come in: there's a very good reason it's done this way. It means that when it comes around to removing the plugin, the manager can cleanly deal with it properly, something that didn't always happen where installer scripts were left to their own devices to manage tables.

Another example from WedgeDesk showing what goes on:

It's a very shortened example but it gives you pretty much everything you need to know.

So it looks to create {db_prefix}helpdesk_tickets (wedge_helpdesk_tickets, then), and this sample has two columns. It should be fairly clear what's going on here, but notice the lack of 'size' being stated. You can, if you want, state the size, but for numeric types, the size will be calculated for you if you don't state it (for example, mediumint always ends up being mediumint(8) if not stated because that's the range mediumint covers[2])

The range of types is increased too, you can have the named int types (tinyint, smallint, mediumint, int, bigint), plus float types (float, real, double), string types (char, varchar)[3], block string types (text, mediumtext)[4] and bitwise types (set, enum)[5]

The other notable thing here is that you just specify the tables. If the structure is different to what's in the current table (e.g. you're adding new features to a mod), the existing table will get the new/changed columns updated. It will NOT remove any existing columns, however.

It will also add any new indexes that you state, but it will not alter or remove primary or unique indexes. This is to protect the integrity of your data; if you need to do this, you can manage it during the enable script, which I'll get into in a moment.[6]

You'll notice the <scripts> block. None of them are required, but you might find it useful to add them if necessary.

These are scripts which run at the appointed time to make any changes that can't be readily automated, especially if they don't occur on a decent number of plugins, or aren't particularly standardised.

WedgeDesk, for example, uses this functionality to make sure that a department exists after installation, when the table may or may not exist, and the table may or may not have a department created even if the table did already exist.

<enable> runs during the time of an add-on being enabled, <disable> when it is being disabled[7], <remove> for when an add-on is being removed, but that data should be kept, and <remove-clean> for when data should be removed as well.

There are also plans to add functionality for adding individual columns and indexes to tables. For the most part the XML looks the same, just it's contained in the <database> tag rather than in a specific <table> tag (there's a <columns> container at <database> level), and each <column> also states the name it should be in, but otherwise it's the same. Ditto for <indexes>.


Phew, so what actually happens when a plugin is enabled?

(Yes, we're almost done!) Enabling an add-on is reasonably straightforward.

The list of operations that is carried out is as follows:
* Verify that the plugin-info.xml file exists and makes some kind of sense.
* Check that any requirements stated in the file are met, in particular hooks. More on this at the end, because some of it is so rarely going to be needed, I haven't bothered to cover it just yet.
* Look at the contents of the <database> tag, whether there are any changes to make. If any are specified, the order of operations is: new/updating tables first, then new columns, then new indexes.
* Run an enable script if specified.
* Check if any settings are stated, whether they already exist and if not, add them to $modSettings.
* Check if any scheduled tasks are stated, whether they already exist, if not add them, otherwise update them.

Lastly, we handle hooks. The process is fairly complex at this point, so if you're really interested, this is the part to pay attention to.

A plugin will have a $folder and an $id. The folder is the physical folder name it has within the /plugins/ folder, and its id is stated in the plugin-info.xml file, right up there in the <plugin> tag right at the top. The distinction is quite important: the id is under the control of the plugin itself, but the folder might not be.

I've seen all kinds of weird cases where folders get mashed by users, sometimes intentionally, sometimes not. So the folder name is made available to the add-on to make use of, and it's always reference-able from its id, assuming it's enabled.

The exact process that happens inside the manager is as follows:
* an array is built, it contains firstly a key 'id' that contains the plugin's id.
* the remainder of the array is simply the list of hooks it refers to, one item per hook, in the format of function-name|file-to-load|plugin (this last is a literal, and helps the hook code identify that it is an plugin that has to be dealt with rather than a hook used otherwise for integration)

This array is stored, serialized, as $modSettings['plugin_' . $folder], and the $folder will be added to $modSettings['enabled_plugins']. (This is simply a string variable containing the folder names of all enabled add-ons, separated by commas)

Note, very importantly, that hooks are not saved to the master hook references (i.e. $modSettings['registered_hooks']) and that except under very specific circumstances, they should not be, either.

This is because during run-time, specifically after $modSettings is loaded, this is when hooks are initialised, and the contents the plugin_* entries are transferred into the hook references. In case a user does something like directly delete an add-on folder, the intention is that it should safely disable the plugin, instead of breaking everything.

During init, $modSettings['enabled_plugins'] is exploded, and each named folder is examined to check it exists, and that there is an plugin-info.xml file in that folder. If so (and only if so), the hooks are engaged, and from that point on, the plugin is enabled and will be called by the system as and when it has indicated it would need to do so.

Three variables are also made available to the system at this point:

$context['enabled_plugins'] - this is an array of all enabled plugins. Unlike its $modSettings counterpart, the key of the array is the plugin's id, and its value is the folder inside /plugins/, so you can always identify where a given plugin should be from that. (And, by proxy, you can also identify which plugins are available.)

$context['plugins_dir'] - another key/value array, this time the key is the plugin's id, and its value is the physical resolved path to that plugin's folder, e.g. /home/myuser/wedge/plugis/myplugin. This is needed so that source files, templates and language files can be loaded.

$context['plugins_url'] - another key/value array, much like plugins_dir, but the URL counterpart. This will allow you to reference your plugin for images in HTML.


Now I have my add-on, what happens when I want to do big stuff with it?

While, yes, small plugins might only be a single small, self contained file, it's highly likely that you're going to want to do bigger plugins sometime, and that means you're going to need to integrate other files, like loading other source files from the plugin directory, or templates, or language files.

It just so happens, hah, that there are functions designed just for this purpose, to make it easier to handle loading things and to abstract away a lot of the underlying mechanics.

I have mentioned these before, but no matter, let's cover them again.

loadPluginSource($plugin_name, $source_name)
 - takes the plugin's id and the file within to load, relative to the base of the plugin itself. If you have an add-on that doesn't have subfolders, just use the file's base name, e.g. MyPluginFile (the .php will be added, as will the rest of the path), as this is analogous to how loadSource works in the rest of Wedge.[8]

loadPluginTemplate($plugin_name, $template_name, $fatal = true)
 - much like loadTemplate generally, you specify the plugin's id, the template name (without .template.php) and whether it should fatal-error if the file couldn't be found. Much like loadPluginSource, specify a path relative to the plugin's folder if you need it, or not if you don't. If you keep templates in a subfolder, like tpl/, then just use tpl/PluginTemplate, or whatever you're using.

loadPluginLanguage($plugin_name, $template_name, $lang = '', $fatal = true, $force_reload = false)
 - used for loading language files. You probably get the idea how to call this by now, and in all respects other than the fact it uses the plugin's directory, it works much as loadLanguage does: you specify the language you want to use and if that's not English, it'll attempt to load that language, falling back on English if necessary.


What about CSS and JavaScript?

While I mentioned $context['plugins_url'] for images, it's generally recommended to *not* use that to load CSS and JavaScript for add-ons. Remember: Wedge has minifiers and similar tools built in that help save bandwidth for both CSS and JavaScript, and really these should be used in preference to adding it manually.

There is a pair of functions for just this purpose, which you can call from your own code.


The structure should be fairly obvious - like everything else, it requires the plugin's id, and the file relative to the plugin's own folder. (WedgeDesk has a css/ and js/ folder)

Notice also that the JS is the only one that actually uses a .js extension, everything else does not. In case you're wondering, the 'true' is a reference to add_css_file, and it means whether or not to include the cached CSS file into the header or not, if true, add it to the header and deal with it automatically (recommended for add-ons), false means to simply return the URL to it.


Anything else of interest?

There is also one last item of general interest in the specification: <acp-url>. In there, you simply put in the part of the URL after index.php? that your add-on's settings live in, e.g. <acp-url>action=admin;area=wedgedesk_info</acp-url>. That way, if your add-on has settings, you can direct users to it, and if not (and not all will), you can leave it omitted.

I think that's pretty much everything about how it behaves.


Hang on, you said about file edits.

I did originally think about adding them, but in the end, I came to the conclusion that it's just too unreliable to allow and that while it might hinder pushing for that very peak of performance, and it does cut out some functionality possibilities, it makes life safer for everyone else, and that is more important to me.


Right now, at least, that really is everything!

I forgot to mention a couple of things. There are a few refinements in the error log system and in the language editor.

Error logging

Since we have a list of all the folders that plugins live in, and we know the file an error occurs in, we can check to see if that file is in one of the plugin folders we know we have - so if an add-on causes an error, it's normally possible to trace it back to that plugin.

Language editor

All the language files (at least, anything matching *.english.php when using English, and similarly for other languages) in a given plugin are shown in the language editor underneath the normal list of strings in themes, so you can edit the language strings for any plugin you have installed.

1.	Depending on your perspective, this may be less or more well behaved.
2.	0 to 16777215 (8 digits for unsigned) vs -8388608 (8 digits for signed) to 8388607, if you're wondering. The others are all worked out for you, for tinyint, smallint, mediumint, int and bigint.
3.	If size isn't given, it defaults to 50 characters.
4.	Naturally, no size is applicable for these, and it won't let you set a default for them either.
5.	You need to add a values attribute, e.g. values="'1','2','3'", with the values as single quoted with commas. It's not a big ask, seeing how it's the same that phpMyAdmin asks of you, and it's not like you're going to do it all that often.
6.	This is a functional improvement over SMF; 2.0 RC3 featured the ability to add new columns to existing tables, just off the create_table call, but didn't touch indexes, by the time final shipped, this was removed, so mod authors were expected to manage any changes themselves as opposed to SMF doing it for them. I not only reinstated the ability to make such changes, I added index changes, and made it possible for the schema changes to be carried out in a single ALTER TABLE statement for each table, rather than one ALTER TABLE per operation that needed to be done. Thus the total number of file operations that involve creating one or more duplicates of the table is limited to one per table.
7.	Think of 'uninstall' in relation to SMF mods, and you have the right idea, but it is typically a disable here rather than undoing code edits.
8.	You shouldn't, generally, need to reference $sourcedir or $pluginsdir manually, which is why loadSource and loadPluginSource even exist.

For years, I've held an account at my first proper paid webhosting service, and they've run my email ever since - that's since 2003.

When I joined them, everything was fantastic, service was prompt and I couldn't recommend them enough. But over the last couple of years they've really gone downhill, and today was the final straw.

For 24 hours straight, the email server's been down, the web-mail login is broken because they didn't set it up properly, that is assuming I'm willing to overlook the fact that they're using self-signed certificates for all their servers... I say servers, but I think they only have one server these days.

Since my old old old site is virtually non existent anyway, I'm now in the process of migrating the email to another provider, and moving the website to my VPS. Given the hoops I've had to jump through to get my old email archive given that their mail service is broken, I can see today is going to be a very, very long day :(

Someone PM'd me about http://www.simplemachines.org/community/index.php?topic=451766.0 and the incorrect header being sent.

It's a valid bug, actually, in SMF, where the header varies depending on browser. I'm happy to report that we don't really do that, and that the rogue header isn't sent like that in Wedge.

We actually modified this behaviour months ago to send one header for IE because IE needs a slightly different header, and one for everyone else - which works, and isn't in the format that causes FF nightlies to fail.

As I work on the plugin/add-on manager, I'm acutely aware that I'm going to be writing a whole lot of mods for it. Notably I'll be porting a bunch of mine from SMF, at least what I didn't include in the core.

And, you know me, I'm already planning ahead as to how best to provide for those mods, in terms of support resources, in terms of availability and so on.

This does, naturally, have wider implications for Wedge too; whether we host mods here on wedge.org or not, for example.

There are arguments for and against doing this, namely that there is an implied burden on us for mods made available here, even to the point of assuming liability for mods.

At the same time, I'm also mindful of the fact that I'm going to be providing mods that aren't free and will also need somewhere to support those too, if nothing else that's going to include WedgeDesk.[1]

So, I'm curious, what do people think?

There are a few things I'd like people to think about:
* hosting on wedge.org or a dedicated site with its own domain, the pros and cons attached to that
* mods being free/paid, and whether such sites should cater to either/both[2]
* should mods be vetted prior to being publicly available
* what resources both mod users and mod authors need in being able to use mods and any support issues that arise

Please, be as thorough as you can because that all reflects on what I get set up (since, ultimately, I'm going to be the one writing the mod site, and whether it ends up being a bolt-on to Aeva, or something more substantial or not) and the more detail means that we can get something together that accurately reflects the needs and desires of the community, rather than cobble something together that doesn't really work that well.

Caveat: I do already have certain ideas in my head already for how all these things can be attended to. I want to know, as far as possible without biasing anyone, whether my ideas are spot-on or wildly off base.

All I will say is that mod support should, theoretically, be easier due to fewer support issues and version bumps.

1.	I have been making a to-do list for it, which I'm slightly bothered to note has reached 29 items, most of which are features I want to implement, rather than bugs, though there are some bugs I've noted that I want to fix.
2.	There are some other matters that are relevant, such as who is legally liable in the event of trouble, but I suspect that can be worked around.

I read a bunch of comics online, but somehow I never found Not Invented Here until today.

In particular, http://notinventedhe.re/on/2010-1-28 struck me as relevant, because it reminds me of a certain platform's approach to a certain administrative area that may possibly need some rethinking :whistle:

I bumped into http://www.simplemachines.org/community/index.php?topic=447676.0 today, makes interesting reading.

Now, we haven't reproduced that header in every file. Apart from the fact it's bloaty, I'm not convinced we need it in full like that, but I DO get the impression that our current header will be seen as bending[1] the rules of the licence, which says:Meanwhile our header is:

Note that while we mention SMF 2.0 in the credits page, our licence document (even the one on wedge.org) doesn't and nowhere do we actually credit their copyright, so on strict letter of the licence, we're not compliant. I know it's petty, and I'd like to think they wouldn't kick off about it, but I think it's clear they probably would and I'm not sure I'd call them wrong for it on this one.

We don't need to have vbgamer's full verbosity:

But I think we can suitably get away with it thus:

One extra line in the header per file and then we're acknowledging their copyright and their licence. Thoughts?

1.	Or breaking them entirely, but I'm ever the optimist here.

I'm aware that I haven't shown too many pictures of the being-revised admin panel thus far but you can see from the changelog that I'm quite keen on cleaning things up and removing options.

I actually believe that, to a point, it is possible to have too much flexibility and that a certain realm of 'this is what you get' is not inherently a bad thing.

Some of the options I've removed I did just remove, some I put up for debate. Here's some more that I'm putting up for debate.

* Enable shorthand date display (for the Today / Today + Yesterday / always just do the date) option

I'm not convinced this needs to be optional, per se. I don't think it would make it any more meaningful for it to be purely optional than it would for it to be permanently set to 'Today + Yesterday'. (Those who want purely relative dates are still going to want more thoroughly relative dates, of course, a setup that is not accounted for currently and is probably not going to be present in the core itself)

How many admins actually change it from the default?

* Members per page in member list

Again, how many admins change it? 30's a nice number, for quite a few reasons; it strikes a balance between limiting how many you're throwing at the member list queries, and again I don't think many admins ever change it because it's a nice amount, not too few, not too many. (Particularly if we do what is in place here, and add the avatar in)

* Show online/offline in posts and PMs

This is off by default, and I have yet to understand why. It's the option which triggers the little green box next to a user's name if they're online, or grey if they're hidden/offline. Seems to me that it's a no-brainer to have it on, and if the theme decides to change it, that's up to the theme, not the core to configure it.

* Disable administration security

I don't like this option as a rule. The only time I ever find myself turning it off is on a localhost machine where I have total control over it anyway.

What I'd probably do here is remove the UI item, but leave the hidden setting in place so that I could still configure it myself on a localhost through a small tweak in phpMyAdmin. It isn't the sort of thing that needs to be left hanging around normally.

* Enable reporting of personal messages

Is there any time you *don't* want PMs to be able to be reported? Should it be a permission?

I'm actually just inclined to remove the option and force it so PMs can always be reported. If it's a problem you can just restrict people from sending PMs, and of course there's always the ignore list.


Thoughts?