Wedge

Public area => The Pub => Plugins => Topic started by: Arantor on September 20th, 2011, 03:01 AM

Title: Documentation for plugin authors
Post by: Arantor on September 20th, 2011, 03:01 AM
This is mostly a descriptive post, of the mechanics of the add-on manager. If you're not a programmer, it won't interest you at all, but if you are, particularly if you ever plan on writing an plugin for Wedge, it will be of use to you.

NOTE: Some of the mechanics are not yet implemented, but this should serve as the reference for its mechanics.

Also note: plugin authors are not required to understand the inner workings of ManagePlugins.php, in order to write add-ons.


So, what is a plugin? How do they work?

At its simplest, a plugin is a folder within the /plugins/ folder. Inside that folder is a file named plugin-info.xml, and one or more files that relate to that plugin. Some of them will be code, some of them perhaps images or other resources, but at least one should be code to be loaded and executed at appropriate points in the runtime of Wedge.

This forms the underpinning of Wedge's plugin system: hooks. There are points in the code, given named references, which routines in your plugins can hook onto, and state that they want to be executed when that code is.

For example, there is a hook in the board index code, where the list of boards is prepared. Any function that wants to be called at that point will be called, and the list of boards that the user should see will be made available to those functions, and can be modified there. There are many more hooks, scattered throughout the source of Wedge, and likely more will emerge over time.

At its simplest, a plugin need only contain a single file containing a single function, and the appropriate hook to call. (The hook will not only run the function, but make sure to load the file first)


An example
Here's about the simplest possible plugin that can be written. It contains two files, the plugin-info.xml and a PHP file, demo_plugin.php.

plugin-info.xml looks like this:
Code: [Select]
<?xml version="1.0" standalone="yes" ?>
<plugin id="Arantor:DemoPlugin">
<name>Demo Plugin</name>
<author>Arantor</author>
<description>Changes the "Home" button on the menu to read "My Home Page"</description>
<version>1.0</version>
<hooks>
<function point="menu_items" function="demo_function" filename="$plugindir/demo_plugin" />
</hooks>
</plugin>

And demo_plugin.php looks like this:
Code: [Select]
<?php

function demo_function(&$items)
{
$items['home']['title'] = 'My Home Page';
}

?>

When unpacked, both files live in /plugins/demo_plugin/.

Now, the Plugin Manager will show this as an add-on to be enabled; it'll provide the name and description to the user, with an enable button, to allow them to enable it.

When enabled, the code is made aware that when handling the main menu (wherein there is a hook called menu_items), it should load $plugindir/demo_plugin.php (note the .php is added automatically), and run demo_function().

Exactly what a given hook will make available to a plugin will be hook specific, but more on this once the list of hooks is completed.

That's really the foundation of how to build an add-on for Wedge: the plugin-info.xml states what should be done when, and one or more extra files contains the rest of the functions to run.

Vastly more powerful plugins can be constructed out of this basic framework, and even just with this toolset, a lot is possible, but if I were to leave it just at that stage, I would be remiss in my developer duties.


More advanced hook setup

Firstly, an extract of the <hooks> block from WedgeDesk's installer:

Code: [Select]
<hooks>
<function point="display_buttons" function="shd_display_btn_mvtopic" />

<language point="lang_who" filename="$plugindir/lang/SimpleDeskWho" />

<provides>
<hook type="function">shd_hook_init</hook>
</provides>
</hooks>

I haven't quite finished renaming all the files (though I have done most of them). In this case, when we display a topic, there's a hook called display_buttons, which is for modifying the list of buttons that apply to a given topic (e.g. reply, print page, and SD/WD add one for moving topics to the helpdesk), which is what's happening there. There's no file to load, because I know that Subs-WedgeDesk.php will have already been loaded, and that it contains that function.

Then, we have this odd type of hook: a language hook. Normally, you'll define functions to be called, but in certain cases, you might simply just need to load a language file, which is exactly what the language hooks do. Most importantly, they're in the help pop-up and Who's Online pages, where add-ons very typically need to add their contents.

Note: There is no more Modifications.english.php file to modify. Things you'd normally add to Who's Online there, or to Who.language.php directly, just dump into a new file and call through the language hook.

Then there's this really odd thing: provides. This is for when plugins actually declare their *own* hooks. You may have noticed there isn't a version check, and that's because the design generally doesn't need one.

The vast majority of plugins are expected to use the hooks provided, and in which case, they don't generally have to care what version of Wedge they're running with. All they need to know is if the hook is available or not. If it's not, generally you won't be able to install it (because it checks the list of hooks you want to use, against the list it knows it has)

The point of provided hooks like shd_hook_init is that plugins might want to extend other add-ons. I fully envisage writing WedgeDesk plugins and making use of those hooks to do so.

There is one other thing I haven't mentioned: optional hooks. Simply add optional="yes" to the <function> item, and if the hook isn't available, it won't prevent installation, but it will still set it up as if it were. The point of this is if you create a plugin that can work with another, but doesn't *need* it to be installed.


Settings

A fair number of mods don't need much but they do depend on settings existing and having default values. Well behaved mods will check on use with empty(), and some others[1] will go away and create the settings in the database if they don't already exist.

Well, in the setup you can declare the settings your plugin uses. It also means that when we get on to clean-up and uninstallation, the settings are cleaned up to the user's request automatically.

Code: [Select]
<settings>
<setting name="shd_new_search_index" default="0" />
</settings>

Very simple structure: you just create a <settings> block that contains all the settings you want to use, and then a <setting> per actual setting. Note that it won't create the setting if it already exists in the system, but if it does already exist it will use the stated default. (You do need to state a default value.)


Readmes

Everyone likes readme files. The setup in the Plugin Manager allows you to provide readmes, and do so in a fashion that allows for multiple languages.

Simply add a block like so:
Code: [Select]
<readmes>
<readme lang="english">$plugindir/readme/readme.english.txt</readme>
</readmes>

As long as you put $plugindir at the front, and the file actually exists, it'll be provided to the user when they click on it.

More specifically, it looks through the <readme> items for languages installed by the user, so if English and French are installed and an plugin supports more, only English and French will be shown, and there will be a little flag icon for them to click on. Remember: this is all pretty much automatically handled on your behalf: you just need to provide the readme files in the add-on and link to them in the plugin-info.xml file.


More hokey magic: scheduled tasks

I don't think this is something that's going to come up that often, but it's convoluted enough to do manually that I wanted to make it easier.

Creating a scheduled task in the system is as simple as adding this block:
Code: [Select]
<scheduledtasks>
<task runevery="1" runfreq="day" name="shd_scheduled" file="$plugindir/src/WedgeDesk-Scheduled" />
</scheduledtasks>

One task per <task> block, and it should be fairly obvious that again, you're indicating how often the task should run, what function to call and a file to load that contains that function.

(NB: Right now the task won't receive a name in the admin panel properly, I haven't yet quite decided how I want to fix that, but rest assured, I'll provide examples once I figure it out.)


Lastly, database changes

This is the big kahuna. If you've used SMF's $smcFunc['db_create_table'] structure, you'll recognise this because it's mostly the same.

Before those who tell me they hate XML or whatever come in: there's a very good reason it's done this way. It means that when it comes around to removing the plugin, the manager can cleanly deal with it properly, something that didn't always happen where installer scripts were left to their own devices to manage tables.

Another example from WedgeDesk showing what goes on:
Code: [Select]
<database>
<tables>
<table if-exists="update" name="{db_prefix}helpdesk_tickets">
<columns>
<column name="id_ticket" type="mediumint" autoincrement="yes" unsigned="yes" />
<column name="id_dept" type="smallint" unsigned="yes" />
</columns>
<index type="primary">
<field>id_ticket</field>
</index>
</table>
</tables>
<scripts>
<enable>$plugindir/enable.php</enable>
<disable>$plugindir/disable.php</disable>
<remove>$plugindir/remove.php</remove>
<remove-clean>$plugindir/removeclean.php</remove-clean>
</scripts>
</database>

It's a very shortened example but it gives you pretty much everything you need to know.

So it looks to create {db_prefix}helpdesk_tickets (wedge_helpdesk_tickets, then), and this sample has two columns. It should be fairly clear what's going on here, but notice the lack of 'size' being stated. You can, if you want, state the size, but for numeric types, the size will be calculated for you if you don't state it (for example, mediumint always ends up being mediumint(8) if not stated because that's the range mediumint covers[2])

The range of types is increased too, you can have the named int types (tinyint, smallint, mediumint, int, bigint), plus float types (float, real, double), string types (char, varchar)[3], block string types (text, mediumtext)[4] and bitwise types (set, enum)[5]

The other notable thing here is that you just specify the tables. If the structure is different to what's in the current table (e.g. you're adding new features to a mod), the existing table will get the new/changed columns updated. It will NOT remove any existing columns, however.

It will also add any new indexes that you state, but it will not alter or remove primary or unique indexes. This is to protect the integrity of your data; if you need to do this, you can manage it during the enable script, which I'll get into in a moment.[6]

You'll notice the <scripts> block. None of them are required, but you might find it useful to add them if necessary.

These are scripts which run at the appointed time to make any changes that can't be readily automated, especially if they don't occur on a decent number of plugins, or aren't particularly standardised.

WedgeDesk, for example, uses this functionality to make sure that a department exists after installation, when the table may or may not exist, and the table may or may not have a department created even if the table did already exist.

<enable> runs during the time of an add-on being enabled, <disable> when it is being disabled[7], <remove> for when an add-on is being removed, but that data should be kept, and <remove-clean> for when data should be removed as well.

There are also plans to add functionality for adding individual columns and indexes to tables. For the most part the XML looks the same, just it's contained in the <database> tag rather than in a specific <table> tag (there's a <columns> container at <database> level), and each <column> also states the name it should be in, but otherwise it's the same. Ditto for <indexes>.


Phew, so what actually happens when a plugin is enabled?

(Yes, we're almost done!) Enabling an add-on is reasonably straightforward.

The list of operations that is carried out is as follows:
* Verify that the plugin-info.xml file exists and makes some kind of sense.
* Check that any requirements stated in the file are met, in particular hooks. More on this at the end, because some of it is so rarely going to be needed, I haven't bothered to cover it just yet.
* Look at the contents of the <database> tag, whether there are any changes to make. If any are specified, the order of operations is: new/updating tables first, then new columns, then new indexes.
* Run an enable script if specified.
* Check if any settings are stated, whether they already exist and if not, add them to $modSettings.
* Check if any scheduled tasks are stated, whether they already exist, if not add them, otherwise update them.

Lastly, we handle hooks. The process is fairly complex at this point, so if you're really interested, this is the part to pay attention to.

A plugin will have a $folder and an $id. The folder is the physical folder name it has within the /plugins/ folder, and its id is stated in the plugin-info.xml file, right up there in the <plugin> tag right at the top. The distinction is quite important: the id is under the control of the plugin itself, but the folder might not be.

I've seen all kinds of weird cases where folders get mashed by users, sometimes intentionally, sometimes not. So the folder name is made available to the add-on to make use of, and it's always reference-able from its id, assuming it's enabled.

The exact process that happens inside the manager is as follows:
* an array is built, it contains firstly a key 'id' that contains the plugin's id.
* the remainder of the array is simply the list of hooks it refers to, one item per hook, in the format of function-name|file-to-load|plugin (this last is a literal, and helps the hook code identify that it is an plugin that has to be dealt with rather than a hook used otherwise for integration)

This array is stored, serialized, as $modSettings['plugin_' . $folder], and the $folder will be added to $modSettings['enabled_plugins']. (This is simply a string variable containing the folder names of all enabled add-ons, separated by commas)

Note, very importantly, that hooks are not saved to the master hook references (i.e. $modSettings['registered_hooks']) and that except under very specific circumstances, they should not be, either.

This is because during run-time, specifically after $modSettings is loaded, this is when hooks are initialised, and the contents the plugin_* entries are transferred into the hook references. In case a user does something like directly delete an add-on folder, the intention is that it should safely disable the plugin, instead of breaking everything.

During init, $modSettings['enabled_plugins'] is exploded, and each named folder is examined to check it exists, and that there is an plugin-info.xml file in that folder. If so (and only if so), the hooks are engaged, and from that point on, the plugin is enabled and will be called by the system as and when it has indicated it would need to do so.

Three variables are also made available to the system at this point:

$context['enabled_plugins'] - this is an array of all enabled plugins. Unlike its $modSettings counterpart, the key of the array is the plugin's id, and its value is the folder inside /plugins/, so you can always identify where a given plugin should be from that. (And, by proxy, you can also identify which plugins are available.)

$context['plugins_dir'] - another key/value array, this time the key is the plugin's id, and its value is the physical resolved path to that plugin's folder, e.g. /home/myuser/wedge/plugis/myplugin. This is needed so that source files, templates and language files can be loaded.

$context['plugins_url'] - another key/value array, much like plugins_dir, but the URL counterpart. This will allow you to reference your plugin for images in HTML.


Now I have my add-on, what happens when I want to do big stuff with it?

While, yes, small plugins might only be a single small, self contained file, it's highly likely that you're going to want to do bigger plugins sometime, and that means you're going to need to integrate other files, like loading other source files from the plugin directory, or templates, or language files.

It just so happens, hah, that there are functions designed just for this purpose, to make it easier to handle loading things and to abstract away a lot of the underlying mechanics.

I have mentioned these before, but no matter, let's cover them again.

loadPluginSource($plugin_name, $source_name)
 - takes the plugin's id and the file within to load, relative to the base of the plugin itself. If you have an add-on that doesn't have subfolders, just use the file's base name, e.g. MyPluginFile (the .php will be added, as will the rest of the path), as this is analogous to how loadSource works in the rest of Wedge.[8]

loadPluginTemplate($plugin_name, $template_name, $fatal = true)
 - much like loadTemplate generally, you specify the plugin's id, the template name (without .template.php) and whether it should fatal-error if the file couldn't be found. Much like loadPluginSource, specify a path relative to the plugin's folder if you need it, or not if you don't. If you keep templates in a subfolder, like tpl/, then just use tpl/PluginTemplate, or whatever you're using.

loadPluginLanguage($plugin_name, $template_name, $lang = '', $fatal = true, $force_reload = false)
 - used for loading language files. You probably get the idea how to call this by now, and in all respects other than the fact it uses the plugin's directory, it works much as loadLanguage does: you specify the language you want to use and if that's not English, it'll attempt to load that language, falling back on English if necessary.


What about CSS and JavaScript?

While I mentioned $context['plugins_url'] for images, it's generally recommended to *not* use that to load CSS and JavaScript for add-ons. Remember: Wedge has minifiers and similar tools built in that help save bandwidth for both CSS and JavaScript, and really these should be used in preference to adding it manually.

There is a pair of functions for just this purpose, which you can call from your own code.

Code: [Select]
add_plugin_css_file('Arantor:WedgeDesk', 'css/helpdesk', true);
add_plugin_js_file('Arantor:WedgeDesk', 'js/helpdesk.js');

The structure should be fairly obvious - like everything else, it requires the plugin's id, and the file relative to the plugin's own folder. (WedgeDesk has a css/ and js/ folder)

Notice also that the JS is the only one that actually uses a .js extension, everything else does not. In case you're wondering, the 'true' is a reference to add_css_file, and it means whether or not to include the cached CSS file into the header or not, if true, add it to the header and deal with it automatically (recommended for add-ons), false means to simply return the URL to it.


Anything else of interest?

There is also one last item of general interest in the specification: <acp-url>. In there, you simply put in the part of the URL after index.php? that your add-on's settings live in, e.g. <acp-url>action=admin;area=wedgedesk_info</acp-url>. That way, if your add-on has settings, you can direct users to it, and if not (and not all will), you can leave it omitted.

I think that's pretty much everything about how it behaves.


Hang on, you said about file edits.

I did originally think about adding them, but in the end, I came to the conclusion that it's just too unreliable to allow and that while it might hinder pushing for that very peak of performance, and it does cut out some functionality possibilities, it makes life safer for everyone else, and that is more important to me.


Right now, at least, that really is everything!
Posted: September 20th, 2011, 02:45 AM

I forgot to mention a couple of things. There are a few refinements in the error log system and in the language editor.

Error logging

Since we have a list of all the folders that plugins live in, and we know the file an error occurs in, we can check to see if that file is in one of the plugin folders we know we have - so if an add-on causes an error, it's normally possible to trace it back to that plugin.

Language editor

All the language files (at least, anything matching *.english.php when using English, and similarly for other languages) in a given plugin are shown in the language editor underneath the normal list of strings in themes, so you can edit the language strings for any plugin you have installed.
 1. Depending on your perspective, this may be less or more well behaved.
 2. 0 to 16777215 (8 digits for unsigned) vs -8388608 (8 digits for signed) to 8388607, if you're wondering. The others are all worked out for you, for tinyint, smallint, mediumint, int and bigint.
 3. If size isn't given, it defaults to 50 characters.
 4. Naturally, no size is applicable for these, and it won't let you set a default for them either.
 5. You need to add a values attribute, e.g. values="'1','2','3'", with the values as single quoted with commas. It's not a big ask, seeing how it's the same that phpMyAdmin asks of you, and it's not like you're going to do it all that often.
 6. This is a functional improvement over SMF; 2.0 RC3 featured the ability to add new columns to existing tables, just off the create_table call, but didn't touch indexes, by the time final shipped, this was removed, so mod authors were expected to manage any changes themselves as opposed to SMF doing it for them. I not only reinstated the ability to make such changes, I added index changes, and made it possible for the schema changes to be carried out in a single ALTER TABLE statement for each table, rather than one ALTER TABLE per operation that needed to be done. Thus the total number of file operations that involve creating one or more duplicates of the table is limited to one per table.
 7. Think of 'uninstall' in relation to SMF mods, and you have the right idea, but it is typically a disable here rather than undoing code edits.
 8. You shouldn't, generally, need to reference $sourcedir or $pluginsdir manually, which is why loadSource and loadPluginSource even exist.
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 01:09 PM
Looks good, how can I check for function dependencies?
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 01:13 PM
Bah, something else I forgot to document.

It's not actually implemented yet but the XML you'd need would be:
Code: [Select]
<required-functions>
<php-function>some_obscure_php_function</php-function>
</required-functions>

Just put in the function name and you're good. (If it's one of those functions where it got enhanced in a later version and you need the latter version's facilities, stick in a PHP version dependency:

Code: [Select]
<min-versions>
<php>5.3</php>
<mysql>5.1</mysql>
</min-versions>

(Both of those are optional, so you can force a minimum MySQL version or a minimum PHP version or both. Version checking on PHP and MySQL is already implemented, though.)
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 01:15 PM
Okay, looks good :). I read the code for the minimum php/mysql versions but couldn't find any mention for function checks. Also, can you extend min-versions to allow checking for other addon's versions?
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 01:17 PM
Yeah, it's not there yet, but it is on my to-do list, like a lot of the functionality here.

I really don't want to get into add-ons having version dependency on each other if at all possible. I'm not sure addons are going to cross-support each other half the time anyway and those that do will almost certainly be using hooks anyway.
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 01:18 PM
Even when using hooks, sometimes version checking might be required. Addons tend to change behaviors for existing hooks, although rare, still quite possible.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 01:21 PM
Add-ons adding their own hooks is going to be rare enough, and frankly if they're going to do that, they really should be renaming it anyway, because that's the only way to guarantee it will be handled properly.

You see, if version checking starts coming into play as a general rule, it's going to get back to hacking version numbers in things to make them work when they inevitably get unmaintained. Been there, done that, want to avoid it. That's why, all the way along, I've been pushing for hook detection and I've been mentioning this caveat since I first said about this direction, since I've always known this would come up.
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 01:22 PM
I know, version checking's a mess. Just a bit of thought I put in.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 01:26 PM
*nods* Yeah, it's a valid concern and one that did need to be reiterated, but that I'm firmly against putting too much  support in for version checking.

In the case of PHP and MySQL, there are some functions whose behaviour varies between versions and can't readily be detected in any other fashion. (I mean, in some cases, you can imply a PHP version through looking for a function that only exists in a given release, but it's not particularly reliable when you need something that changed in, say, 5.3.2 vs 5.3.1)

From my point of view, it's the lesser of two evils, and I felt that the approach I'm adopting (i.e. changing the hook's name if you non-trivially change its behaviour) is a lesser evil than version dependence.

Note that file edits do have a version dependence and that's not likely to go away. Mind you, it's not encouraged anyway so hopefully it won't be a big problem...
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 02:22 PM
Currently it's including the files declared blindly, if an add-on has an invalid file name it'll completely break the forum when the addon is enabled.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 02:28 PM
That's one of the many refinements that can be added, that when setting up the hooks, we test for file existence prior to registration.

I didn't particularly want to add it to loadAddonSource though since I didn't really see a need for it (much as loadSource doesn't)
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 02:54 PM
Currently I cannot modify Wedge's tables without creating a PHP script, anyway that can be worked around? I want to add a few columns to existing tables.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 02:59 PM
It's only because it's not yet written... as stated it will be added.

Meantime, add it manually in a PHP script, loaded through <enable> in the <scripts> inside <database>. I forgot to mention it above, but any of the enable/disable/remove/remove-clean scripts should really do a test for the constant WEDGE_ADDON being declared and exiting promptly if not. (It's only defined in the add-on manager.)

Failing that, they can use SSI if they really want but it's not best-practice IMO.
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 21st, 2011, 03:01 PM
Quote from Arantor on September 21st, 2011, 02:59 PM
It's only because it's not yet written... as stated it will be added.
I was thinking removing the whole check for protected tables would suffice, only having it if the table is being dropped.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 21st, 2011, 03:06 PM
That would work as a temporary measure, but long term, modders who want to add columns to core tables should be doing it on a case by case basis.

The long term reason is what happens when there is cleanup done in the remove-clean scenario, whereupon it will remove tables and columns and so on.
Title: Re: Add-on Manager: Mechanics
Post by: Nao on September 23rd, 2011, 12:26 PM
Now we'll be able to move this to the Wedge documentation and split it into several areas or something because it's probably a bit too monolithic to digest... ;)
Title: Re: Add-on Manager: Mechanics
Post by: dorje on September 30th, 2011, 11:47 AM
Well... I think that it is a great way to manage the add-ons.

You know: I feel my itchy hands cannot wait to write something... :eheh:
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 01:52 PM
:)

Same here, I now have a very large list of plugins I want to write, and over the next few days I'll probably bash a few of them out because the only way I can prove the system works, and spot any holes in it, is to actually thoroughly use it.

I do already foresee one problem: folks who have masses of things installed - it's going to make a very long page, so I might have to do something about that, but that's the sort of thing only experience can teach, as theory only goes so far.

I have to admit, I have browsed through the list of mods for SMF lately, to see the different sorts of things people have done, which has given me a variety of ideas, and already highlighted a couple of weaknesses that can readily be addressed and a couple of things that do need refinement in the system (not specifically plugin related)

I'm going to find it interesting going back over my old mods though, to see how cleanly they can be rewritten ;)
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 03:10 PM
I just realised a weakness in the system that hasn't been accounted for: new actions.

Right now in order to add new actions, a file still has to be loaded (or already have been loaded) to extend the action array. Now, properly complex action juggling is going to have to remain in the hook behaviour (like what WD does, until I add more hooks :D) but for simple hook behaviour there's no need for having to load a file.

What I'm thinking is that there would be an <actions> block:

Code: [Select]
<actions>
   <action action="myaction" function="myfunction" file="$plugindir/MyFile.php" />
</actions>

On a technical level, I'd push that into the per-plugin cache in $modSettings['plugin_' . *] as a new key, and pop it off the stack during the load phase.[1]

It's less of an issue for menu changes because menu changes all require more than a single line of code to define, so loading a file for those isn't the hassle it is for actions.[2]
 1. Yes, even before starting to post, I'd already assessed all the technical implications :P This is more of a pre-flight warning as to what I'm going to implement...
 2. Your choices are to have a tiny file just for the action change plus the action itself as a separate file, minimising parsing but has the file overhead, or to have a single file that contains the action's code as well but then you have to load a larger file every single page load, unnecessarily in most cases.
Title: Re: Add-on Manager: Mechanics
Post by: Nao on September 30th, 2011, 03:19 PM
A template block? You mean using the ui to store non-ui data? I'm sure we could think of something more elegant..?
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 03:22 PM
Quote from Nao on September 30th, 2011, 03:19 PM
A template block? You mean using the ui to store non-ui data? I'm sure we could think of something more elegant..?
Hmm? I is confused.
Title: Re: Add-on Manager: Mechanics
Post by: Nao on September 30th, 2011, 03:25 PM
Oh you mean in plugin-info.XML? You had me confused with blocks ;)
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 03:27 PM
Uh, yeah... you wouldn't store that stuff in the UI anywhere, it's purely in the XML for a given plugin, and avoids having to include a file just to have a function with one line in it.
Title: Re: Add-on Manager: Mechanics
Post by: Nao on September 30th, 2011, 03:41 PM
Sorry for the mixup.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 03:46 PM
No worries - with the mad late-summer heat here it's entirely possible I'd got confused. You know what they say about mad dogs and Englishmen and the mid-day sun, right? We've just had mid-day sun run on for much later in the day at the moment :(
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 06:03 PM
OK, so today I've been working on a DTD for the plugin XML format, to describe what it should contain and in what order.

The real question I've been wrangling with is whether I should enforce the order of things in the file, e.g. <hooks> can contain 1+ function tags, 1+ language tags and 0 or 1 provides tags. Now, in my example files and all my own files, I've always put functions first, then language files, then provides tags.

Now, the parser itself doesn't care too much, it's built to be more forgiving, but I'm thinking that it shouldn't be so forgiving, that having a set process/standard will encourage better behaviour than being more forgiving, so that I'm thinking about the DTD validator being more strict.

So, for example, I wouldn't have it enforce the order of most tags, but stuff like <hooks>, I'd enforce the order like that.

Thoughts?


Disclaimer: there is a secondary reason for doing it like this: it's actually a pain in the arse writing a DTD that has rules where there are mandatory tags and optional tags, and that there isn't a set order because you have to define all the possible permutations.[1]
 1. Like the DTD definition for the HTML head tag, as it has to state a mandatory title tag, an optional base tag and 0+ optional other tags, and that's still a multi-line definition through a %entity replacement.
Title: Re: Add-on Manager: Mechanics
Post by: Dragooon on September 30th, 2011, 06:09 PM
Be an ass, make it ordered!
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on September 30th, 2011, 06:16 PM
Well, the document is very much WIP, so it's not like I can't make it unordered, just I get the impression that if I force it to be in more set orders, it'll encourage authors to be more thorough. Especially if I add validation into the upload/publish process :niark:
Title: Re: Add-on Manager: Mechanics
Post by: Nao on September 30th, 2011, 06:18 PM
+1
Title: Re: Add-on Manager: Mechanics
Post by: live627 on October 1st, 2011, 06:37 PM
Will there also be a XML block for inserting new rows into the database?
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on October 1st, 2011, 06:48 PM
No, because the manager can't know when it should remove them, at least not reliably, since potentially you're talking about any row in any table. Plus, you run the risk of duplicating rows depending on the table in question.

If you need to do so, add a script in the <enable> block and do appropriate clean-up during <disable>, <remove> or <remove-clean>.

WedgeDesk does this when it auto creates a department on install, because the only way it can know whether to add that row is another query first - and that kind of logic can't be represented in a sane fashion in XML.

That said, give me a solid use case for adding it, ideally with a method to automate clean-up, but at least that can safely avoid duplicates, and I'll revisit it (it's not like the plugin manager is finished right now, it's sufficiently done that it's usable, but there's plenty to do)

Note that the two most common tables you'd need to manually add to (settings and scheduled tasks) have defined interfaces for that purpose, and if there's another table that's similarly likely to be updated, I'm happy to include an interface for that, but I'm not enthusiastic about a totally generic row-insertion setup, at least not without refinements to the idea.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on October 3rd, 2011, 03:06 AM
Getting back to the DTD, I found the whole process of trying to set out the DTD fairly frustrating, since I concluded that I didn't want to have to set the order for every little thing, so in the end I started experimenting with RelaxNG instead. Similar idea but you write it in XML instead of some arbitrary format, and while validation is slightly more effort, the specification is easier to maintain IMO.

(See, I discovered that even the different miniature plugins I had didn't really conform to set orders of things, and I found it fairly irritating to have to set even the order of name/description/author/website in that order.)
Posted: October 3rd, 2011, 01:23 AM

I've written the bulk of the RelaxNG spec, just debugging it at the moment, it weighs in at 462 lines :/ - and it only covers about 3/4 of the actual specification.
Title: Re: Add-on Manager: Mechanics
Post by: Nao on October 3rd, 2011, 08:11 AM
Sounds like fun... :-/
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on October 3rd, 2011, 09:05 AM
There's the thing: once it's done, it's done and it can be used to automate testing of plugin-info.XML files.
Title: Re: Add-on Manager: Mechanics
Post by: Arantor on October 3rd, 2011, 11:54 AM
Also, this one's more for Dragooon and live to answer since I'm imagining they're the ones most likely to use it.

We have the required-functions construct. I'm wondering if I shouldn't extend that to classes and methods; all the really fun and useful extensions in PHP are not functional the way they were in PHP 4, but they're classes.

So I'm wondering if I should rewrite <required-functions> into:
<required>
  <function>functionname</function>
  <class>classname</class>
  <method class="classname">method</method>
</required>

You can, sort of, do some of the method calls through the existing interface - provided that you want a static method, if you don't, you're screwed.

I'm not worried about details of implementation (I'm thinking, ultimately, it would be done through instancing the Reflection API) at this stage, merely curious to know if it would be that useful.
Title: Re: Add-on Manager: Mechanics
Post by: live627 on February 9th, 2012, 02:21 AM
Quote
! Remove file edits from the master copy of the plugin info specification. It isn't implemented currently and isn't going to be in future, so no need to document its format. (plugin-info.rng)
The OP needs updated to reflect this.
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 02:46 AM
It needs updating to reflect the correct title too. I'll deal with it when I get chance, thanks for reminding me :)
Posted: February 9th, 2012, 02:30 AM

Now updated to actually be more accurate.
Title: Re: Plugin Manager: Mechanics
Post by: Nao on February 9th, 2012, 07:22 AM
Shall we remove the package manager by now?
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 08:59 AM
Well, it still has some useful stuff like dealing with unpacking mods and so on - but the more I think about it the more I am kind of leaning towards what XenForo does - and not having any kind of unpacking on the server. It certainly solves permissions headaches but isn't so friendly.
Title: Re: Plugin Manager: Mechanics
Post by: Nao on February 9th, 2012, 09:58 AM
I thought that unpacking through PHP would on the contrary ensure that files are all writable by PHP..?
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 10:08 AM
Making it writable from PHP means exposing it to being writable by the web server. Convenient, sure, but on shared hosting, you have to make it writable to all users since on most shared hosts I've seen, the web server user is not the same as the owner of the files and very often isn't in the same group - meaning that your files have to be world writable by definition.

This is why systems get hacked: folks change the permissions and trigger the system to abuse it. Now, if you're not modifying core files, you limit the damage that can be done - but still all the plugin files are within the cross-hairs, and anything on that server that happens to get compromised, can compromise any and all plugins. Even attachments aren't really that safe - but they're safe in that they can't directly be executed by the server (or at least, on anything remotely sane, shouldn't be able to be executed), though attachments I accept have to remain world writable by definition.

On the other hand, if users have to upload plugins, they get them uploaded via FTP and invariably that just works as expected. The plugin gets uploaded, it's not writable by the web server so it's protected from being infected by any other compromised site on the server.

Yes, it's not as convenient as having something that does it all magically, but I'd argue it's a lot safer and doesn't require what amounts to a workaround.

What's more important: being secure or being convenient? You know as well as I do that users don't actually care about being secure, but they love convenience, and we have to make a call on what we think as more important, because we have to care when the user doesn't.
Title: Re: Plugin Manager: Mechanics
Post by: Nao on February 9th, 2012, 10:34 AM
Y'know, I simply really like(d) the idea of downloading gzipped plugins from within Wedge... :-/
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 10:40 AM
Yeah, it's one of those things that would be very awesome, but the security aspect makes me incredibly wary about doing it.

Certainly it isn't required for alpha and can be shunted back to beta, but given that other systems are doing this exact thing (and they're paid, no less) with little apparent ill-will as a result, it does give me confidence that it might not be so bad after all.

It would certainly encourage people to think a bit more carefully about whether they really need a plugin or not.
Title: Re: Plugin Manager: Mechanics
Post by: PantsManUK on February 9th, 2012, 11:03 AM
I'll take security over convenience every time (in the past our site has been compromised many times, and cleaning up just takes too much time), but I understand the difference; there will be folks for whom the opposite applies. To massage an apposite saying, you can't please all the people all the time...  :whistle:
Title: Re: Plugin Manager: Mechanics
Post by: MultiformeIngegno on February 9th, 2012, 12:17 PM
I know you guys don't like Wordpress much, but it does allow (as SMF) to download and install plugins/themes/upgrades directly from the web interface. What technique did they use? Same as SMF (so files with 777 permissions)?
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 12:18 PM
Yes, it's the same as SMF, basically, and it's a large contributing factor to why WP gets hacked quite often.
Title: Re: Plugin Manager: Mechanics
Post by: Nao on February 9th, 2012, 12:29 PM
Quote from MultiformeIngegno on February 9th, 2012, 12:17 PM
I know you guys don't like Wordpress much
I don't mind WP, personally. If anything, the very fact that it's widespread means that it got something right... Maybe not in the code, but at least in its community handling.
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 12:41 PM
Bear in mind it's about 9 years old now, it's pretty much the oldest PHP blogging platform, and it's still going, and its versatility with themes doesn't hurt it either.

(How many other blogging platforms running on PHP can you name?)
Title: Re: Plugin Manager: Mechanics
Post by: MultiformeIngegno on February 9th, 2012, 01:44 PM
IMHO one of its driving forces is that it's extremely flexible. It's not just a blogging platform, it's a CMS (okay, not a forum software, but a good CMS). Check out the custom template feature (90% like SSI.php), you can use WP to replace Joomla, it can be used in websites where you need to handle a lot of pages and people contribution..

I shouldn't care about security, I'd just add a 128px text (okay, maybe a little smaller :P) saying that if they're under a shared hosting and they use the web interface to upload & install stuff, they are at risk..

Why penalize users that know what they're doing (and maybe aren't on a shared hosting..)?
Title: Re: Plugin Manager: Mechanics
Post by: Arantor on February 9th, 2012, 01:55 PM
Oh please. Calling it a CMS is like calling a go-kart a car. Both do the same job but one is a shabbily built, usable but neither efficient or elegant. Or it's like comparing a basic Nokia handset to an iPhone because they can both make calls when you want more than just calling functionality.
Quote
Check out the custom template feature (90% like SSI.php)
It's nothing like SSI.php except vaguely conceptually.
Quote
it can be used in websites where you need to handle a lot of pages and people contribution..
Not without significant work.

OK, I'm going to reiterate a point that I think has been forgotten.

I'm not bashing WP based on what I've seen. I USE WP CURRENTLY ON TWO SITES. I know exactly what it's capable of. Anything beyond a basic blog, it just can't handle. You cannot even have a page that's visible to signed in members only without *custom coding*, and not trivial custom coding, to boot. That alone rules it out of being a good CMS.
Quote
I shouldn't care about security
So when users get hacked, I can send them your way, can I? Because you know if a Wedge install gets hacked, they feel it's our fault first and foremost and never theirs.
Quote
saying that if they're under a shared hosting and they use the web interface to upload & install stuff, they are at risk..
Most users do not understand what shared hosting means. And they won't listen to that warning, they'll upload and install stuff regardless - but it'll be our fault when the shit hits the fan because "[Wedge] should have prevented there being a problem" and anything else is making excuses.

There's no best answer, only a selection of varyingly-bad answers, and right now I'm just sensing that not having an upload feature (like, I'll note, XenForo and quite probably vBulletin, though I haven't used it) is actually the lesser evil.
Title: Re: Documentation for plugin authors
Post by: Sabre™ on July 23rd, 2015, 05:38 AM
The first few pages of this thread, although quite aged, were Very informative. :+1:
Thank You for this info, I'm trying to find more of what I don't yet know I need :blink:
Title: Re: Documentation for plugin authors
Post by: CerealGuy on July 24th, 2015, 12:18 PM
Quote from Sabre™ on July 23rd, 2015, 05:38 AM
The first few pages of this thread, although quite aged, were Very informative. :+1:
Thank You for this info, I'm trying to find more of what I don't yet know I need :blink:
I'm not 100% sure, but I think what is missing here is the mods.xml part. There's not much you need to know about, if I remember correctly it's working the same like in SMF (i never did a real plugin/mod for smf so i don't know).
If you need some help, just ask, I'm using wedge's plugin system since months and it's awesome ;).
Title: Re: Documentation for plugin authors
Post by: Sabre™ on July 24th, 2015, 01:10 PM
Thank you buddy :cool:
I've noticed there is not alot of Q&A threads, posts and documentation around, which I believe is a benefit to any software, for those that are thinking of using it will see similar queries to their own, and feel it's less daunting.
Unless we are only aiming toward the technically adept, and not 'the average Joe Blow', thus limiting the audience and the wide spread of the software.
I haven't looked into the code in depth as of yet, just combing the site for any knowledge on structure, hooks etc etc...
Information on these are Very limited, but what is around is Very resourceful.
I've read the majority of your posts in regards to Q&A's, and others posts aswell, they've been Very helpful.
I'm intending on being active here, as well as SMF and others until life's realities draw me away.
So if it's permitted, I will be flooding the fora with plenty of questions! :lol: ;) All to simplify the directness of the program of course :)
Ok, I won't draw too much time in this section, as it isn't relevant to the thread.

Thanks again mate :+1: