So yeah, one can spew all over bad decisions, poor coding, not enough visions
FWIW, I don't care about who did it, or even why. It's simply not relevant any more. What I'm trying to get at is what happened, and what resulted because of it in terms of how it affects what we do going forward.
The point being made, which was clearly missed, is that the full upgrade cycle being pushed every 6 months, give or take, combined with mods having to be reinstalled (assuming mod authors bothered to update) was a major PITA for most admins. It has led to a great number of admins just not updating their installations, even leaving them rather dangerously insecure.
That, combined with the fact that mods could not be reliably guaranteed to work between even RCs (I remember having to give SimpleDesk specific code changes between each and every single RC, and a further change because of a regression bug in 2.0 final, though in each case other than the last, I'm not disputing that the result was for the better) and the fact that modders were sometimes lynched for not getting updates out within *hours* of a new release, it put a lot of them off updating until final. This is a cycle I do not want us to get locked into, and why for the most part I'm pushing people away from manual edits.
It was mostly a statement in response to Unknown's question about upgrading from 2.0 RC5 to final being a full update (which it is, unless you're insane enough to go through and make the changes to each file by hand, as I did in Wedge's case)
Anyway, now to answer the meat of stuff.
I do agree that most mods should be made with hooks. I just don't think it's possible for every single mod, and I think it's limiting. If the average user has more than one mod that does direct edits (excluding updates), that may be a bad thing.
Ah, that means I've been reading between the wrong lines. The apparent level of enthusiasm for raw edits seemed to suggest otherwise
;) It also means we're far closer in thought than I'd actually envisaged. Though, honestly, I was prepared to let mod authors who need manual edits deal with it themselves (much as you'll find in some of the other communities)
And again, some of the issue is "contention" for certain areas of SMF that are common to modify. For this reason, conflicts and problems and order-of-installation are much larger problems. Every area of contention definitely should be a hook, solving that issue.
Yes, that's the plan. I look to SimpleDesk on this one: there are some places I do actually do template edits in SD 2.0, and even though I hate the general concept of having to do that, I'm not entirely uncomfortable about it because of the specificity of the templates and of the changes themselves. It is unlikely that many mods would ever update the track-IP screen or the manage-attachments screen unless they had very, very good reason (and SD does, in both cases) - and since they are uncommon templates, I'd never had a conflict.
The main areas for conflict in SMF are the admin panel, the main menu, the permissions code and the actions list, all of which got hooks in RC4/5. One for the credits page would have been good, as would for the 'illegal permissions' setup. But there are plenty of places that hooks would be useful and aren't - such as in the post view to add more things to work with in terms of styling (like the suggestion I made about putting the topic author in to the post as a CSS class, for example). Not all things are expressly hook points as we normally think of them.
But you note that it won't go away, I'm just giving a real-world use case.
Oh, I have plenty of real-world cases to draw from, and have had to deal with. The reality in the end is that you can't fix everything, but that you can design other things to minimise the problem. SimpleDesk does some seriously hokey things in the board index, for example, and it has caused problems with custom themes and some mods - but because I can see the cause and effect (i.e. that SD does some hokey stuff, and the other mods and themes rely on assuming that the board index is untampered-with), I can mitigate it in SD itself by catering to those assumptions.
My point here, I guess, is that mods can be written with other mods in mind, if the author chooses to be careful about it.
That doesn't seem crazy. That said, if you add a hook for every possible file edit, I promise you that you'll regret it later. More on that later.
I never intended to write hooks for every single possible point. The reality is that you actually don't need to be that thorough. Since there's a separation, by design, between content processing and display, and everything is routed to $context, there's no overarching reason why you couldn't just throw a hook at the end of every action and make sure $context is available, plus anything that might be useful in the context of the action itself, and that would solve a great number of problems before you start. (It would also be quite hacky, I think, but it would certainly solve some of the issues.)
This is a fairly easy argument to refute. I'm going to keep it brief because it's getting late, and I need to get some sleep, but:
Not going to go into detail, because I knew it was a fairly weak argument to begin with, but the question does throw up some interesting permutations, as you go on to expand with.
I never wrote a driver or made some of the changes you did, so yes, I hit the apps side rather than the mods side. Yes, I also knew about the A-vs-W stuff as I started out writing on WinNT 4 and made them work on Win98 and WinXP as well. Really, the point being made was that if we can do some of this stuff in other ecosystems, it should be possible to do some of it in the Wedge ecosystem, and it was more a comment to encourage people to stop and think for a moment.
Mods in the current ecosystem very broadly fall into two categories: mods that add new behaviour (which are ultimately more like apps) and mods that modify existing behaviour, and I've written my share of both for SMF.
A lot of the very popular mods do fall into the first category, and ideally I want it to be as painless as possible for them to be developed, because if I can get that to be the case (like supporting something like SimpleDesk), I can then focus on how to improve the end to end experience for the modifying-existing-behaviour case too. It's all parts of the same problem, and there isn't a one size fits all solution for it. I had actually debated leaving just hooks in the core and making file editing be managed through an official add-on, that could essentially be a dependency for mods that actually needed it. (This sort of behaviour is not unheard of even in the forum ecosystem. Whether it's desirable is another matter entirely.)
A number of reasons (although if there's enough semantic hook detection, this can be mitigated.)
A well developed mod will be shipped as one version, even if there are multiple popular current branches of Wedge currently being used. In some fashion, different hooks need to be installed potentially for different versions - consider that Wedge 2.0 adds a new hook that this mod wants to use, but in Wedge 1.x, it needs to use other hooks (or file edit the hook in, for BC.)
You can certainly do that. I've done it myself, writing mods for both the 1.x and 2.0 branches of SMF and bundling them in the same package. But it's hard work because of the nature of changes in SMF between the two branches and in all honesty, I'd personally rather have not bothered with the dual packaging. (In fact, I shied away from making mods for both branches at once, simply because invariably I had to write it twice, because everything was different, most importantly $smcFunc vs db_query)
Users, generally, don't seem to have a problem with mods being in separate packages for different versions provided it's made clear to them what package is what, and the concept behind my hook detection methodology would mean that Wedge's main core would know what hooks it had available to use, and thus what hooks a given mod could use, while the main site would know all the hooks in every version.
As I mentioned somewhere, not sure if it was public, there was very much an awareness in the design about freezing the API, so that if a hook changed its structure, it would no longer exist in the previous form. That's no different to saying "versions 1, 2, 3 support this, version 4 does not", just you're not relying on a human doing the legwork in stating it in the package.
It was suggested before that SMF's mod site could attempt to perform installations on different versions and see if the file edits were successful, in a similar manner, but that it would ultimately cause more problems than it solved if other behaviour were broken - the plan, then, would be to make it so hooks generally wouldn't change between versions unless absolutely necessary - then break it in a non BC manner.
I would point out that it is our intention not just to throw out a release and then declare it feature frozen for the life of that branch. We always planned to handle it iteratively, in a kaizen-style of development whereby each version adds something as well as just bug fixes, some method of improving how we do what we do - instead of just giving users a 'this version just fixes some bugs' reason to update (which is a push factor), we want to give them some pull factors too.
This can be mitigated by some sort of hook detection, but this has its own set of problems. One option is to run plugin init code on every page view, but this has performance problems with many mods. Another is to re-run hook init logic for every different version number internally and/or whenever plugins are installed/removed, so the logic can be updated and cached. This has its own complications, mostly that it isn't simple to write.
I'd already written a prototype of it, run on install/remove, and a much more naive, limited version already exists in SimpleDesk's plugin manager.
I don't know the specifics, but it looks like it was a file-uploaded-but-shouldn't-have-been-allowed type of issue, or maybe a CSRF. Assuming this is true, and attachments or cache are writable, you're probably screwed no matter what.
It was a file being uploaded that was able to be executed. Which then tries to infect any PHP file it can modify.
I don't think users like dealing with order-of-installation issues, or contention issues, or any of those things. I also don't think they like dealing with manual edits or uploads, incompatibilities, or etc. either. Users also don't want indirect problems like core development slowdown or not being able to upgrade because a new version has different hooks.
Trying to think only of the user often ends up ignoring a significant part of their concern. Best to think of everyone and only prioritize the users where possible, IMHO.
That said, I know that you care about the users already from past conversations, and that's a very good thing. I'm not knocking that at all.
Order of installation is an issue that, really, only comes about because of multiple mods doing file edits in contentious areas - and specifically because one mod makes a broader selection than it needs to. The contentious areas can be removed with hooks, meaning that with very few exceptions, order of installation is a non issue.
(There is one exception I'm aware of, where there's a mod that is explicitly hook only, even in SMF, and it checks periodically to make sure that for one particular hook, it is always called last. But such things do not strike me as common with properly written mods using hooks.)
From a plugin perspective, there are two groups of user I have to consider, and only two. Yes, other users need to be considered for other things, but purely for plugins and their related infrastructure, I need to consider mod developers and mod consumers, aka forum admins.
The main concerns from mod consumers are that mods work with given versions of the software, that if they pull down an update, that their mods will continue to work (or at the very least, safely stop working, pending an update from their authors), and that mods work with as little fuss as possible, which generally means few file edits and working on as many themes as possible without any hassle.
Most mod consumers seem to understand that some mods will require manual work if they are particularly complex, but do not relish the frequency of manual installation that mods seem to need, especially if there are file permissions issues, or custom themes. There is almost a sub-industry lurking in SMF of people who will perform mod installs for people for money, usually only a small amount, but the fact that people are prepared to pay others to install mods does bother me slightly.
On the other hand, mod developers do seem to like the accessibility into SMF's internals, but most of them do not like the level of support requirement, namely having to update the listing for given versions (because even if a mod package explicitly lists 1.1.* as a working version number, the listing on the mod site won't, and users will *definitely* ask if a given mod for 1.1.x will work on 1.1.y)
The biggest problem for a long time was the lack of tools given to mod authors for SMF - there are now tools available for doing mods that work without much/any hassle, and minimising the amount of hassle there is on other mods. SimpleDesk is no different, and about half of the edits it used to make (that were low-maintenance) now become no-maintenance because they're no longer edits.
There are naturally downsides to this: the implementation method of SMF's hooks is a little bit narrow-minded, and there aren't enough hooks in some of the areas that would be useful to have hooks. Almost none of the admin panel has any hooks, only the menu really does, and the 'Modification Settings' entry in the menu, which is fine if you're doing huge mods that add new menu items or new menus entirely, or one-shot settings, but anything in the middle isn't really catered for unless you abuse the tools given to you.
And while I'm bound to draw criticism for the above comment, the simple fact is that: the idea of expanding the hooks is good, I welcomed the change because compared to what there was, it was a massive improvement. But it did lack one really big change that Nao added almost straightaway to Wedge: the ability for a hook to load a file when necessary. You want to use a hook in SMF... fine, just use one of the few points you actually get to load a file, which means you invariably load files on init rather than where it would be useful to load them.
That, and the fact most mod authors simply do not understand how to make the hooks work properly, because the only documentation on the hooks was either written by Orstio way back or by me for 2.0 RC3 - which doesn't include the multi-calling ability, or indeed the method by which you now add callees to hooks.
This was just one of the historic "bad decisions" we made. This came up several times while we were working on the feature. I noticed it and considered it, we talked in the dev team about it, and the team discussed it. But usually briefly, and the answer was always "ah, it's fine to reinstall every time."
It's not really great for users. Most of them will understand, to a point, once you've beaten them over the head, but half the time they won't understand off the bat, especially if their mods won't reinstall properly (or worse, as happened during more than one SMF RC upgrade, the mods continued to be listed as installed when they weren't).
There is, buried in the depths of the upgrader code, some facilities for this but I don't think they're enabled and even if they are, I don't think they work properly.
At one point, someone had proposed a more robust system, that tracked exactly what to do, installation order if necessary, and did a dry run unapplying, updating, and reapplying, and told you which mods weren't compatible that way. I don't remember how it totally went, but we didn't do it because we figured the effort wasn't worth it.
That would be a major step up from where SMF is now, but honestly that's not a road I want to go down if I can help it. I want to make it as painless for admins as possible, even to the point where if something breaks, it can always be unbroken again with as little effort for them as possible: most admins are not that technically inclined (much as I find the idea weird: why run something without learning a bit more about it?), and most will simply ask for help rather than anything else.
Here's where it gets weird, though: it never seems to be a mod author's fault that mods have the hurdles they do. For the very most part, problems with mod installation always seems to be SMF's fault in the user's mind, which leads to SMF getting unfair comments that it doesn't deserve.
Presumably, not a ton of code has changed between SMF 2.0 RC5 and SMF 2.0, right? So in theory, this sounds right. I'm not sure what it means to check the compatibility; from context, I assume this is some manual process?
Users should check that mods have been updated for 2.0 by looking at the mod's page. Especially since each RC has brought some interesting changes with it (as it happens, there are behavioural changes between RC5 and final that some mods will have problems with)
Oh, and before I forget, as it's relevant. You asked what version-emulate was. In 1.1.x, version_emulate was a parameter to add to the URL while in the package manager to override the internal SMF version, thus you could install a package that only listed 1.1.13 in it, while you were running 1.1.14, for example. SMF 2.0 took it further and put a UI in the package manager itself that let you type in this version you wanted to use. (As it happens, I wrote a UI for the 1.1.x version as a drop-down to ease the process).
You can, then, attempt to install any mod on 2.0 without much fuss.
These are a common concept in integrations now, and something I strongly recommend for Wedge. I would imagine that file edits might be a permission.
Incidentally that's actually where I was going with my own package stuff. The manifest declares what hooks it uses, plus optionally things like obscure PHP functions from extensions, PHP version, MySQL version, that sort of thing.
I hadn't really thought of making file edits a permission in that respect, because I don't like the idea - I am only too aware how users are becoming 'trained' to accept stuff. That said, I guess it could be done, especially if it makes mod authors less inclined to do so where possible. It would probably be better than my app-wide solution at least.
I think that administrators can be made happy without having a rigid product. Ultimately I see both extremes as "unhappy administrators" just for different reasons.
I'd rather not have unhappy users if possible. I don't really see us as being that rigid by doing this, either, but I know full well that I'm biased towards this approach, but I'm also aware that I'm going to end up writing a lot of mods for functionality not in core that other systems have.
Although I value a lot SMF's flexibility of being able to change almost every imaginable thing in the system I just love mods like SimpleDesk (which is a very nice piece of software, congrats Arantor!) or AEVA, the way they install, uninstall and upgrade without issues with the core and then the amazing gui and amount of freedom and features they provide inside the mod itself (which to me cover 99% of the aspects I would like to be able to customize in such a mod).
Interesting you should mention both of those. In both cases they qualify as 'big mods', but they both have to deal with certain things: most notably, they both find SMF's permissions system framework inadequate and quickly end up replacing it or augmenting it (SD uses absolutely none of SMF's permissions, Aeva only uses a few for very granular control), and they both make fairly minimal (but surgically precise) changes to SMF's own core and move everything into their own files and user areas, and as such are reasonably free of the constraints attached to smaller mods that embed themselves into SMF deeply.
That said, there are some integration-y bits in SD that bend the rules, and expectations.
---I think I got everything.---
