FWIW, just to be clear:
I do agree that most mods should be made with hooks. I just don't think it's possible for every single mod, and I think it's limiting. If the average user has more than one mod that does direct edits (excluding updates), that may be a bad thing.
And again, some of the issue is "contention" for certain areas of SMF that are common to modify. For this reason, conflicts and problems and order-of-installation are much larger problems. Every area of contention definitely should be a hook, solving that issue.
With that said...
I want mods that do not have to rely on version checking. I want to avoid all the explicit checks that SMF has to make.
Hmm. I agree that SMF is probably overly cautious with version number requirements. I based the model on Firefox, which I think has laxed up their requirements a bit. I'm going to save this for later. If it had enough users, something like the Add-on Compatibility Reporter could help the situation.
It also doesn't discourage abandonment - mod authors, generally, want to do as little as possible to maintain their mods.
I suggest this is a separate issue. I see no reason why hooks don't discourage abandonment (work or no work.) I think you mean to assert that abandonment itself is less of an issue with a hook-only system.
But ultimately, I think abandonment affects everything - offhand, it doesn't seem like things that are abandoned are popular in any context, for a number of good (logistical) reasons. They may work, but all software has bugs, and so do modifications, so abandonment is always an issue.
I don't know what this is but it sounds like a bad fix for a good problem.
(And if I can encourage them to do it a better way by making it less work for them, awesome.)
I think a sane approach to attempt would be to change the format from what SMF uses drastically. Have an optional or required minimum version number, and have an optional (and never used in the examples) maximum version. You'll probably disagree with me, but I do think there is a place for a maximum version.
I don't think a lot of mods used it, but the intention was for one mod to be installable in more versions of SMF. But because SMF went 4 years without a major update, this didn't end up being useful. Maybe it's not actually that useful, but there is Windows software and various other things that have BC code that is applied to older versions.
Consider, a mod that comes with a bugfix (or maybe a dependency to a common bugfix mod) for an older version, such as SMF 1.1. This bugfix is one that, for stability reasons, was not backported to 1.1.x proper, but exists as a checkin. This is a common thing that happens in software all the time. The package manager was designed (but never documented) to handle this case, so that it could optionally install some backported bugfix that affected the mod, but did not really affect SMF itself much.
Template edits in SMF are a huge problem, and a very separate one in my mind. The issue there is the propagation and full-file defaulting, and also the lack of any sane way to modify small pieces. TOX-G was designed to solve each of these problems in the simplest way possible.
What tends to happen is that mods which hit up the same place tend to collide, for the obvious reason
Right. This means that they are contending for that area of code, which means a hook should exist to handle what they are trying to do. Definitely.
but any mod that then tweaks the layout (or has a different layout entirely) is going to conflict.
But you note that it won't go away, I'm just giving a real-world use case.
1. It makes people have higher permissions than they actually need.
2. It is a major cause of people hitting mod_security errors where they throw 777 permissions at everything.
3. It is very likely also a contributor towards installations being compromised and hacked.
I'm confident that the problems are solvable. I made changes to the webinstall.php file, and couldn't find anyone that had problems using it even on servers with mod_security, suhosin, CGI suexec, or etc.
The intention was for the package manager to automatically run the change file perms routine after installing, or else set files back, but I was concerned about some of the details (such as a mod adding files that did or did not need to be writable.)
The package manager was one of the first things I changed, and the main thing I did was move it from boardmod to a format that provided more options and detail, because of some of the annoyances I had had with boardmod. File permissions at that point were kept the same, and quite frankly, I didn't even think about it until people started mentioning it.
Also, it was popular then (as it still is now) to run PHP in an insecure setup where PHP can always change its own files. This was considered by many to be the most secure setup, and at first I believed them. I understood the drawbacks of the other popular method (PHP as a single user on shared servers) and basically figured there was no way around that risk because of the db password and attachments directory.
I would probably take it more seriously now, and also more aggressively and clearly indicate the correct and secure setup for file permissions and PHP to run on any server.
I don't know the specifics, but it looks like it was a file-uploaded-but-shouldn't-have-been-allowed type of issue, or maybe a CSRF. Assuming this is true, and attachments or cache are writable, you're probably screwed no matter what.
I know of another hack that people blame on file permissions, which infects files with <script> and such, generally with names like index.php, index.html, default.html, etc. This is a client side hack (which might be delivered by a compromised HTML, jar, etc. file) that uses your FileZilla/SmartFTP/Dreamweaver/etc. saved FTP settings.
The only true way to fix this issue is to encourage the separate partitioning of uploads. For this reason, I suggest S3 or making it the default option to enter an FTP account setup to store uploads in. This encourages better practices.
While it can be 'made more reliable', I have lost count of the number of support issues that emerge every single day with SMF because of file permissions.
Incidentally, the webinstaller can do a better job at installing with the correct file permissions (and not mangling file names) than the average user would using an FTP client and the Wikipedia entry for CHMOD. I wouldn't expect these questions and problems to go away even without the package manager.
I think the problem is that no one took the initiative to test and track down the issues. They just figured that some workarounds were this and that and SMF could never really do it. Add 4 years. Kinda silly in my mind.
I could go on, I can find you more and more reasons to discourage file edits, and I hope this is enough to discourage mod authors where possible.
I don't think file edits are "easier." You have to edit, and then pull out your changes. In any significantly different codebase, this won't be preferrable to "edit the php file that hooks, and see your changes immediately, and then zip that exact file without additional testing needed." As long as people have good examples to go from, I don't think they will lean on file edits unless necessary.
we make the user have to manually enable it first, to understand all the consequences. Plus, we can manage it through the repository to advise the user that manual edits will be required.
I still think you want updates applied in an automated fashion. I find asking a user to fire up an FTP client (and what that is is not common knowledge, at least in my experience) to install updates more or less draconian.
I walked a client through installing a software that's installed in the typical draconian fashion. She did end up figuring out FTP, but wasn't really sure on the chmod parts and I ended up having to help her. She took notes, and learned from the experience, but ultimately I can be fairly confident she will never upgrade that software, because installing it was enough of a hassle.
That's why I see going back to the "upload the contents of this zip file" model kinda like trading in a shiny new Prius for some car from the early 1990's that gets terrible mileage and barely passes emission standards. It's hard for me to even understand the motivation, even if you think the electric engine in the Prius is a bit complicated or causes problems sometimes.
But I think any more than that will be a case of putting out the tools and hoping developers make use of them
This is always the most important thing.
I'd like to reiterate my original idea.
Disable file edits at least until Wedge is out of beta.
If people rely on file edits when they can't find a hook, we'll never get people to get us to add hooks.
That doesn't seem crazy. That said, if you add a hook for every possible file edit, I promise
you that you'll regret it later. More on that later.
A number of reasons (although if there's enough semantic hook detection, this can be mitigated.)
A well developed mod will be shipped as one version, even if there are multiple popular current branches of Wedge currently being used. In some fashion, different hooks need to be installed potentially for different versions - consider that Wedge 2.0 adds a new hook that this mod wants
to use, but in Wedge 1.x, it needs to use other hooks (or file edit the hook in, for BC.)
This can be mitigated by some sort of hook detection, but this has its own set of problems. One option is to run plugin init code on every page view, but this has performance problems with many mods. Another is to re-run hook init logic for every different version number internally and/or whenever plugins are installed/removed, so the logic can be updated and cached. This has its own complications, mostly that it isn't simple to write.
I imagine you'll want a minimum version. I can't think offhand of any system that doesn't allow for a minimum version requirement.
And the stuff I wrote 10 years ago... with one exception, it all still works on Windows 7. That particular exception is because of an API that was deprecated back in 2000, and naturally doesn't exist now. If you can do this on an operating system, why can't we apply the same basic techniques to programming in a higher level environment?
This is a fairly easy argument to refute. I'm going to keep it brief because it's getting late, and I need to get some sleep, but:
1. Apps are not mods. If your intent is to only support apps in Wedge, like Facebook apps and iPhone apps, then file edits are pointless. That seems a silly place to aim for, IMHO.
2. Drives have had major version requirements in Windows. Obviously you never wrote a driver.
3. Have you ever tried to change the task bar, Alt-Tab menu, or install tabs in Windows Explorer? I have done all of these things. They are highly Windows version dependent.
4. Microsoft has but a lot of effort and man hours into maintaining compatibility heuristics, rules, and settings in Windows. Maybe you've never noticed them, which is a testament to them doing something right. It's quite possible they are responsible for making your apps work, in some cases (especially where UAC might be concerned.) Are you intending to put in that level of effort?
5. Zip tools often had major problems with Windows updates when they came out with XP, which had its own zip tool that stomped over people's existing integration methods. ConTEXT also had some issues, although they were worked out. I know tons of software has had to be updated for UAC as well.
6. Windows' API is full of compatibility functions. Did you realize, every function in the API has two versions? An "A" version and a "W" version. The W versions were added later, and support Unicode. The A versions support multi-byte encodings where the codepage is set properly. They maintain both versions and it's a huge overhead.
7. There are bugs in many of Windows' APIs, some of which are documented, and must stay that way for compatibility.
8. Were you aware that Windows programs define the version they intend to compile against (although usually this is done for you automatically)? Windows then may enable certain bugs in API calls or etc. so that it will work more like that particular version.
Are you intending to do all of these things and deal with all of these problems?
I want to be able to port my mods to Wedge. I want to be able to write some of the other mods I've thought about over the last 2 years, except the frustration involved in writing most of them in a way that won't cause me to do support is why I didn't.
If it's a hassle, it needs to be fixed, agreed. There are compromises to do this without veering into an unknown space that may have its own new set of challenges not yet imagined.
Sounds ambitious, and in a way i see what you mean, I like to write themes or mods too, that will always work, or don't rely on anything changes etc. But i think thats an utopia, as long as the host script changes, you need to change with it. And the host script need to be possible to change, your knowledge will change, technologies arrive that require it etc. I bet your old programs, while they still run, could use updating still. Being content with it "just working" doesn't bode well for innovatism IMHO...
But I've love to see me proven wrong of course lol. :)
With themes, I think it's mostly impossible. But I'm hoping that if they are organized right, the correct parts can be frozen, and most themes can be written to just modify all the parts that are common that don't need changes, and the main code can use overlays to apply to older themes to add new features, such as new fields.
Which brings me to "freezing." With any API, including a hook API, freezing is important. This is something SMF never did and it paid for it. Facebook also (purposefully, I'm fairly sure) does not freeze their APIs, whereas for example Google does freeze their APIs.
A frozen API means that it doesn't change. Once you sent the parameter the wrong way, you're living with it. You're passing an id and you want to pass an object now? Too bad. It's frozen. You have to create a new hook.
Some people think freezing is a bad thing, and leads to stagnation. Making it "wrong" to improve the code means that the code will get messy, and have more weird problems in it. This is why some developers intentionally do not freeze their APIs.
I personally prefer it, though, because it means that compatibility can be achieved in some sane way. When I trust an API is frozen, I'm much more interested in writing for it, unlike an API that I expect could easily change within a couple months invalidating my work. It sounds like you agree, and also want this.
Just want to mention some of its detriments so they are not ignored:
1. When freezing APIs, most people take the course of making as few as necessary. This is because, the more you have, the more locked doors you're putting in the way of your own development.
2. These APIs also typically either have feature branches (which can be detected for) or versions, and phase out older versions or feature branches over time. This means additional management and documentation overhead, but it's not too bad.
3. You should think hard and probably have a review process on hooks before they are finalized, so that the pains of maintaining "I wish I'd never made that hook" problems go away.
4. You need to consider more carefully what happens when a feature is heavily overhauled. For example, suppose hooks were used in the personal message system, prior to it being changed in 1.1 iirc to have labels and such. You have to make a decision to design the new features around the hooks, or to throw the hooks out, or do a shade of grey in between.
5. Also, with hooks, you have to carefully consider chaining, which is a more important issue than it may seem at first. When using frozen APIs, this gets more complicated: suppose you implement an "extended" version of the hook in a new version, whilst retaining the old. Now a user has two plugins, each using the hook, but different versions. You have to handle the chaining.
Oh, of course they could be updated, because I've learned a great deal in the last decade. But here's the thing: won't someone PLEASE think of the users? :P
I don't think users like dealing with order-of-installation issues, or contention issues, or any of those things. I also don't think they like dealing with manual edits or uploads, incompatibilities, or etc. either. Users also don't want indirect problems like core development slowdown or not being able to upgrade because a new version has different hooks.
Trying to think only of the user often ends up ignoring a significant part of their concern. Best to think of everyone and only prioritize the users where possible, IMHO.
That said, I know that you care about the users already from past conversations, and that's a very good thing. I'm not knocking that at all.
Now, if "you" can manage to keep mods working across version updates, I'll love you forever; having to (re)install mods every time the host app changes isn't fun (can anyone say "SMP minecraft server"?)
This was just one of the historic "bad decisions" we made. This came up several times while we were working on the feature. I noticed it and considered it, we talked in the dev team about it, and the team discussed it. But usually briefly, and the answer was always "ah, it's fine to reinstall every time."
I agree that was a bad decision, but the reality is we just didn't think it was a huge problem, and at the time, there weren't a whole lot of mods to reinstall anyway. After that, it sorta got baked into our heads a bit, and so we stopped considering the idea of trying to retain installed mods on upgrade.
At one point, someone had proposed a more robust system, that tracked exactly what to do, installation order if necessary, and did a dry run unapplying, updating, and reapplying, and told you which mods weren't compatible that way. I don't remember how it totally went, but we didn't do it because we figured the effort wasn't worth it.
In part, we really weren't considering the case of many mods being installed. This was in part an over-reaction to "Supermod", which for YaBB SE was this ugly, Frankenstein thing
that was a scary mess that had all sorts of fun bugs and made mod authors mass delete their mods. Because of the issues there, I think none of us were really thinking of lots of mods as a reasonable use case at all.
Still, I'm always surprised that things like that didn't change in the last 5 years.
Anyway, to simplify my point, one site I administer is using SMF 2 RC5 will plenty of mods...now with SMF 2 Final, I am reluctant to upgrade since it will involved the same amount of work during the initial installation; that is checking if the mod is compatible with the version and re-install...
it would be better to simply upgrade the forum software only without all the hassle of checking the compatibility of the mods since to me it come from the same branch
Presumably, not a ton of code has changed between SMF 2.0 RC5 and SMF 2.0, right? So in theory, this sounds right. I'm not sure what it means to check the compatibility; from context, I assume this is some manual process?
Let me ask you this. The mods you have installed now, did you have to make any manual edits to install them all successfully? I don't know if you have a bunch or if you use themes, but if the answer to either of those is yes, I presume you had to do manual edits, and you're wanting to avoid making those again, right?
I assume there is an update package for SMF 2.0 RC5. I'm guessing it does not apply cleanly on top of your existing mods, right?
Separately, I think that Chrome handles these very well, although as mentioned, there are buggy extensions that don't work in the current version.
If you've ever built one, you'll know that it automatically packages things easily for you, and also that it uses permissions:http://code.google.com/chrome/extensions/permission_warnings.html
These are a common concept in integrations now, and something I strongly recommend for Wedge. I would imagine that file edits might be a permission.