This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
5581
Plugins / Re: Plugin manager, phoning home and privacy
« on November 3rd, 2011, 04:31 PM »
OK, so I've spent most of my day thus far understanding WTF WordPress does. If I didn't already hate WordPress, I really hate it now.
What the fuck is the point of a publicly accessible API, that's in use every day, that doesn't have documentation?[1] That assumes, of course, I already had that information to hand, but no, a trawl through the source and eventually just doing a search on wordpress.org told me what I needed to start off with, the URL of the repository. Lots of Google searches and code searches ensued.
Of relevance, is http://markjaquith.wordpress.com/2009/12/14/excluding-your-plugin-or-theme-from-update-checks/ and as a bit more background, http://w-shadow.com/blog/2010/09/02/automatic-updates-for-any-plugin/ (I don't expect anyone to read these, they're pretty WP centric, but the point should be made that underscores the very concerns I have had here, that private plugin data could be submitted to WP's master servers)
Interestingly the performance aspect of dealing with all the requests[2] is ignored.
Anyway to recap. They consider it a problem that plugins could call home and ask for updates, and want an opt-out, whether that's for privacy or practicality,[3] and it would be nice to do it in a better form than having to write a several-line function to be called from a hook before doing an update.[4]
Now, several suggestions have been put forward in how this might be dealt with, including plugins specifying a 'home' repository but I think we have to do more than that.
The typical case is a plugin that will reside on Wedge.org itself in some fashion, and nowhere else, no mirrors, no private repositories etc. Since that's the default I see no reason to actually alter it, so plugins that are just bog standard off-the-main-site ones need do nothing. And come the daily check for updates, those should be requested.
The next case is a plugin that maintains a third party repository but is also distributed on the main site. In SMF's case, this is SimpleDesk, SimplePortal and others, projects that feel they are too big in size or scope to remain totally on the home site and form their own.
Now, in those cases, it's likely that they'll have updates ahead of the main site so the plugin can specify its own repository, but that it should not automatically use that; I think it should be opt-in to use third party repositories (unless you add them manually) and there are people who won't use plugins that aren't on the official site anyway, but if they opt in, it should use the third party repository in priority to the official site.
Then you have the case of plugins that might be available on third party sites, but that won't be on the main site for whatever reason. Again, I think the adding of details should be opt-in but the plugin itself needs to indicate an opt-out for the main site.
For example, there's no need for WedgeDesk to phone wedge.org for updates because it won't get any.
The last thing to note: if a plugin contacts a third party and a response is given for that plugin, it shouldn't phone wedge.org even if the opt-out isn't used. So if a plugin originally contacts a third party but the third party is defunct, it should query wedge.org for that plugin. If the opt-out of using wedge.org is specified, it should be made clear to the user that either they use the third party server or check manually.
It's tricky!
Heh just after I hit the post button, a separate thing occurred to me.
Instead of automating look-ups, we can indicate sources for plugins instead, and have an opt-out of updates for that plugin, and when opting-in or -out for updates for a plugin, it can indicate and sort out where the plugin should be queried for. That seems more practical to me, actually, because it gives users the control they want and doesn't seem to me to have any other problems.
What the fuck is the point of a publicly accessible API, that's in use every day, that doesn't have documentation?[1] That assumes, of course, I already had that information to hand, but no, a trawl through the source and eventually just doing a search on wordpress.org told me what I needed to start off with, the URL of the repository. Lots of Google searches and code searches ensued.
Of relevance, is http://markjaquith.wordpress.com/2009/12/14/excluding-your-plugin-or-theme-from-update-checks/ and as a bit more background, http://w-shadow.com/blog/2010/09/02/automatic-updates-for-any-plugin/ (I don't expect anyone to read these, they're pretty WP centric, but the point should be made that underscores the very concerns I have had here, that private plugin data could be submitted to WP's master servers)
Interestingly the performance aspect of dealing with all the requests[2] is ignored.
Anyway to recap. They consider it a problem that plugins could call home and ask for updates, and want an opt-out, whether that's for privacy or practicality,[3] and it would be nice to do it in a better form than having to write a several-line function to be called from a hook before doing an update.[4]
Now, several suggestions have been put forward in how this might be dealt with, including plugins specifying a 'home' repository but I think we have to do more than that.
The typical case is a plugin that will reside on Wedge.org itself in some fashion, and nowhere else, no mirrors, no private repositories etc. Since that's the default I see no reason to actually alter it, so plugins that are just bog standard off-the-main-site ones need do nothing. And come the daily check for updates, those should be requested.
The next case is a plugin that maintains a third party repository but is also distributed on the main site. In SMF's case, this is SimpleDesk, SimplePortal and others, projects that feel they are too big in size or scope to remain totally on the home site and form their own.
Now, in those cases, it's likely that they'll have updates ahead of the main site so the plugin can specify its own repository, but that it should not automatically use that; I think it should be opt-in to use third party repositories (unless you add them manually) and there are people who won't use plugins that aren't on the official site anyway, but if they opt in, it should use the third party repository in priority to the official site.
Then you have the case of plugins that might be available on third party sites, but that won't be on the main site for whatever reason. Again, I think the adding of details should be opt-in but the plugin itself needs to indicate an opt-out for the main site.
For example, there's no need for WedgeDesk to phone wedge.org for updates because it won't get any.
The last thing to note: if a plugin contacts a third party and a response is given for that plugin, it shouldn't phone wedge.org even if the opt-out isn't used. So if a plugin originally contacts a third party but the third party is defunct, it should query wedge.org for that plugin. If the opt-out of using wedge.org is specified, it should be made clear to the user that either they use the third party server or check manually.
It's tricky!
Posted: November 3rd, 2011, 04:30 PM
Heh just after I hit the post button, a separate thing occurred to me.
Instead of automating look-ups, we can indicate sources for plugins instead, and have an opt-out of updates for that plugin, and when opting-in or -out for updates for a plugin, it can indicate and sort out where the plugin should be queried for. That seems more practical to me, actually, because it gives users the control they want and doesn't seem to me to have any other problems.
| 1. | Go to http://api.wordpress.org/plugins/info/1.0/ and click on the link for "API Docs". I did, later, find some documentation on the API, on the blog of 'DD32' but that's another story. |
| 2. | And by criminy there will be a lot of requests, as this happens every day or so from every outbound-calling WP install. |
| 3. | And I just considered the privacy angle. There are practical angles too, such as modified updates that you wouldn't necessarily want to be notified about, but if you're doing that, you're aware of the implications of such. |
| 4. | See the link to Mark Jaquith's blog, where he discusses doing that in WP. It's not pretty, and FWIW if Mark is saying it, it's probably the best there is for WP, I've encountered posts from him before. Note also that doing it from a hook implies the plugin is active, when it might not be. |
5582
Features / Re: Ideas from the oddest places
« on November 3rd, 2011, 04:21 PM »
That's the thing I'm most surprised at, actually. I took a cruise through their anti-spam plugins, and I find very few actually new ideas, many of which I'd already thought of and either implemented or rejected because they aren't viable for widespread use.
I'm also slightly surprised to note that there's only 16,000 plugins; given the relative dominance of WP over SMF, you'd think there would be more than 10x the plugins...
I'm also slightly surprised to note that there's only 16,000 plugins; given the relative dominance of WP over SMF, you'd think there would be more than 10x the plugins...
5583
Features / Re: Ideas from the oddest places
« on November 3rd, 2011, 04:08 PM »The only thing that will make a difference to me when it comes to avatars, is finally getting rid of any possibility of loading them through PHP
I just noticed it and thought I'd share. I didn't expect it to be greeted with mass enthusiasm, because it doesn't really work in our case, but I figured I'd share just in case.
5584
Features / Re: Thought system
« on November 3rd, 2011, 04:07 PM »Any issues with the field being called 'data', btw...? That's what it is called in my local install. It's not very original and can be confusing, but at least it makes for a nice $user_info['data'] array.
Well there's definitely some overhead too for the AeMe cache and it's stored in the profile table...
But in WD's case, the overhead is actually more trivial; it's not just looking up last-read-id vs last-actual-id or similar, there's a whole bunch of determination of ticket status vs author vs permissions to figure out how many tickets should be shown if the user is staff, and how many if they're a regular user. I spent far too long fighting it and it would save so much effort to just put it in the members table where it can be updated as needed rather than queried and regenerated at best every few minutes (and at worst far more often)
Horses for courses. Do what works best for what you're doing, and in my case I realised too late that I should have put it in the members table.
Yeah, that's the thing with the data field: it's only for when you need to update one member at a time... Which is why the custom function I added for it (updateMyData(), can be renamed later) doesn't take a $members param -- it uses $user_info['id'] directly, at least for now.
Do you have any stats, honestly.....?
And while there is an argument to be made about using chmod, the idea of making the Plugins folder a+rw doesn't fill me with confidence, especially because it then becomes an attack vector (especially on shared hosts) - it's not like the attachments or avatars folders, where you're expressly locking down content from being executed, the contents of Plugins and its related folders absolutely *must* remain executable. Which means elevating privileges, uploading, then dropping privileges again.
| 1. | The only winning move is not to play. |
5585
Other software / Re: So, SMF has declared the cold war...
« on November 3rd, 2011, 03:52 PM »Which is why we need to start documenting Wedge ASAP... There are so many features we added that are barely mentioned in these forums. It'd be a shame if they weren't put to use by plugins!
I think it is a great feature, which should continue to get attention and improvements in SMF. Just a couple of general-purpose routines would save mod authors from having to roll their own timers. I'll suggest it to the devs.
More hokey magic: scheduled tasks
I don't think this is something that's going to come up that often, but it's convoluted enough to do manually that I wanted to make it easier.
Creating a scheduled task in the system is as simple as adding this block:Code: [Select] <scheduledtasks>
<task runevery="1" runfreq="day" name="shd_scheduled" file="$addondir/src/WedgeDesk-Scheduled" />
</scheduledtasks>
One task per <task> block, and it should be fairly obvious that again, you're indicating how often the task should run, what function to call and a file to load that contains that function.
(NB: Right now the task won't receive a name in the admin panel properly, I haven't yet quite decided how I want to fix that, but rest assured, I'll provide examples once I figure it out.)
There is also a secondary subsystem in Wedge that is available to authors though not through this interface, whereby they can schedule future-dated one-off tasks to be carried out. Worth bearing in mind (and again, it's mentioned in the changelog, so it's not like I'm giving away any secrets)
| 1. | At the time the post was written it was still called the add-on manager. |
5586
Features / Ideas from the oddest places
« on November 3rd, 2011, 03:44 PM »
In the depths of my research into how WordPress updates (for plugins) work, I bumped into a separate site entirely.
In particular, this post caught my eye: http://w-shadow.com/blog/2011/01/31/lazy-load-avatars/
Now, it makes more difference on WordPress where the bulk of content will be without avatar, and only the comments are, but it strikes me that it could make a difference in terms of perceived load performance if applied to avatars in Wedge.
Thoughts?
In particular, this post caught my eye: http://w-shadow.com/blog/2011/01/31/lazy-load-avatars/
Now, it makes more difference on WordPress where the bulk of content will be without avatar, and only the comments are, but it strikes me that it could make a difference in terms of perceived load performance if applied to avatars in Wedge.
Thoughts?
5587
Other software / Re: So, SMF has declared the cold war...
« on November 3rd, 2011, 03:18 PM »As for the task scheduler -- just like Pete said. Being backwards compatible means making choices. (Plus, I never really got the hang of the task scheduler.)
5588
Features / Re: Thought system
« on November 3rd, 2011, 03:17 PM »Looking at some of your code in updateMemberData...
I take it you agree that a data field is a good idea then
I didn't know that instant_messages was calculated when sending a PM. I thought it was done when going to your PM area or something... So, scrapped.
unread_messages is probably in a similar situation.
Clearly, though, I'll be doing the data field because it's a good opportunity for plugins to store profile-specific data without having to add their own database fields (or custom fields).
Let me put it another way. Right now, one of the things I came to regret in WedgeDesk was the way the number of tickets is cached and not stored in the profile of the user. The overhead, despite caching, for that is surprising - and it has other consequences too that I won't get into.[1] So for the next version I'm going to be putting it into the members table.
While at first blush it would make sense to put it into the cache you're proposing or into the themes table, the best solution for me is a new column in the members table. From my perspective, the value is obtained immediately (though as I discussed elsewhere, I don't really like the fact that $user_settings is persisted and contains the *entire* row, regardless of what $user_info has), and I can update it when necessary regardless for users with a single UPDATE, which I can't do in the case of either the cache (because I then have to do one update per user) or the themes table (because I have to establish the value exists to UPDATE it or do an INSERT first)
It's useful for data that's per user but only accessed and updated by that user, or at least, very infrequently by other users.
I always wondered about that but didn't want to break it.
Ah, yes, that was it... I remember now how I was frustrated having to cancel my upcoming commit about that...
We could also simply say -- if your upload process doesn't work, use FTP and don't bother us with that! Just outputting a nice error message explaining what happened is okay to me...
| 1. | Suffice to say, SSI has fun with it, and the logic around cache purging when tickets are updated is ridiculous. |
5589
Other software / Re: So, SMF has declared the cold war...
« on November 3rd, 2011, 02:03 PM »
It also doesn't help that there's not a good, easy way to add scheduled tasks. (The only way to do it is a manual DB query during mod install, I did document it many months ago)
Still, there is a lesson to be learned and it's made me go back and think about other things too...
Still, there is a lesson to be learned and it's made me go back and think about other things too...
5590
Other software / Re: So, SMF has declared the cold war...
« on November 3rd, 2011, 01:29 PM »
Ah, a case of always working until it doesn't, and then the house of cards falls. Unfortunately been there too many times :(
The task scheduler actually isn't really an option, without creating a separate 1.1.x and 2.0 branch of Aeva, which considering its current status is somewhere between unnecessary and futile, as 1.1.x has no task scheduling facilities.
The task scheduler actually isn't really an option, without creating a separate 1.1.x and 2.0 branch of Aeva, which considering its current status is somewhere between unnecessary and futile, as 1.1.x has no task scheduling facilities.
5591
Features / Re: Thought system
« on November 3rd, 2011, 01:12 PM »id_thought and thought can be in it, because it's only going to be used in the member's sidebar.
Things like smiley_set or id_theme/skin could also be in it (although the latter would prevent easily giving stats in the Change Skin page.)
Maybe timezone and language stuff, too. (It's used in a user's profile, though. But we could have some kind of function that gets the serialized array and extracts some data from it. As long as it's not used for more than *two* users at a time (the viewer and the viewee), it's probably safe.)
message_labels maybe... But it's a text field so it could end up eating the new data field's space. We could make the data field a LONGTEXT but it could end up wasting a lot of time unserializing data that's never used (outside the PM area.)
instant_messages and unread_messages are good candidates, too.
Well... What do you think? I for one would sure enjoy seeing a stripped down member table!
And TE's importer can easily deal with serializing imported fields.
Oh, while looking at the member table, I found a couple of definitions ending with default 0 instead of default '0'... I turned them to '0' but I've always wondered the point of using a string when an integer is perfectly fine... Maybe it was for compatibility with SQLite or PGSQL or something...? Maybe we should just go ahead and use integers....
It wasn't compatibility, SQLite and PGSQL got their own install SQL scripts.
Yes we did. And like with so many conversations, I guess nobody thought of adding this to their to-do list...
I completely agree with your view on FTP, but there's really nothing we can do.
I'd tend to say, "to hell with people who use strange filesystems", but the more we say that kind of thing, the less users we'll have...
| 1. | Honestly, how many sites actually do the whole multi-smiley-set thing? Most people neither know nor care. But I don't have enough reason to remove that. |
5592
Off-topic / Re: Battlefield 3
« on November 3rd, 2011, 12:01 PM »
In other news, I've been playing Voxatron from the current http://www.humblebundle.com/ package (I bought it when it first came out, so I didn't have to do anything special to get the Binding of Isaac and Blocks That Matter, though I already had that one anyway)
5593
Features / Re: Thought system
« on November 3rd, 2011, 11:50 AM »Well, I'm thinking we may have to do a join on the display page... I don't like the idea, but... Well, I don't know. I'd like more opinions.
That plugin feature is a bitch eh?
| 1. | Seriously, while I understand the principles and so on of the way Linux/Unix file permissions are done, the whole lot of having to use FTP to elevate permissions on most systems instead of allowing the PHP process to run as the right user is just so mindnumbingly broken... |
5594
Features / Re: Thought system
« on November 3rd, 2011, 10:58 AM »
Oh, I see, you always need the latest, public or not, for any given user - even if you're going to be serving the public one inside topic views.
There's no good way around that. Either you have to store both in the members table, or you have to store them both in the main thoughts table and query for them appropriately. Or one of each.Quote I know exactly what you mean :/
As for progress, commit what you're happy with.
There's no good way around that. Either you have to store both in the members table, or you have to store them both in the main thoughts table and query for them appropriately. Or one of each.
I'm feeling like everytime I want to tackle a small task like thoughts, I end up spending ten times longer on it than I first expected.
Posted: November 3rd, 2011, 10:58 AM
As for progress, commit what you're happy with.
5595
Features / Re: Template skeleton!
« on November 3rd, 2011, 01:51 AM »
Hmm, I think I found another bug, this time in wetem::layer. Just want to confirm I'm using it correctly or not as the case may be.
This one's from WedgeDesk.
Code: [Select]
It's a long story as to why that's a replace up in the first load call, but that's what it has to be for the time being until I fix other things. This is based against r1142, but I have something to add in a moment about r1143.
Anyway, my understanding is that I can dynamically create a layer through wetem::layer and have it positioned where I want in the list, and I want to add it dynamically based on a variable that I know to exist and be set correctly. Except that although the if() is executed, wetem::layer never adds ticket_replies as a layer before ticket_pageend, and do-replies is never added to the list.
I know I can work around it by declaring the layer up front in the main load call and then dynamically removing it if the relevant variable is empty, but that seems clumsy to me.
Also, r1143 will break any instance of array_insert; all places it was called (e.g. the template skeleton) relied on it modifying $array by reference but now it's returning a value - and wetem (amongst other places) isn't picking that up. I'll fix it tomorrow if you don't get to it first.
This one's from WedgeDesk.
wetem::load(
array(
'shd_post_nojs',
'post_navigation',
'preview',
'ticket_info',
'ticket_subjectbox',
'ticket_meta',
'ticket_postbox',
'ticket_footer',
'ticket_pageend',
'ticket_proxy_js',
),
'default',
'replace'
);
if (!empty($context['ticket_form']['do_replies']))
{
wetem::layer('ticket_replies', 'ticket_pageend', 'before');
wetem::load('ticket_do_replies', 'ticket_replies', 'add');
}It's a long story as to why that's a replace up in the first load call, but that's what it has to be for the time being until I fix other things. This is based against r1142, but I have something to add in a moment about r1143.
Anyway, my understanding is that I can dynamically create a layer through wetem::layer and have it positioned where I want in the list, and I want to add it dynamically based on a variable that I know to exist and be set correctly. Except that although the if() is executed, wetem::layer never adds ticket_replies as a layer before ticket_pageend, and do-replies is never added to the list.
I know I can work around it by declaring the layer up front in the main load call and then dynamically removing it if the relevant variable is empty, but that seems clumsy to me.
Also, r1143 will break any instance of array_insert; all places it was called (e.g. the template skeleton) relied on it modifying $array by reference but now it's returning a value - and wetem (amongst other places) isn't picking that up. I'll fix it tomorrow if you don't get to it first.