-
Splitting this topic into its own from the original Selectbox topic...
Please read the bold characters below and tell us your opinion!
-
Chosen is slick, really slick. It even works great on iPad.
I'm not sure what the options on Noisen are. Normal topic privacy setup would imply:
* topic starter only can see it
* topic starter and their contacts can see it
* topic starter and moderators
* anyone who can access the board
This should cover all the main cases. I don't think list of groups is needed, and I think that if it's down to the above, a simple number could deal with it.
-
Chosen is slick, really slick. It even works great on iPad.
Unfortunately, I tested it on my iPod and it has issues:
- there are graphic glitches inside the input boxes. Minor, but visible.
- because there's always a combo box, Safari will enlarge the page and focus on the input box. And because of that, you get the keyboard and you suddenly can't see the list of items at all... Good luck browsing it!I'm not sure what the options on Noisen are.
Come to think of it... I doubt anyone even noticed the feature itself.
Go to any non-blog board index.
Find a topic started by you. Just below the title, there's a faint icon of a key visible at the left. Click it. There you go...
Now, click 'Groupes Spécifiques' (I'm sorry that it was never localized to English...), which means 'Custom Groups', and a new popup shows up to its right, where you can fine-tune your selection.
I think it's a pretty neat way of selecting privacy... And because I'm not going to keep 'groups' in the list anyway, I was thinking I could instead show a list of checkboxes for friends -- either a complete list of friends, or simply 'friend types', something I will probably add in the future (close friends, family, co-workers, etc.) I'm just not sure whether it'll be something that doesn't hurt performance, really.Normal topic privacy setup would imply:
* topic starter only can see it
* topic starter and their contacts can see it
* topic starter and moderators
* anyone who can access the board
This should cover all the main cases. I don't think list of groups is needed, and I think that if it's down to the above, a simple number could deal with it.
- if we add a new option in the future and it should be placed before 'Just Me' in the list (because 'Just Me' is really the deepest level you could have), the select box would have <option>1</option><option>3</option><option>2</option>, to simplify. I'm not a big fan... Although, who cares :P
- friend granularity would be hard, or even impossible, with a simple number. Unless we give friend groups a unique ID for everyone (like, I'm new to this site, and the first friend group I'll create will have id #2356 because there are already 2000+ other contact lists), and we start these IDs above 3 or 4 (used for default, logged in members, and all friends.)
Maybe we could also have 'by default' a general contact list for everyone, created when they first make a friend, and then they can add their contacts to other contact lists (with their own contact list ID), mutually exclusive or not... hmm... It would probably make a lot of sense, and would certainly help with sql queries.
-
Unfortunately, I tested it on my iPod and it has issues:
- because there's always a combo box, Safari will enlarge the page and focus on the input box. And because of that, you get the keyboard and you suddenly can't see the list of items at all... Good luck browsing it!
either a complete list of friends, or simply 'friend types', something I will probably add in the future (close friends, family, co-workers, etc.) I'm just not sure whether it'll be something that doesn't hurt performance, really.
I've seen plenty of requests for the 'just me' and 'just me + moderators' setup. I'm really not comfortable with having just me always show moderators, though. I'm thinking the journal/blog setup where you have private posts and public posts and some of those are going to me really 'me only' items. I'd be OK with admins (only) having access but not moderators. Just me implies a certain level of privacy, after all.- friend granularity would be hard, or even impossible, with a simple number. Unless we give friend groups a unique ID for everyone (like, I'm new to this site, and the first friend group I'll create will have id #2356 because there are already 2000+ other contact lists), and we start these IDs above 3 or 4 (used for default, logged in members, and all friends.)
It would probably make a lot of sense, and would certainly help with sql queries.
-
- No, it's something else.
- The keyboard sometimes gets hidden automatically. It's still not very practical. Yet, it looks good, needless to say... But I'm sure we can do better. Heck, actually I'm thinking a lot about going for the 'Wedge menu' style, like John suggested... Plus, I just know myself, I'll probably just rewrite the whole thing from scratch... Maybe using ideas and samples from other libraries.
- Okay for moderators, you've convinced me they should be kept out. Still, I think it's best to leave that option aside -- just have 'Just Me'.
- I'd like to have some user opinions on contact lists. Who do you think does it best? Noisen.com (if you ever used it)? Facebook? Google+? SMF?
- Additionally, how would contact granularity hurt performance more than a general 'My Contacts' choice...? Considering we'll be hitting an extra table in every case?
-
- Okay for moderators, you've convinced me they should be kept out. Still, I think it's best to leave that option aside -- just have 'Just Me'.
- I'd like to have some user opinions on contact lists. Who do you think does it best? Noisen.com (if you ever used it)? Facebook? Google+? SMF?
- Additionally, how would contact granularity hurt performance more than a general 'My Contacts' choice...? Considering we'll be hitting an extra table in every case?
If it's kept as a simple number, it's possible to solve a touch more efficiently, because what you can then do is figure out who the users are who have the current user as a friend, and turn it into (where topic starter = me OR (privacy = friends AND topic starter IN (list of people who friended me)).
The one thing to realise about topic privacy, and this is quite important: it is going to suck compared to board privacy. It's unavoidable, because there's no way to do it in a way that adds extra conditions that can be evaluated without ORs (except in the just me or everyone cases) - ORs are bad for performance because they're an extra branch and often virtually a sub-query in their own right.
-
No, I had a specific reason for 'just me + moderators'. There's a poor-man's helpdesk, there's for discussing reasons for bans etc, discussing 'application forms' on clan type sites. The list goes on, and it's much easier for us to implement it there in the core than it would ever be to bolt it on later.
- I'd like to have some user opinions on contact lists. Who do you think does it best? Noisen.com (if you ever used it)? Facebook? Google+? SMF?
It was kind of rhetorical. Noisen has asynchronous contacts, plus the ability to hide selective contacts from viewers. That one will be in Wedge too, once I get to implementing contact lists...Because it wouldn't just be an extra table hit. If you want to store anything other than a simple number, you have to either implode it and store it inline, or you have to store it in another table, which means that's *two* extra tables vs what we have now, not one.
Although I'm not sure we'll be using that field much in the future... But IIRC there are reasons to leave it in.
If you start setting privacy settings on everything in your profile for instance, it'll be a disaster if you have to set multiple contact lists in each. I think it's much smarter to encourage people to put their 'safe' friends into a special list, and give that list all permissions, and deny the rest to anyone else -- guests, members and contacts that aren't in the safe list.And believe me, the notion of putting an imploded field in the topics table is a no-no, seeing how it would make the entire topics table an order of magnitude slower because right now there are no variable-size fields in it, which is a very, very good thing.
I'm thinking of a structure like this:
wedge_members
...
contacts (1,2,3,4,5)
wedge_contact_lists AUTO_INCREMENT 10
id_member (id_owner?)
id_list
name
description (?)
wedge_contacts
id_member
id_list
(possibly store the list's id_owner as well, not mandatory)
So when we check for a topic's privacy validity, we just retrieve its privacy setting, if <10 (for instance) it's a special setting like 'guests' or 'members' or 'just me' or 'just me + mods' or anything else we can think of, if >=10 it's a contact list, so we INNER JOIN wedge_contacts AS c ON t.privacy = c.id_list AND c.id_member = {int:myself}, or something like that...
Well, that's the basic idea.If it's kept as a simple number, it's possible to solve a touch more efficiently, because what you can then do is figure out who the users are who have the current user as a friend, and turn it into (where topic starter = me OR (privacy = friends AND topic starter IN (list of people who friended me)).
Does that make sense...?The one thing to realise about topic privacy, and this is quite important: it is going to suck compared to board privacy. It's unavoidable, because there's no way to do it in a way that adds extra conditions that can be evaluated without ORs (except in the just me or everyone cases) - ORs are bad for performance because they're an extra branch and often virtually a sub-query in their own right.
I'm not making advances when it comes to the select box, BTW... I'm still unsure where to start from!
-
Hmm.. Okay, okay... But I don't see this being useful in Thoughts, for instance...
It was kind of rhetorical. Noisen has asynchronous contacts, plus the ability to hide selective contacts from viewers. That one will be in Wedge too, once I get to implementing contact lists...
Oh... I see what you're talking about -- selecting several contact lists for viewers.
For instance -- if you have a family-only post, you select your family. If you have a work-only post, you select your co-workers. If you have a friends-only post, you select your friends. Among which can be some of your family and co-worker list members, of course. The point is having the ability to put some people into multiple lists. Then, when a list is modified, the 'buddy_list' field in {db_prefix}members is updated to reflect the entire list of contacts.
Although I'm not sure we'll be using that field much in the future... But IIRC there are reasons to leave it in.
If you start setting privacy settings on everything in your profile for instance, it'll be a disaster if you have to set multiple contact lists in each. I think it's much smarter to encourage people to put their 'safe' friends into a special list, and give that list all permissions, and deny the rest to anyone else -- guests, members and contacts that aren't in the safe list.
So when we check for a topic's privacy validity, we just retrieve its privacy setting, if <10 (for instance) it's a special setting like 'guests' or 'members' or 'just me' or 'just me + mods' or anything else we can think of, if >=10 it's a contact list, so we INNER JOIN wedge_contacts AS c ON t.privacy = c.id_list AND c.id_member = {int:myself}, or something like that...
As I see it, it's (WHERE i'm_admin OR topic starter = me OR (privacy >= 10 AND me IN (SELECT id_member FROM wedge_contacts WHERE privacy = id_list))
Does that make sense...?
I don't even know HOW exactly it's not KILLING performance, this one...
I'm not making advances when it comes to the select box, BTW... I'm still unsure where to start from!
-
Not so important for Thoughts, but it is important for topic privacy.
I'm starting with the basics:
1- everyone
2- members
3- just me (author & admins)
This is what will be in the code for now.
I'm turning oThought's privacy array into an object so we can easily manipulate item position within the select box.
Dunno if 'everyone' should be set to zero, though...It was, yes, which is why I didn't really get into it. Facebook is now sort of asynchronous, Google is asynchronous by design and creates multiple lists for you to be asynchronous with. But I think that might be a bit too complex for what's needed in Wedge.
Here's what I've come up with for now... Do tell me if it seems reasonable.
#
# Table structure for table `contact_lists`
#
CREATE TABLE {$db_prefix}contact_lists (
id_list mediumint(8) NOT NULL DEFAULT 0,
id_owner mediumint(8) NOT NULL DEFAULT 0,
PRIMARY KEY (id_list),
KEY member (id_owner)
) AUTO_INCREMENT=10 ENGINE=MyISAM;
#
# Table structure for table `contacts`
#
CREATE TABLE {$db_prefix}contacts (
id_member mediumint(8) NOT NULL DEFAULT 0,
id_list mediumint(8) NOT NULL DEFAULT 0,
is_synchronous tinyint(1) unsigned NOT NULL DEFAULT 0,
position tinyint(4) NOT NULL DEFAULT 0,
updated int(11) NOT NULL DEFAULT 0,
hidden tinyint(1) unsigned NOT NULL DEFAULT 0,
PRIMARY KEY (id_member, id_list)
) ENGINE=MyISAM;
is_synchronous could be a boolean set to true if we find out that the target member also has you in their list(s). It's currently used in your contact lists, where they're separated by sync status. Also, 'position' is the position inside the list, which you can manually modify. 'updated' is the last updated date for the contact list -- although we could have both a created and updated field... Or none at all. 'hidden', as I mentioned before, prevents anyone (but the list owner) to see the name show up in the contact list.
Now we'll need to update the Import tool to actually convert buddy lists to contact lists (and automatically create a default list for every user that has at least one buddy.) I don't know if it's best to do it from the importer tool, or from within Wedge if the table is empty etc... I'd say the importer.
Thorsten, are you reading this? :PIf buddy_list is a single list of comma-separated users, it's queryable without having to do an extra query. It'll be slow, but it'll be doable.
The problem with subselects, is that they often (always??) require a table scan to complete, even if done on the proper index...
This is something that doesn't happen with INNER JOINs.If it's *anything* else, we'll have to query it independently, decode it (I'm assuming unserialize), then build a query based on that. It's going to suck in performance terms.
It's still multiple lists to manage, and I just don't think that's entirely necessary. On a social network like Facebook or Google, where you're inherently sharing information that may be suitable for some but not all people, it's important to have that granularity. On a forum, it just isn't necessary.
I don't know about that. I think contact lists would have more pros and cons. And no one is forced to create multiple lists... It's just good to have them.
Heck... Either lists (id >= 10) or 'all contacts' is easily doable in a subselect. We either select id_members who are associated to our stored id_list (>= 10), or id_members who are associated with the id_member owner of the list. In which case it'd be best to store the list owner's id in the contacts table as well, to save time. Or just do a find_in_set on their buddy_list of course... But buddy lists are limited in size, unlike the contacts table.Don't inner join. I get where you're going but inner join is a bad place to be. All of the queries (or at least, all the *important* queries that rely on topic visibility; there are many but the important ones like topic display for example) rely on having only one result returned, and inner join will generate multiple rows in the result.
Also, as I said above, from my experience, subselects don't use indexes. If you could help here, because you're the mysql specialist out of us both, it'd be nice to be able to use subselects, if only because it'd make life a hell of a lot easier when using {query_see_topic} in the code I'll eventually import from the Noisen diff...It's hurting but you probably wouldn't notice it until you got to really huge boards with many many many topics.
Then again, it may never happen at all.
The only performance bottleneck I've been told so far, is the random list of items in the media homepage. That's because it retrieves all entries, randomizes them, and returns the first few. I still have an entry in my to-do-list to add a pseudo-randomizer variable in each item...I'm not making advances when it comes to the select box, BTW... I'm still unsure where to start from!
I should be getting ready to analyze the code for each plugin, and determine whether I can 'merge' at least two of them to get the 'best of both worlds' (or more.)
The last selectbox I suggested has issues in my iPod, as I already said. The keyboard problem is gone for now (now it automatically disappears a second after clicking...), but for instance, if you have a multi-select box, every time you click something in Chosen, the list closes and you have to reopen it... A 'regular' selectbox object will actually stay opened to let me select other options. Regular wins when it comes to accessibility...
-
So we need to establish a list of privacy IDs and their corresponding meaning...
I'm starting with the basics:
1- everyone
2- members
3- just me (author & admins)
Now we'll need to update the Import tool to actually convert buddy lists to contact lists (and automatically create a default list for every user that has at least one buddy.) I don't know if it's best to do it from the importer tool, or from within Wedge if the table is empty etc... I'd say the importer.
Slower than its equivalent subselect with a secondary table?
The problem with subselects, is that they often (always??) require a table scan to complete, even if done on the proper index...
This is something that doesn't happen with INNER JOINs.
But you need to be careful. INNER JOIN may be faster but you then have to process the results of a result-table that now has multiple rows, potentially many many rows you didn't want in the first place.
It's one of the reasons the board index query is fucked up, because it inner-joins the moderators table to the list of boards, so if you have 100 boards, each with 2 moderators, you get 200 rows back of which most of it is duplicated.The only thing we can/may/should/will/shall/would/whatever store in the data field of the member table is the list of contact lists you have. $contact_lists = unserialize($member['data']['contacts']) or something. Would need to be done on every page load (to get the list of friend groups for thought privacy), and other uses (such as users viewing a profile etc) can be done through a quick sql query.
So... You're suggesting no lists at all? Just plain contacts...?
I don't know about that. I think contact lists would have more pros and cons. And no one is forced to create multiple lists... It's just good to have them.
Heck... Either lists (id >= 10) or 'all contacts' is easily doable in a subselect. We either select id_members who are associated to our stored id_list (>= 10), or id_members who are associated with the id_member owner of the list. In which case it'd be best to store the list owner's id in the contacts table as well, to save time. Or just do a find_in_set on their buddy_list of course... But buddy lists are limited in size, unlike the contacts table.
Also, as I said above, from my experience, subselects don't use indexes. If you could help here, because you're the mysql specialist out of us both, it'd be nice to be able to use subselects, if only because it'd make life a hell of a lot easier when using {query_see_topic} in the code I'll eventually import from the Noisen diff...
My idea was to have such boards rely on Wedge...
Then again, it may never happen at all.
The only performance bottleneck I've been told so far, is the random list of items in the media homepage. That's because it retrieves all entries, randomizes them, and returns the first few. I still have an entry in my to-do-list to add a pseudo-randomizer variable in each item...
Right now, access is controlled at board level. Once you enter a board, you don't have any incremental performance concerns about access rights. A board with 1 topic has the same overhead as a board with 1m topics in it as far as assessing access to that board goes.
If you have topic-specific granularity, you have to do more work to assess it, specifically there's an extra overhead on the board index, message index, display, attachments... anywhere that has to assess topic access, which is more complex than board access (because you have to implicitly do both, though you can do it so that board access is evaluated first and if that's not going to let them in, you can skip the topic access checks)
Consequently it must slow things down compared to a base SMF install, but you probably wouldn't notice it on Noisen until you got to boards that had many many many topics, because the one-off cases like attachments or topics themselves, the incremental cost is not significantly higher, it's for the cases where you're assessing a lot at once (like board index, message index)Maybe not, but none of the ones you've seen thus far are ideal, so you're going to end up Frankensteining two or more together, and then putting your own spin on it anyway...
-
So we need to establish a list of privacy IDs and their corresponding meaning...
I'm starting with the basics:
1- everyone
2- members
3- just me (author & admins)
Other possible privacy values..?
"18 years old or older"
"13 years old or older"
Etc...
I'd say it's possibly better than having a 'mature' flag on posts.The table structure seems straightforward enough to me.
As for when to do it, it should be done from the importer.
Slower than its equivalent subselect with a secondary table?
You're right that it will cause a table scan, but only if the subquery is used IN () directly (this is not something I've done very often) but it's interesting to note that 3 years ago it was flagged as being solved in MySQL 6 as per http://bugs.mysql.com/bug.php?id=18826
Is this just in an IN () situation?
I mean, could it work with something like SELECT t.id_member WHERE (SELECT TRUE FROM wedge_other AS o WHERE t.id_member = o.id_other).....? (Just off the top of my head.)But you need to be careful. INNER JOIN may be faster but you then have to process the results of a result-table that now has multiple rows, potentially many many rows you didn't want in the first place.
Anyway -- query_see_topic does an inner join, and it's fucking ugly because it makes inserting the query_see_topic variable more compatibled than, say, query_see_board. Having it use an IN() would make it much more elegant.It's one of the reasons the board index query is fucked up, because it inner-joins the moderators table to the list of boards, so if you have 100 boards, each with 2 moderators, you get 200 rows back of which most of it is duplicated.
If the list's owner is stored in the table of contacts, why does it even need to be in the members table at all? Index the owner and you're golden.
Personally I just don't see the point. I can't think that many people are going to create topics that are visible to only a subset of a subset of friends.
If anyone is reading this -- please tell us whether you think that it would be nice to be able to create contact lists (i.e. friends, family, work...) and whether you'd use the feature to fine-tune your topic/board privacy settings, or you just wouldn't bother yourself?Then again, I know it happens on LiveJournal which does make it a viable target for us (blogging context), I guess.
Consequently it must slow things down compared to a base SMF install,
-
Other possible privacy values..?
"18 years old or older"
"13 years old or older"
Etc...
I'd say it's possibly better than having a 'mature' flag on posts.
I was thinking maybe add a 'privacy' field for contact_lists as well... Although granularity for this would be hell -- what if I want my contact list to be able to browse the list they're in? Or if I don't want to? Shall I be able to select multiple viewing groups...?
Uh. v6 is a long time from now...
Is this just in an IN () situation?
I mean, could it work with something like SELECT t.id_member WHERE (SELECT TRUE FROM wedge_other AS o WHERE t.id_member = o.id_other).....? (Just off the top of my head.)
I believe it can be made to work like that but I'm not sure, I don't think in subselects very often.I'm not sure... If there's an IN (), we'll also be getting multiple entries just the same...?
Anyway -- query_see_topic does an inner join, and it's fucking ugly because it makes inserting the query_see_topic variable more compatibled than, say, query_see_board. Having it use an IN() would make it much more elegant.
The ability to create a blog for your professional friends... And another for your drinking buddies. (Same goes for topics, although less important.)
LJ is definitely not 'the' popular blog platform these days, but they still have got 'something'. They're also the ones who have rotating avatars, which I like... (Although not THAT much, eh.)
(We could also offer to disable topic privacy settings...)
If anyone is reading this -- please tell us whether you think that it would be nice to be able to create contact lists (i.e. friends, family, work...) and whether you'd use the feature to fine-tune your topic/board privacy settings, or you just wouldn't bother yourself?
(New topic, perhaps?)
-
Okay, I tested all of these:
SELECT * FROM _messages AS m WHERE id_msg IN (SELECT n.id_msg FROM _messages AS n WHERE n.id_msg < 100000);
SELECT * FROM _messages AS m WHERE (SELECT 1=1 FROM _messages AS n WHERE m.id_msg = n.id_msg AND n.id_msg < 100000);
SELECT * FROM _messages AS m INNER JOIN _messages AS n ON n.id_msg = m.id_msg AND n.id_msg < 100000;
They all returned 685 entries on Noisen, so that's the exact same request being done. NOTE, though, that because both tables are the same, MySQL may be applying specific optimizations that wouldn't take place if using two different tables. You may want to rewrite the queries with two different tables and tell me your results.
The first query does a full table scan and returns in X milliseconds. The second query does the exact same full table scan, and returns in X milliseconds (sometimes a tad more but they're all very variable). Finally, the last query does an index search, and returns in X*2 milliseconds. It's also the most 'stable' of all queries, but it's stable in that it's always slower than even the slowest return time for the first two queries...
As a reminder, the messages table on noisen has 30k+ entries. I could do a test on Cynarhum.com (220k+ entries), but I suspect I'll get similar results overall...
Talk about clusterfuck!This seems like it's potentially *very* complicated and something I'm not entirely sure I want to get into, to be honest.
This sounds to me like overthinking for SCIENCE!
But the bug report was 3 *years* ago. I have no idea whether that's since been backported to 5.5 or 5.6 or 5.WTF any more. I think we need to try it sometime.
I believe it can be made to work like that but I'm not sure, I don't think in subselects very often.
It sort of depends, really. There are times it will, times it won't. It also depends on whether DISTINCT is present or not, which would solve the multi-row return case regardless of joins or in clauses.
I'm currently helping her set up a SOAP client in PHP for a WSDL app at Oracle. Another clusterfuck, BTW... We have no idea what functions we're supposed to call (and AFAIK there's no way to request a list of available methods once the custom object is created), how we're supposed to identify, etc... Thank you Oracle for zero documentation. Plus it doesn't help that neither her or I have any prior experience with SOAP... :-/
I don't know why I'm mentioning that... Maybe there's a SOAP specialist around here :PThe ability to create a blog for your professional friends... And another for your drinking buddies. (Same goes for topics, although less important.)
They're the only popular hosted blog platform I know of that still functions with a community aspect to it.
In the end, even as long as there is one line of code for it, it is going to be slower than SMF.
We can mitigate it (and given other things will be altered, it may even out in the end) but it must make a difference.
Because that's the thing here... When we work with normal-sized boards, there's no such thing as a slow query. Start adding bot posters or scraping or simply have a hugely successful site, and you get your first performance issues.If anyone is reading this -- please tell us whether you think that it would be nice to be able to create contact lists (i.e. friends, family, work...) and whether you'd use the feature to fine-tune your topic/board privacy settings, or you just wouldn't bother yourself?
(New topic, perhaps?)
-
The first query does a full table scan and returns in X milliseconds. The second query does the exact same full table scan, and returns in X milliseconds (sometimes a tad more but they're all very variable). Finally, the last query does an index search, and returns in X*2 milliseconds. It's also the most 'stable' of all queries, but it's stable in that it's always slower than even the slowest return time for the first two queries...
But if we start thinking this way, then we might as well drop the concept of topic privacy entirely...?
Nope, not backported. From what I gather in the link you posted, subqueries are implemented in 5.x in a way that they can't use an index, and it works in 6.x not because they fixed that bug, but because they rewrote their subquery code.
I didn't, until we dropped support for MySQL 4.0... And, turns out, I always hated doing inner joins...
My girlfriend suggested I use DISTINCT for these no later than last night.
I'm currently helping her set up a SOAP client in PHP for a WSDL app at Oracle. Another clusterfuck, BTW... We have no idea what functions we're supposed to call (and AFAIK there's no way to request a list of available methods once the custom object is created), how we're supposed to identify, etc... Thank you Oracle for zero documentation. Plus it doesn't help that neither her or I have any prior experience with SOAP...
I don't know why I'm mentioning that... Maybe there's a SOAP specialist around here
Ah yes, you just reminded me why I'm looking up to them... LJ is the only blogging platform that actually encourages communication between blog authors, rather than having parallel blogs with their own comment authors and such. It's like LJ is a huge forum with boards set up as blogs, just like on Noisen. Back when I created Noisen (2007-2008), LJ was the only similar example, and I actually used them as an example of *why* it would eventually work. Noisen was pretty much supposed to be the 'French LJ'... Too bad it never really got momentum. I'm not very good at advertising my work. I prefer development work.
Because that's the thing here... When we work with normal-sized boards, there's no such thing as a slow query. Start adding bot posters or scraping or simply have a hugely successful site, and you get your first performance issues.
-
I'd love to see the results of EXPLAINs on those queries.
I should have copied the explains but they were nothing special -- just had the number of processed rows over 30k in the first two queries, and at 500+ in the last.But if we start thinking this way, then we might as well drop the concept of topic privacy entirely...?
Basically, if you tell bloggers that they have to create a new blog for their private friends, they may say "okay", or they might just turn away and leave. I don't know, I just implemented stuff that people asked of me back in the day... And it seemed quite logical to do it.
The only issue with topic privacy, to me, is not really about performance, but mostly about not ever forgetting to add query_see_topic in every single topic query I'll have. That was a PITA, but I'm ready to do it again... (Especially since I now have the diff for that... And I've already decided that I'll mark every {db_prefix}topics as {db_prefix}topics_done or something so that once everything from the diff is implemented, I can simply look for {db_prefix} to find any remaining offenders...)I thought 5.5 was actually a bastardisation of the 6.x branch anyway.
SOAP is... evil, and I'm not just saying that as a smelly hippie code hacker :P But if it's involving WSDL... WSDL is a language that indicates what services exist at a given URL, and what inputs are expected and what outputs will be given. It's sort of like XML-RPC but more convoluted IMO. (Yes, I've done SOAP work. It's not that exciting, but it should be manageable. It really depends whether you're doing it all by hand in PHP or using something like the Zend_Soap components.)
I don't know, if you're up for it and if you think you can help, would you be willing to help her by e-mail...? We already spent two evenings on the little fucker... (The first one was devoted to installing a virtual host on her PC, which didn't work until I realized the error code was 403 and I'd actually forgotten to add +Indexes to the httpd conf file... Oops. But she was like, "wow", when I fixed it. I liked that. :P)*nods* And given that LJ is really not working out so well at the moment (it's not been the same since it was sold off a bit back), maybe we should be pushing that harder.
Yup, but we can still try and optimise as best possible for those cases.
BTW, did you ever check out the SQL queries for the thought system? I'm always afraid of forgetting something in them... Most of the stuff is in Ajax.php where I nearly committed something that would have allowed anyone to modify anyone else's thoughts just by editing their HTML source... Oops. That was scary.
-
If anyone is reading this -- please tell us whether you think that it would be nice to be able to create contact lists (i.e. friends, family, work...) and whether you'd use the feature to fine-tune your topic/board privacy settings, or you just wouldn't bother yourself?
Unfortunately my lot are the laziest mofo's ever.
Heck, they don't even post!
-
Added a poll, and I encourage everyone to vote...
Posted: December 1st, 2011, 07:49 PM
Very quiet here this evening...... :P
-
It doesn't help that I've been wrapped up in a bastard funk mood lately that has just flatlined my enthusiasm for just about everything :( It's just like I've been so overloaded with *stuff*, both digitally and in real life.
Posted: December 2nd, 2011, 12:58 AM
Also...Yes -- everyone, or just me (i.e. just the ability to write drafts...)
-
It's not exactly the same -- a draft is stored in a special area of your profile, while a 'privacy' draft is stored inside your forum/blog and you can access it easily until the moment you decide to make it public.
Would that be disorientig...?
Regarding the poll, it would seem that:
- a third of our users don't care about topic privacy settings,
- the rest does,
- everyone but you seems to be interested in multiple contact lists, rather than a single contact list.
I'm not sure I can make a decision based on a relatively small sample of users (11 people, 9 not including us.)
It's not like it's a piece of cake to implement this... Although it's not going to be a disaster, either :P
Oh, and regarding your post-moderation UI issues - how about you set it aside and come back to it in a month or two?
-
I think it would potentially be confusing to treat it as a draft, but personal posts I can see the logic of, e.g. blog post with very personal details.
Re moderation, I think I'm going to have to leave it for now and come back to it.
Also, one thing that I'd address: how many people are voting on what sounds useful vs what they would actually use if it were available...?
-
Also, one thing that I'd address: how many people are voting on what sounds useful vs what they would actually use if it were available...?
-
I think option one is the easiest, as any other combos really can be handled in Personal Messages.
-
Not if you want a poor man's helpdesk, for example... There are plenty of cases where you might want to allow multiple users but not everyone to see a given topic.
-
Good idea though: sending pms to a contact list :P
-
Now I have time to come back and look properly, I've realised I select the wrong option :unsure:
If I could change my vote, I'm torn between "same as SMF" (the one Pete voted for) and the "same as Google+ circles" below it. I can see value in both approaches.
Just to add, I think I'd vote "same as Google+ circles", with the caveat that while I wouldn't want ordinary members having these facilities available to them, I can see them being useful to staff.
-
/meinvokes the demons of moderation and allows people to change their vote!
-
* PantsManUK sacrifices a chicken to the demons in thanks.
-
It seems to me that what is wanted is the ability for the user to
* create user-controlled custom groups, to which the user can add whomever they want
* An option to automagically maintain a user-controlled custom group of "buddies"
* create user-controlled custom profiles, in which the user can specify whichever permission they want for each group (general or custom)
* create the ability to apply custom profiles to topics, rather than to boards. topics without profiles would use the board's profiles. Permission on any topic would be the most restrictive of the combination of board and topic permission. This will be tricky, combining all those A's, D's, and X's properly for people who are in multiple groups, where they might have A, D, and X permission on the same pouvoir.
SMF 2.0's interface for groups would work OK, but you would only want the current permissions profile interface available as "advanced permission setting" option. You'd want a dead simple interfact that allows them to choose only "groups which can see this post", or whichever set of options you want to present, but then passes that information to a method that then either creates a proper permission profile, or else re-uses one of the standard permission profiles (which must be provided, and you cannot allow ADMIN to delete them, or else the whole thing breaks).
-
This will be tricky, combining all those A's, D's, and X's properly for people who are in multiple groups, where they might have A, D, and X permission on the same pouvoir.
The real problem isn't even figuring out ADX permutations, the trick is to make it parsable in a fashion that means it can trivially be dropped in (or at least, insertable easily) into some of the existing queries and to do so without killing performance. The real hurt is going to come on things like unread and the message index where you have to evaluate a lot of topics at once, and even the stuff I did in SimpleDesk (which is verging on scary-complex, because that's author+staff, which is a trivially definable group and isn't set per topic but per board) isn't as complex as this is.
-
Yes, contact list are pretty much like membergroups in SMF...
In fact, Noisen.com offers (or used to offer -- I think I disable it) the ability for users to create their own custom membergroups, for which they'd then be able to give proper privacy settings. It did work, but I felt uncomfortable giving users access to the admin area, even if just for that particular membergroup page, plus it was a bit confusing to end users anyway.
That's why I want to go between the two opposites - and use a technique that's closer to what Facebook and Google+ have.
I don't know if I'll go all the way Ajax to manage your lists, but I can sure give it a try...
Anyway, the poll's results are now clearly pointing towards a contact list solution, and I'll deal with that later this week. I just don't know if I'll do it before or after the selectbox rewrite...
-
It will be complicated.
And scary, because you need to allow "everyone" to use stuff that was meant for only admin.
Easy enough to make the user moderator of the user's membergroups. New to make them able to create usergroups.
New and confusing to give them a simple new UI, then putting that output into creating an actual permission profile.
And new to let them apply a permission profile to a topic, a thing that doesn't have permission profiles in SMF.
And new to force Wedge to honor per-topic permission profiles.
When do you feature-lock this puppy?
-
This wouldn't use the admin member group management area but be custom. It doesn't take long to whip up such things once we know what we're trying to build.
There's no way it would create a full member permissions profile, as that would massively overcomplicate something that doesn't need that level of depth. Permissions are inherently much simpler.
As far as implementing privacy goes, Nao already did a fair amount of the hard work before in Noisen.
-
@Pete> I'm still unsure whether our query_see_topic should be before WHERE (i.e. use an inner join just like at noisen), or after it (i.e. use a subquery). Did you think about it..?
It will be complicated.
And scary, because you need to allow "everyone" to use stuff that was meant for only admin.
Easy enough to make the user moderator of the user's membergroups. New to make them able to create usergroups.
The main reasons for not using membergroups are (1) the fact that the UI is complex, although it could be solved by writing an alternative UI for non-admins (I couldn't care less at the time...), (2) if you're someone 'popular' in a certain community and lots of people put you into their follow/contact lists, you'll end up with dozens of membergroup ownerships in your member account. If a forum is extremely popular, you could end up with your group field being overfilled (its size is 64KB at best IIRC). This could be solved by having a new table where members and membergroup relationships are stored. This isn't something I've explored so far... Maybe Pete has an opinion on this...?When do you feature-lock this puppy?
It will be feature-locked when it goes beta, I suppose. (And even then, we'll still be flexible about adding stuff that make sense.)
We used to have schedules and everything... Back when our relationship with SMF was bad at best. We wanted to release something in mid-2011 but because SMF eventually went gold around that time, it pretty much freed us of any restraints. Even when the main Noisen features are finally integrated, I'll still be wanting to finetune the overall design of Wedge because I'm not entirely happy with it. An important milestone will be when I install Wedge on this very website. This could be done today because the thought system is now ready to use (something Pete was adamant should be kept on Wedge.org), but thoughts led me to rethinking the privacy system, which in turn led me to rethink the select box stuff... Etc, etc. It's probably never going to end, but what matters is that Wedge is constantly evolving and not staying in its corner without any updates etc.
-
I guess I can still be convinced to use membergroups, but I'll need someone to debate with me
This could be solved by having a new table where members and membergroup relationships are stored. This isn't something I've explored so far... Maybe Pete has an opinion on this...?
That said, I'm inclined to go with one row per association rather than denormalise the data and stuff it in a text. It's cleaner, easier to maintain, less likely to get screwed up (cf the comments in Load.php about loading the user's groups and 'history shows that these can go bad')This could be done today because the thought system is now ready to use (something Pete was adamant should be kept on Wedge.org)
@Pete> I'm still unsure whether our query_see_topic should be before WHERE (i.e. use an inner join just like at noisen), or after it (i.e. use a subquery). Did you think about it..?
-
I guess I can still be convinced to use membergroups, but I'll need someone to debate with me
Also, any extra feature added to contact lists could be used as well in regular membergroups: 'hidden' state (rather than hide the entire membergroup from view, why not hide just one person from view...?)
I didn't really consider this whole aspect because I'd given up on it at Noisen, but I didn't as well consider why I removed it from noisen -- primarily the UI issues, which I couldn't bother to deal with. Noisen's friend lists use the UltimateProfile code to manage friends, and this is something I'm going to rewrite anyway -- it's not like UP is BSD or whatever, AFAIK...That said, I'm inclined to go with one row per association rather than denormalise the data and stuff it in a text. It's cleaner, easier to maintain, less likely to get screwed up (cf the comments in Load.php about loading the user's groups and 'history shows that these can go bad')
Of course, it would also get us rid of these annoying find_in_set calls...I've thought about it and I just don't know. I can see benefits both ways of doing it - doing it before the WHERE will probably make it faster, but doing it after the WHERE only makes it easier to maintain and makes it easier for modders to apply it properly in their own things later on.
Either way, adding in {query_see_topic} "forces" us to always declare topics AS t, because the extra code will definitely use the alias. But IIRC it's the same with query_see_board anyway...
I'll just add that query_see_topic can be moved before or after the WHERE whenever we want -- it's not that big a deal because it's called at most 60-70 times in our code so it can be redone manually...
-
Also, any extra feature added to contact lists could be used as well in regular membergroups: 'hidden' state (rather than hide the entire membergroup from view, why not hide just one person from view...?)
On the other hand, if it were turned into a global ignore feature to hide users who are a pain in the arse... that's something else entirely.I didn't really consider this whole aspect because I'd given up on it at Noisen, but I didn't as well consider why I removed it from noisen -- primarily the UI issues, which I couldn't bother to deal with. Noisen's friend lists use the UltimateProfile code to manage friends, and this is something I'm going to rewrite anyway -- it's not like UP is BSD or whatever, AFAIK...
I'm not sure about rewriting all of the membergroup-related queries to use an extra table though...
Of course, it would also get us rid of these annoying find_in_set calls...
I'd personally rather use the slower, but more 'natural' one. (Maybe it'll become a bigger issue if we ever have some successful installs.)
Either way, adding in {query_see_topic} "forces" us to always declare topics AS t, because the extra code will definitely use the alias. But IIRC it's the same with query_see_board anyway...
I'll just add that query_see_topic can be moved before or after the WHERE whenever we want -- it's not that big a deal because it's called at most 60-70 times in our code so it can be redone manually...
Unless it's made into a subquery, it's impossible to avoid forcing a table join with alias.
-
I'm going to have to get started on this before I use Wedge on Wedge.org, I guess...
So, things to do: (feedback welcome.)
- query_see_topic everywhere (gonna be a lot of fun...)
- contacts table (still unsure whether 'contacts' is the right term to replace the awful 'buddies'. I still like 'friends' better, but if we want it to be usable by work colleagues and family etc, it should be a more generic term I think.)
- importing 'buddies' into contacts table at SMF import time, writing the UI to manipulate contact lists (will take it mostly from wedge.org...)
- would like to know the current 'official' positions of everyone on contact lists -- is it okay to use a contacts table, or should we ditch the whole concept of contacts, and instead allow everyone to create their own private/public membergroups?
-
I'm in agreement with the use of 'contacts' vs 'friends'.
As far as going with contacts, having a facility to track such things is no bad thing, especially as it covers stuff like easy-contact lists for the purpose of sending PMs, though I guess it depends how readily users can make contact lists themselves and how many lists they can have.
-
One of my issues with contact lists is due to the privacy of current thoughts at wedge.org.
It'd be way easier for me to set privacy to id_group=friends group, like I did on the test site. But then new thoughts can't be posted to that group only because it's not implemented in the UI. And doing it... Well, maybe users would start getting confused between membergroups and contacts when time comes to choose a target privacy... thus, headache.
Additionally, I'm starting to think that I'll need extra columns for privacy... Just having a privacy flag won't cut it. I need id_group and id_contact_list fields, or something. Hm...
-
- contacts table (still unsure whether 'contacts' is the right term to replace the awful 'buddies'. I still like 'friends' better, but if we want it to be usable by work colleagues and family etc, it should be a more generic term I think.)
- would like to know the current 'official' positions of everyone on contact lists -- is it okay to use a contacts table, or should we ditch the whole concept of contacts, and instead allow everyone to create their own private/public membergroups?
-
Okay, I'll stick with 'contacts' then ;)
Admins are always going to be able to view custom groups, aren't they...? Would probably be the case for contact lists as well... Well, now that I'm thinking about it, it might be a good idea to hide these even from the admin if they're marked as private. I mean it's not like people can break a website this way...
Hmm, speaking of renaming...
"Send topic". This always bothered me. Not because I would never use that feature (I still wouldn't), but because it doesn't seem natural to me, in the age of Facebrood crap.
Why not rename it to "Share"...? This way, we could 'hook' other items into it -- share by e-mail/Facedoom/Twatter/Redditboughtthetshirt...
Heck, the French translation already says "Partager" (Share.)
PS: before anyone says so... I doubt very much that people would confuse it with the "Split topic" feature. :lol:
-
Bump.
Also... I'm currently trying to determine what's best for implementing {query_see_topic}.
I'm thinking of a relatively different way, compared to what I did with Noisen.
Instead of looking for {db_prefix}topics and adding {query_see_topic} to these queries, I could instead look for any queries that have a "t.approved" test in them... (or just 'approved', just need to make sure it's for a topic)
Because these queries are seen as "protected", I should be able to replace these tests with query_see_topic. And of course, I'll be moving the t.approved test to the query_see_topic string... Which totally makes sense.
Only issue is, I'm not *sure* all of our added queries correctly follow t.approved and such... What do you think? Is this the right approach? Or should I just follow the query list from my Noisen patch, considering that it's been 100% proven as working?
-
Yes, t.approved (and m.approved, ma.approved and all the other aliases that get used on the tables in those queries) are 'protected' but they're protected in a funky way, i.e. based on owner and a permission. I'd argue that you need to leave that component alone, as well as adding the see_topic component to said queries.
Therein lies the problem: can, or should, a topic whose privacy is not explicitly 'everyone' be able to have unapproved posts?
Any queries we've added to the code should adhere to t.approved where it's currently been used, I can't think of a single query we might have added where it wasn't respected, with the exception of anything added in the admin panel where the rules are different anyway.
I'd also argue you pretty much do have to go by what's there and not necessarily by what's in the Noisen patch because there are going to be differences now that weren't in the original SMF code, which does mean looking through approved - there are, unfortunately, quite a few of those.
-
Yes, t.approved (and m.approved, ma.approved and all the other aliases that get used on the tables in those queries) are 'protected' but they're protected in a funky way, i.e. based on owner and a permission. I'd argue that you need to leave that component alone, as well as adding the see_topic component to said queries.
e.g. if I'm putting the condition at the same level as the is_approved, which is probably what 'should' be, then why not simply have the approved test inside the query_see_topic...? I really can't think of a reason not to, right now. If someone has the right to modify/delete/whatever a topic because of a permission, then surely they should be allowed to view it if it's not technically a topic they should be able to read... It's a moderator's job. No?Therein lies the problem: can, or should, a topic whose privacy is not explicitly 'everyone' be able to have unapproved posts?
Any queries we've added to the code should adhere to t.approved where it's currently been used, I can't think of a single query we might have added where it wasn't respected, with the exception of anything added in the admin panel where the rules are different anyway.
I'd also argue you pretty much do have to go by what's there and not necessarily by what's in the Noisen patch because there are going to be differences now that weren't in the original SMF code, which does mean looking through approved - there are, unfortunately, quite a few of those.
BTW, if no one comments on the logo topic any longer... Does that mean I get to choose our final logo? :whistle:
-
e.g. if I'm putting the condition at the same level as the is_approved, which is probably what 'should' be, then why not simply have the approved test inside the query_see_topic...?
That's the thing about permissions... being able to split, merge, delete, whatever, they're all dependent on being able to see the topic. If they can't see it, they can't see it and no other granting of permissions will change that small detail, not really should it.I have no idea... I'd tend to say yes, otherwise it gets complicated.
There are only a few queries about topics in the admin anyway.
I'm a bit wary of removing the patch code without checking the implementation...
BTW, if no one comments on the logo topic any longer... Does that mean I get to choose our final logo?
-
I'm starting to wonder if I shouldn't just give up on topic privacy... :(
-
It's a worthy cause and it looks like it would be quite popular, so I wouldn't give up on it, but I get the impression it does need some serious planning to make work.
-
Nobody uses this feature on noisen actually. Neither does anyone here. I'm just... Convinced it's a great feature.
But I'd like it to be as simple as possible. Plus I need to add contact list support so there's always gonna be a performance issue.
Shit.
-
You could make it known that such a feature exists, divert it to a plugin, or postpone it for 1.1.
-
The problem with baking something like privacy into a plugin is that other plugins may or may not use it properly.
-
Well, I'll try working on this today... -_-
I've decided that I'll try and put the approved test into query_see_topic.
If there's a permission involved in the original query, then I'll add an additional approved test. Is that all right with you?
-
I'm not quite sure what you mean by additional test... if (empty($settings['postmod_rules'])), approved shouldn't be tested at all. Otherwise, the test needs to be approved = 1 || (approved = 0 && (t.id_member_started = $user_info['id'] || allowedTo('approve_posts')))
-
Instead of (I tried to find the query you mentioned...)
WHERE t.id_board = {int:current_board}' . (!$settings['postmod_active'] || allowedTo('approve_posts') ? '' : '
AND (t.approved = 1 OR (t.id_member_started != 0 AND t.id_member_started = {int:current_member}))') . '
AND ' . $sort . ' ' . $sort_methods[$sort_by]['cmp'] . '
We could have that:
WHERE {query_see_topic} AND t.id_board = {int:current_board}' . (!$settings['postmod_active'] || allowedTo('approve_posts') ? '' : '
AND (t.approved = 1 OR (t.id_member_started != 0 AND t.id_member_started = {int:current_member}))') . '
AND ' . $sort . ' ' . $sort_methods[$sort_by]['cmp'] . '
See..? Does it make sense? It doesn't save any time in the query if it has to execute the approved test, but at least it'd work...
Posted: February 17th, 2012, 07:17 PM
BTW, what did we decide already, regarding the position for {query_see_topic}...? As an INNER insertion, or a subselect in the WHERE?
Posted: February 17th, 2012, 07:18 PM
Bump..?
-
I didn't list a specific query, more the fact that those are the rules for topic visibility.
We didn't decide anything specific re where qst should go, though from a convenience perspective, being just a WHERE condition would be pretty handy, especially since being a subselect it wouldn't interfere with the criteria of the parent query so you wouldn't have to ensure t was aliased or anything.
-
I was working on this in the short time I was away... Managed to make it work as a WHERE argument.
Heck, it's really very easy at this point...
admin? WHERE 1=1
guest? WHERE t.approved=1 AND t.privacy="default"
member? WHERE (t.id_member_started=myself OR (t.approved=1 AND (t.privacy="default" OR other privacy settings...))
My main breakthrough is the t.id_member_started test... It's totally upsetting that I never optimized the query into testing for that FIRST... :lol: I used to do it differently, nice waste of time. Then again, at the time I was on a SQL server that didn't support subselects... Which I need for my 'contacts' privacy test.
As for the t alias... I'm afraid there's nothing that can be done. I can't just test for "privacy" because the query might be mixed in with a table that has a privacy field in it... Unless I rename the field, but, well... I'd also have to do that for the 'approved' field in topics because I'm definitely inserting it into query_see_topic. Really, not sure about the whole renaming thing.
Posted: February 18th, 2012, 01:28 AM
A bit of an issue maybe...
Didn't really think about this -- what if the user has moderation perms on the board that a private topic is on...? It should be visible to them, shouldn't it...?
Maybe this could be tested though $user_info['mod_cache']['mq']... But the problem I mentioned, is that it expects the query to have a "boards AS b" in it... :-/
I don't know how to deal with this particular case.
-
Not if it's a subquery, i.e. WHERE (SELECT...)
what if the user has moderation perms on the board that a private topic is on...? It should be visible to them, shouldn't it...?
-
The only subquery is when contacts are being looked through. Otherwise it's as easy as usual.
Posted: February 18th, 2012, 09:57 AM
Crap... So, I've got this query in SSI.php (always...)
WHERE {query_see_topic}
AND att.attachment_type = 0' . ($attachments_boards === array(0) ? '' : '
AND m.id_board IN ({array_int:boards_can_see})') . (!empty($attachment_ext) ? '
AND att.fileext IN ({array_string:attachment_ext})' : '') .
(!$settings['postmod_active'] || allowedTo('approve_posts') ? '' : '
AND t.approved = 1
AND m.approved = 1') . '
After some consideration, it doesn't really make sense to call query_see_topic at the beginning. Because if I'm not the author, then it'll return false, even before evaluating the allowedTo('approved_posts') that would automatically give me a positive and thus skip the approved test...
So I would logically put that instead of the t.approved test, but not in a 'higher' level place.
That makes sense, right?
Now... What about approved_posts? Is this a forum-wide permission only? If yes, then it WOULD make sense to have it in our query_see_topic test, wouldn't it...? Plus, I've seen plenty of places where SMF just tests for postmod_active, and not for that permission... That would allow us to 'clean up' that code.
Any opinions?
Posted: February 18th, 2012, 12:49 PM
Seriously everyone... Just have a look at ssi_recentPosts().
It doesn't test for allowedTo('approve_posts') before it runs m.approved = 1... Does that make sense? AFAIK, SSI calls can and should support permissions...
Posted: February 18th, 2012, 03:06 PM
Bump.
-
approve_posts is a board level permission, not a global one and in SSI, it isn't loaded. Nor is it loaded for search, recent, unread or indeed anywhere that doesn't explicitly contain topic= or board= in the URL string, so it will always return false, with the exception of true for administrators due to the hard-wiring of always-return-true-for-group-1-users.
Does SSI need to support post moderation? That's a good question. My gut instinct says that for the purpose for which post moderation is normally applied and where SSI is used, it's actually not necessary to worry about and that it would actually likely confuse users rather than reassure them.
The original SSI code does this - excludes unmoderated posts even if the user is otherwise empowered to see them, so in those cases I wouldn't worry about testing for approve_posts or is-owner, I'd simply validate that the post is approved or not (and subject to privacy anyway)
-
approve_posts is a board level permission, not a global one and in SSI, it isn't loaded. Nor is it loaded for search, recent, unread or indeed anywhere that doesn't explicitly contain topic= or board= in the URL string, so it will always return false, with the exception of true for administrators due to the hard-wiring of always-return-true-for-group-1-users.
I'd suggest:
- doing {query_see_topic} anywhere in the WHERE when no particular params are given,
- doing {query_see_topic} instead of t.approved = {int:is_approved} anywhere t.approved shows up in a query.Does SSI need to support post moderation? That's a good question. My gut instinct says that for the purpose for which post moderation is normally applied and where SSI is used, it's actually not necessary to worry about and that it would actually likely confuse users rather than reassure them.
I do know, though, that global moderators (group ID 3 or something?) probably don't get the benefits of viewing unapproved posts... That should be changed, too...?
Oh, I'm so rusty when it comes to permissions. Heck, I was more knowledgeable about them back when I did my first query_see_topic implementation...
-
I'd suggest:
- doing {query_see_topic} anywhere in the WHERE when no particular params are given,
- doing {query_see_topic} instead of t.approved = {int:is_approved} anywhere t.approved shows up in a query.
I do know, though, that global moderators (group ID 3 or something?) probably don't get the benefits of viewing unapproved posts... That should be changed, too...?
To change it so that global moderators (or any non-admin, really) can view unapproved posts, means you have to make multiple other queries to identify who can actually approve posts, with boardsAllowedTo('approve_posts') and then you can do the comparison which strikes me as a headache, especially given that I don't think it's actually that useful an exercise.
-
Maybe we could cache in the member table a list of boards they can moderate? Serialized in data...
-
That's great until you change it, because then it's whatever queries you have to update the board information, then a query for every single user that it directly affects. To take sm.org as an example that means you'd have to issue ~50 queries.
-
So. Just give up...?
-
No... just requires more effort in solving the problem. I don't have any answers for this right now :(
-
I mean, give up on board moderators, especially in SSI, and simply focus on replacing t.approved=1 with {query_see_topic} everywhere it shows up..? That would seem like the 'safe' solution, at this point...
-
But that's the thing... is it the safe solution? It's fine in SSI, sure, but what about recent, unread, unreadreplies... places where you would likely need to be made aware of unapproved posts...?
-
I don't know. If I don't get started it'll never get done.
Posted: February 20th, 2012, 07:23 AM
And, really, Noisen.com has been running fine for 4 years with these privacy settings. I could simply get them straight from Noisen, but I just wanted to be 'cleaner' and move t.approved to query_see_topic... It's really the only difference. I don't think it'll hurt in any way.
You already know that, Pete, but sometimes you tend to overthink it ;)
-
Yes, but how often do you use post moderation? How often are there board (or global) moderators who can approve posts? How often are these things viewed outside of boards?
Yes, I also tend to overthink things but for something like this, there is a part of me that feels I have to because it can have all sorts of nasty consequences.
-
Yes, but how often do you use post moderation?
How often are there board (or global) moderators who can approve posts?
How often are these things viewed outside of boards?
Yes, I also tend to overthink things but for something like this, there is a part of me that feels I have to because it can have all sorts of nasty consequences.
-
There's the rub. Post moderation is not that common on Noisen. Meaning that you may or may not have noticed whether there were any glitches attached to it with topic privacy, e.g. moderated items being visible or not visible.That's the point, they may or may not be able to. Generally, yes, they will, but it's possible to construct a setup where a moderator can moderate but not view unapproved posts - since you can set up per-board profiles, you can do it on a per-group-per-board basis if you wish.
I don't know, but I know that Wedge.org uses SSI to load the latest post list... And for that reason, I'd like for my moderators to be able to see unapproved posts from the list...
In any case, here's the problem. How do you know, when viewing the front page, who is a moderator and who isn't? approve_posts won't be loaded at all. The only way to determine for certain is to run boardsAllowedTo() and then query filtering on those.
-
No i dont need ssi but it's easier to get the list of topics this way.
Also. Why wouldn't global mods be automatically able to approve posts?
-
I just noticed that SMF/Wedge never actually (easily) disclose the fact that someone is a global mod...
There's $user_info['is_admin'] which is based on in_array(1, $user_info['groups']), but there's no in_array(2, $user_info['groups']) in the entire codebase -- there's an $user_info['is_mod'] setting, but it's only set within a given board, according to local board moderator settings.
To me, it would make a lot of sense to set $user_info['is_mod'] at the same time as 'is_admin', and then do a logical OR on it when we're inside a board to add the board moderators...
What do you think?
-
No i dont need ssi but it's easier to get the list of topics this way.
Also. Why wouldn't global mods be automatically able to approve posts?
The whole point is that global moderator is not a magic group. 1 is a magic group, 3 is a magic group but there is absolutely nothing special about 2, you could even delete it and things should still work normally. That's why we don't do a test for group 2. On this site, for examine, it would magically ignore me as being a global moderator, which means you end up relying on an assumption that just does not fly.
The only way that works is essentially if you neuter the ability to create global mods just as you cannot create another admin group. You can make it mostly the same but not actually the same for real.
-
One of the issues with topic privacy tests being put inside a 'postmod_active' test is exactly that -- topic privacy won't work at all if you didn't enable post moderation. Which didn't occur to me simply because I always enable it...
Ouch, another headache... ;)The whole point is that global moderator is not a magic group. 1 is a magic group, 3 is a magic group but there is absolutely nothing special about 2, you could even delete it and things should still work normally.
To me, if $user_info['is_mod'] is set on board moderators, if that very variable exists, it should mean to be used, regardless of actual permissions given to the group...
Otherwise, I don't really see the point. And it's exactly the same for global moderators. If one is in that particular group, no matter the permissions, they *should* be able to moderate everything on the forum.The only way that works is essentially if you neuter the ability to create global mods just as you cannot create another admin group. You can make it mostly the same but not actually the same for real.
-
Oh, I see... But aren't people in group 3 also linked to a specific set of permissions...?!
To me, if $user_info['is_mod'] is set on board moderators, if that very variable exists, it should mean to be used, regardless of actual permissions given to the group...
What is a moderator? How do you define a moderator? A moderator is someone who has the permission to moderate_forum or moderate_board. Nothing more. You can, quite readily, create a moderator who has actually less powers than a regular user if you so wish, because you can assign permissions to it.
That's one of the underlying problems in SMF, really. You have the admin group, it has special behaviour because it's the admin group. (It has quite a bit of special behaviour, in fact, not just board access shortcuts, permissions overrides but it's also the only group defined internally as protected, though you can create others, and even then I think group 1 is still considered special in ways other groups are not.)
But the global moderator group is not in any way special, except for the fact it exists in a default installation. Now, in SMF 2.0 (certainly prior to RC4), the group was actually very slightly special, in that it had permissions that were granted and in the tables, but that were nominally disabled (the post moderation permissions were still granted, even if post moderation was disabled) so it gave them access to the mod center even when theoretically they shouldn't have had it. But it was only special because of a buggy permissions loader, not because the group is special itself.Otherwise, I don't really see the point. And it's exactly the same for global moderators. If one is in that particular group, no matter the permissions, they *should* be able to moderate everything on the forum.
Create a new group, give it all the same permissions as global moderator, specifically moderate_forum. What's the difference between that new group and the original global moderator? Nothing. There is (or at least, should be) no difference whatsoever between that new group and group 2, except for id and irrelevant details like name.
This is what I mean: you can create new groups that behave identically to the global moderator role, and they will behave identically in every single way. But you cannot create a new admin group or a new board moderator group that will behave identically.
You can create groups that emulate their functionality, in that you can create a new group that has every permission in it, is protected etc - but it will not be the same as the real admin group. You can create a group that only has moderate_board permissions in a certain permissions profile, and only apply that profile to a single board, which is the mechanical equivalent to a board moderator as far as permissions go, but it's not the same as a real board moderator.
Groups 1 and 3 are special groups (just as -1 and 0 are special groups as far as permission handling goes) but there is nothing unique about 2 other than it exists. And you cannot and should not rely on it existing, whereas groups 1 and 3 must exist.
This site does not have a group 2 at all. But everything works nonetheless - because SMF was designed not to rely on group 2's existence, and that's been true for at least the 2.0 series.
(Actually, that's not *strictly* true. There are a few places where such things are used but for the most bizarre reasons, namely:
* the query on ManageBoards.php line 469 or so, which says > board_moderator_id (3) or == global_moderator_id (2), when they could have just done > admin (1) && != board_moderator_id (3) for the same real functionality
* ManageMembergroups.php lines 558 and 994, same reason, as is the if() on line 707, though the code on line 785 is something you'd want to do for other groups, not just group 2 if you perform that operation, and 967-968 allows you to delete or modify the group if it is = 2 or any post count group that isn't the 0-post count group
* ManageMedia.php lines 766 and 1745, same as above
That leaves the credits as the only place I can find that explicitly relies on group 2 being, well, group 2.)
Posted: February 21st, 2012, 12:59 AM
Yup, I'm not arguing that it's a mess. But this is why I'm overthinking things - because we need to figure out how much of it to salvage and how much actually needs rewriting.
-
Hmm... Thanks for the details. This definitely calls for a rewrite. OTOH, if it works... Don't fix it before the rest.
I'll be sticking with the original code and not give a damn about is_mod. :(
BUT, what I'll do:
- definitely will still use t.approved inside the query_see_topic,
- will only show that t.approved on !allowedTo('moderate_forum'), instead of !$user_info['is_admin']. It's probably a tad slower, well really by less than a nanosecond or so, but it's only done once (when query_see_topic is filled), so that's okay...
What I won't do, is check for a way to make my SQL faster. I'm still shitty when it comes to MySQL optimization, and I still remember how people complained that SMF's handling of t.approved made it slower etc etc, even with the table indexes and all... So I'm just going to leave it without an index, and ask for your help on this Pete.
-
Actually, we could even do !allowedTo('moderate_board'), but ONLY if the $board variable is set... Right?
OTOH, if it works... Don't fix it before the rest.
will only show that t.approved on !allowedTo('moderate_forum')
-
Actually, we could even do !allowedTo('moderate_board'), but ONLY if the $board variable is set... Right?
-
That should do, yes. Mind you, you don't really need to validate that you're in a board, moderate_board will only be set if you're in a board anyway and have the permission (as it can't be set otherwise without mangling the DB directly)
-
Oh yeah, right!
-
In which case !allowedTo(array('moderate_forum', 'moderate_board')) should do the job ;)
-
And even better:
!allowedTo(array('moderate_forum', 'moderate_board', 'approve_posts'))
Right? ;)
-
Yup, only needs one of those things and you get false back overall ;)
-
Considering that SSI.php starts with "$user_info['is_mod'] = false;", do you think it'd be safe/okay to replace false with allowedTo('moderate_forum')? It's better than nothing... (Would have to move the stuff to after the loadPermissions() call though, I guess.)
Or... just not bother?
Posted: February 22nd, 2012, 12:07 AM
I'm doing this in my setup.
$user_info['can_skip_approval'] = empty($settings['postmod_active']) || allowedTo(array('moderate_forum', 'moderate_board', 'approve_posts'));
Should be enough to get things done, including in SSI.php...
For instance, the ssi attachment code:
WHERE att.attachment_type = 0' . ($attachments_boards === array(0) ? '' : '
AND m.id_board IN ({array_int:boards_can_see})') . (!empty($attachment_ext) ? '
AND att.fileext IN ({array_string:attachment_ext})' : '') .
(!$settings['postmod_active'] || allowedTo('approve_posts') ? '' : '
AND t.approved = {int:is_approved}
AND m.approved = {int:is_approved}') . '
My version:
WHERE {query_see_topic}
AND att.attachment_type = 0' . ($attachments_boards === array(0) ? '' : '
AND m.id_board IN ({array_int:boards_can_see})') . (!empty($attachment_ext) ? '
AND att.fileext IN ({array_string:attachment_ext})' : '') .
(empty($user_info['can_skip_approval']) ? '
AND m.approved = 1' : '') . '
Please tell me if you find anything that's logically wrong.
-
Considering that SSI.php starts with "$user_info['is_mod'] = false;", do you think it'd be safe/okay to replace false with allowedTo('moderate_forum')? It's better than nothing... (Would have to move the stuff to after the loadPermissions() call though, I guess.)
Or... just not bother?
Please tell me if you find anything that's logically wrong.
-
- Actually, given how little 'is_mod' is used throughout Wedge/SMF, I'm thinking it might actually be smarter to entirely remove it, if it's gonna bring only confusion... Or update it everywhere it shows up, to make it more useful.
- Eheh.
- Okay, a funny one... You said it'd be best to call boardsAllowedTo for full rights etc... And that it wasn't free so we couldn't really use it in SSI... Well, we can, using the mod cache. Look at Subs-Auth.php:
$_SESSION['mc'] = array(
'time' => time(),
// This looks a bit funny but protects against the login redirect.
'id' => $user_info['id'] && $user_info['name'] ? $user_info['id'] : 0,
// If you change the format of 'gq' and/or 'bq' make sure to adjust 'can_mod' in Load.php.
'gq' => $group_query,
'bq' => $board_query,
'ap' => boardsAllowedTo('approve_posts'),
'mb' => $boards_mod,
'mq' => $mod_query,
);
$user_info['mod_cache'] = $_SESSION['mc'];
The 'ap' is what we wanted. I already used the mod cache before -- just that it's so confusing, most people (including me) wouldn't think of it first!
-
Cripes, even I didn't know about that.
-
I didn't remember. Then I found out that my noisen patch used it in an earlier version of query see topic. Look into it ;)
I think it warrants some reworking of the system...
-
I will look into it, need to go out for a bit first.
-
The #1 issue with all this is that there are too many things to go through if you don't know better. (Heck, even if you know better...)
- Have I got approve_posts permission on this board? (allowedTo('approve_posts'), $user_info['mod_cache']['ap'])
- Have I got moderate_board permission on this board? (allowedTo('moderate_board'), $user_info['mod_cache']['bq'])
- Am I a moderator on this board? ($user_info['is_mod'], $user_info['can_mod'], $user_info['mod_cache']['bm'])
And so on. At this point, it becomes really unclear what 'can_mod' is for, why 'is_mod' is reset in SSI when really it could be using the mod_cache, etc...
There's a strong, huge feeling that SMF has been overhauled many times when it comes to permissions, and once again different developers had their own idea of how to make it 'simpler', and failed to make them commonly used. That mod_cache thing is saddening, because it's exactly what's needed (a session cache of some expensive queries), but also it doesn't save everything it could ($boards meaning the list of boards you have moderate_board on, is not in it, even though it's calculated), it's not clear what the difference between being a board moderator and having moderate_board permission is, and finally I'd tend to say that using acronyms for permission names is a bit silly. mod_cache is only accessed exactly 30 times in the entire codebase (plus the 3 times it's set up), so it's not like it's so complicated to type $user_info['mod_cache']['board_query'] instead of $user_info['mod_cache']['bq']... Plus, it makes the thing easier to understand.
Have you got some thoughts to share on all of this, Pete..?
-
Have you got some thoughts to share on all of this, Pete..?
The #1 issue with all this is that there are too many things to go through if you don't know better.
For the most part, you only need to be bothered with standard permissions checks (allowedTo style) because it's surprisingly infrequently that you worry about board permissions outside of a board context. Even in SimpleDesk where I had the same situation to encounter, even there it actually was not a problem in the same fashion (and there was no such thing as general vs 'board' permissions, everything was implied to be a board permission of sorts)
But in SD's case, I didn't have separate logical structures to contain things - when you loaded permissions, you did so in a single hit, and loaded access to every board's permissions in a single hit, meaning you always knew which departments you had what permissions for.Am I a moderator on this board?
At this point, it becomes really unclear what 'can_mod' is for, why 'is_mod' is reset in SSI
There's a strong, huge feeling that SMF has been overhauled many times when it comes to permissions, and once again different developers had their own idea of how to make it 'simpler', and failed to make them commonly used.
* profiles
* post approval
* the mod_cache
(The whole simple vs classic thing is purely cosmetic. It's also stupid, but that's a debate for another day.)
The whole profiles thing is interesting. I actually remember an argument I had with Antechinus about that over reverting to 1.1.x local board permissions vs global ones, but honestly that's about as confusing! The reason for the argument was because of an important lack of understanding - there was a *reason* profiles were adopted instead of local/global board permissions, and that was to simplify applying the same permissions to multiple boards at once. It did, incidentally, fix a few bugs in 1.1.x's 'profiles' as they stood, too.
Fortunately for the most part, the profiles mechanism doesn't really make that much difference for us in terms of mechanics, because whatever we do, it will probably use some of the same infrastructure.
Then there's post approval. The whole permission implementation of that had some convenience-for-UI-building factors (and still does, in ways I have not yet solved for moderation filters) but it causes the problem we've been bouncing around here - that you need to have some conceptualisation of board-level permissions being applied to a context outside of boards.
Prior to post approval's introduction, there was never a context in SMF where you'd ever need to understand board level permissions properly in places that have a meta board context - yes, search/unread/unreadreplies/recent had *some* idea of board level permissions, for whether you could reply to a post or edit it or whatever, but all that was doable after the event, as it were - you only had to worry about board *access* to see posts, not a permission attached to whether you could see an individual post.
This brings us to the mod cache - whose sole existence, if I'm any judge, is to mitigate the very performance issues that I'm getting at. Now, here's the thing: the reason it's not used that much in SMF is simply because there are that few places where having board-level permissions outside of a board just doesn't make much difference. Primarily it supports post approval - but the rest are nice convenience features, and that allowedTo() is probably more important.
So, where does that leave us? It leaves us with a permissions system that I've wanted to overhaul for a while - because SMF's permissions system is unintuitive. It's very, very powerful but the price paid is the complexity of actually trying to use it - and almost everyone I know (including me) has misused it and used it incorrectly to do things that shouldn't be done that way.
For example, if you have two or three 'teams' that are all, practically speaking, global moderators but have different badges, the correct thing to do is make new groups for each team that all inherit from a parent group so you manage the permissions centrally - but most don't.
The really sad part is, I don't have an answer yet. I've had bits of answers floating around and my gut instinct is to dump SMF's permissions core and merge in SimpleDesk's in some respects but that's not really where I want to go with this.
What is clear is that SMF's permissions are broken, and a lot of that is down to perception: as I identified when figuring out SD's permissions, a lot of the problems in SMF are really caused by people conflating groups and roles, and building on that incorrect premise. It's almost like we need to separate groups from permissions properly.
-
This is a special case by definition because you can be a moderator on a board without being a board moderator (and without being a global moderator) It is this level of complexity that actually makes SMF so difficult to configure, even if conceptually/codewise using the permissions system is made so easy.
can_mod, supposedly, is 'I can moderate somewhere' while is_mod is 'I can moderate *here*'. Primarily IIRC can_mod was supposed to be used to control access to the moderation centre but this changed in RC4 when fixing a related bug.
can_mod is "do I have access to the moderation center, or am I allowed to moderate at least one membergroup or board, or approve posts to some extend?"
That's a hell of a vague setting to me...
And the ONLY place it's used (although it's repeated a couple of times), is whether the user can see the Warning button on a userbox...
That smells like "REMOVE!" to me...there was a *reason* profiles were adopted instead of local/global board permissions, and that was to simplify applying the same permissions to multiple boards at once.
For instance - just an idea - one could simply 'hide' them profiles at first. You set up permissions for a board. Then when you create or edit another board, you get the ability to apply another board's permissions. At which point, its permission profile is used. Otherwise, a new permission profile is created for the new board. Only problem is the 'default' name for these profiles.This brings us to the mod cache - whose sole existence, if I'm any judge, is to mitigate the very performance issues that I'm getting at.
Hey Aaron, this one's for you... There's Doctor Who on the telly (France 14 channel), dual language, I switched to English of course, and just a minute ago the Doctor said his line from your signature, "after all this time..." It made me smile :)For example, if you have two or three 'teams' that are all, practically speaking, global moderators but have different badges, the correct thing to do is make new groups for each team that all inherit from a parent group so you manage the permissions centrally - but most don't.
The really sad part is, I don't have an answer yet. I've had bits of answers floating around and my gut instinct is to dump SMF's permissions core and merge in SimpleDesk's in some respects but that's not really where I want to go with this.
-
Back to topic, too...
This is defined in my local copy in the topics table:
privacy enum('default', 'members', 'groups', 'contacts', 'author') NOT NULL DEFAULT 'default',
(Also, a TEXT field named 'tags', which will most likely be useful in the future.)
Okay, what I want to say... 'author' is 'justme' (I just renamed it), 'contacts' is 'friends'.
Now, what about 'groups'. It's the same issue as with thought privacy: should we allow a specific group to read the topic/thought? I thought, yeah why not, just get a list of the user's current groups (or all groups for an admin or moderator), and list them as choices. This is most interesting for Wedge.org's thoughts, because all earlier thought replies are set for membergroup=18 (friends) only. Oh, probably also the wedgeward and consultant groups. Anyway... So, we CAN get rid of it for Wedge, the main argument being that this is opening the door to increased complexity.
Now, the thing is, I certainly want people to be able to select a *contact list* for thoughts and topics... i.e. not available to my colleagues, but available to my family. Which pretty much forces me to introduce a privacy_id field or something. So, might as well use that to store group IDs... I don't know.
So, the questions would be:
- leave 'groups' in as a possibility? Can always be removed later...
- add a 'privacy_id' field?
- best solution for performance? What should the extra index(es) look like?
- am I chasing a mirage, or just plain crazy?
-
I'm just wondering what board moderators are allowed to do that is, or is not included in the moderate_board permission...
can_mod is "do I have access to the moderation center[1], or am I allowed to moderate at least one membergroup or board, or approve posts to some extend?"
Instead of the other way around... In that respect, I don't know why Load.php has $context['allow_moderation_center'] = $context['user']['can_mod']; at some point when it should be using the permission, directly...
And the ONLY place it's used (although it's repeated a couple of times), is whether the user can see the Warning button on a userbox...
That smells like "REMOVE!" to me...
The UI could be better, I think.
For instance - just an idea - one could simply 'hide' them profiles at first. You set up permissions for a board. Then when you create or edit another board, you get the ability to apply another board's permissions. At which point, its permission profile is used. Otherwise, a new permission profile is created for the new board. Only problem is the 'default' name for these profiles.
I love how you are quick to adapt! Earlier today you said you'd never heard about it.
There's another thing with inherited board permissions. We absolutely need to document this in a Help icon or something. Have something right next to the option list, saying what they'll do. I always, ALWAYS get confused between copied permissions and inherited permissions (i.e. nothing is copied, just using the original's on the fly). Perhaps we ought to rename the terminology. "Copy permissions" and "Mirror permissions", maybe...
The other thing with the mirrored permissions is that most mods for SMF never used them properly - not even SimpleDesk got it right, and I don't believe Aeva did either. (Case in point, I reordered all the groups on my one site to have a bunch of groups, one per user, for unique badges/titles because I'm like that. Each of these new groups was based on a higher group that controlled permissions - but neither SD nor Aeva responds to the child groups properly and I had to end up assigning the permissions in both to all the groups, not just the parent group.)We'll probably have to leave it at that for the first versions. There's no time to deal with that, and the SMF importer will get the original permission profiles anyway, so people are likely to have already solved their permission problems already... (It's for new users that it gets annoying.)
leave 'groups' in as a possibility? Can always be removed later...
- add a 'privacy_id' field?
- best solution for performance? What should the extra index(es) look like?
Putting it in the table as a textual id with a list of groups means the query becomes WHERE t.privacy = something OR (t.privacy = 'groups' AND FIND_IN_SET(...)) which is expensive enough.
Meanwhile, putting it in a separate table means either a subquery, or a join, both of which being applied to potentially every topic in a board (or worse) to validate access is going to get very unfriendly very quickly.
I don't think you're crazy, but I also don't think there's a good way to achieve this meaningfully for topics.
Regarding applying it to thoughts, I'd actually be inclined to handle that in permissions (permission to post, permission to read, permission to read replies, permission to reply, etc.)
-
They get their name on the linktree, on the board index and their badge is altered on the fly.
They're also, silently, added to group 3 when in that board, which means you can do permissions checking in that respect.
Don't they automatically get all post/topic-related permissions, more simply?Again that's the historical aspect. It was never supposed to be entirely based on the permission but showing you the area if you could do at least one thing there - much as you get access to the admin tab if you can do any of the admin like features (e.g. news, manage permissions, manage membergroups etc.)
And the ONLY place it's used (although it's repeated a couple of times), is whether the user can see the Warning button on a userbox...
That smells like "REMOVE!" to me...
Heck.
allowedTo('issue_warning').
This must be some kind of a joke... I mean, moderators get the 'right' to see the Warning button, but then if they try to click it (from what I gather), they'll get kicked out because area=issuewarning only checks for issue_warning perms to be set...
And AFAIK, this kind of permission isn't given automatically to moderators.Which is virtually how 1.1.x did it, except it still created proper per-board permissions rather than having discrete profiles.
I wasn't very clear on that point. I knew of the mod cache subsystem, but I wasn't aware of all the details and precisely what it covered - namely the ap element.
But combine the places it is used, with the knowledge of what it actually contains and it becomes rather clear (to me at least) what it was intended for. It's like having 3/4 of the pieces of the puzzle and then suddenly you find that last piece and it all suddenly makes sense.
Tell you what. How about you look into it a bit? Now that my work is committed, you may want to simply deal with it by yourself, because nobody will deny you the fact that you're one of the most knowledgeable SMF/Wedge devs when it comes to permissions. And optimization.Oh, it's horribly confusing, which is why I didn't even implement it in SimpleDesk but left it simply as the ability to copy a role, rather than to mirror it.
So, do you think we should rename..?The other thing with the mirrored permissions is that most mods for SMF never used them properly - not even SimpleDesk got it right, and I don't believe Aeva did either.
(Case in point, I reordered all the groups on my one site to have a bunch of groups, one per user, for unique badges/titles because I'm like that. Each of these new groups was based on a higher group that controlled permissions - but neither SD nor Aeva responds to the child groups properly and I had to end up assigning the permissions in both to all the groups, not just the parent group.)
Dragooon, are you reading this?I like the idea but I'm concerned it would potentially lead to greater complexity.
I'm also not sure you can actually optimise it *that* far, though, and there's a lot of places that use ?topic in the URL to handle permission loading.
Storing it in the topics table is going to hurt. Storing it outside the topics table is going to hurt too, just in a different way, and neither is lightly ignored.
Putting it in the table as a textual id with a list of groups means the query becomes WHERE t.privacy = something OR (t.privacy = 'groups' AND FIND_IN_SET(...)) which is expensive enough.
When it comes to contact lists, would be a sub-select. (Pretty much like what UltimateProfile does...)Meanwhile, putting it in a separate table means either a subquery, or a join, both of which being applied to potentially every topic in a board (or worse) to validate access is going to get very unfriendly very quickly.
Now, of course, this performance issue would only happen for members (guests and admins are easily discarded), and only for topics/thoughts where a non-default choice is set... (And when it comes to topics, it's always fast enough to scan that table innit?)I don't think you're crazy, but I also don't think there's a good way to achieve this meaningfully for topics.
Regarding applying it to thoughts, I'd actually be inclined to handle that in permissions (permission to post, permission to read, permission to read replies, permission to reply, etc.)
Oh, another possibility...
topic has privacy set to contact list = 42.
user is in contact lists 41 and 42.
user wants to view. Test is done on "privacy=contacts_41 OR privacy=contacts_42". Well, obviously that would make a VERY long query if user was in thousands of contact lists, but then we're pretty sure it's only ONE field that's indexed... Even if it's a string field by now.
:^^;:
-
Which I don't like, BTW... On wedge.org/new, we no longer have our Developer badges, because we're moderators. It sucks.
What permission checking? You mean do in_array(3, $user_info['groups'])?
Don't they automatically get all post/topic-related permissions, more simply?
Sure, but the admin area (which was never just only about administrators) isn't like the mod center, which was designed specifically to allow people to deal with whatever sub-par permissions they had, without giving them access to the admin panel.
This must be some kind of a joke... I mean, moderators get the 'right' to see the Warning button, but then if they try to click it (from what I gather), they'll get kicked out because area=issuewarning only checks for issue_warning perms to be set...
And AFAIK, this kind of permission isn't given automatically to moderators.
Yeah, and it's just simpler in appearance, but still the same under the hood...
Complexity internally? I don't think so. In the UI? Definitely.
So, what about it then...? Shall we use it in query_see_topic?
Tell you what. How about you look into it a bit? Now that my work is committed, you may want to simply deal with it by yourself, because nobody will deny you the fact that you're one of the most knowledgeable SMF/Wedge devs when it comes to permissions. And optimization.
Also, I thought you were still working on it at present?Mirroring is still a great thing. I mean, it allows you to give someone a different membergroup name, even if it's the same group in the end.
So, do you think we should rename..?
I'm not sure what this implies.
Although I could imagine people wanting to send to multiple contact lists, which sort of defeats the concept of having a select box for privacy...
When it comes to contact lists, would be a sub-select. (Pretty much like what UltimateProfile does...)
Now, of course, this performance issue would only happen for members (guests and admins are easily discarded), and only for topics/thoughts where a non-default choice is set... (And when it comes to topics, it's always fast enough to scan that table innit?)
SMF team already complained enough about the performance killer that is m.approved... Heck, it's just a simple flag...!
We'd actually get more benefit from keeping an accurate track of moderated posts in a topic and topics with moderated posts in a board - so if there's nothing that has pending moderation status in that board or topic, don't perform that test.Because you don't have to expressly store it in the table then. For example, if you handle 'view replies' as a permission, you can simply exclude massive portions of the table, very cheaply.topic has privacy set to contact list = 42.
user is in contact lists 41 and 42.
user wants to view. Test is done on "privacy=contacts_41 OR privacy=contacts_42". Well, obviously that would make a VERY long query if user was in thousands of contact lists, but then we're pretty sure it's only ONE field that's indexed... Even if it's a string field by now.
-
There is an override in loadMemberData that if you're an admin, the badge won't be overridden by the moderator badge.
They do get all the related permissions automatically,
but on top of that a plugin or whatever can expressly check for being a moderator with that in_array check, should they want to do so (as opposed to checking allowedTo(moderate_board))
Hell yes. The biggest complaint I've had is taking into account approve-posts permission - and there it is, right there, pre-cached and 'dealt with' already.
Can you share your opinion on this? It's untested...
<?php
$user_info['can_skip_approval'] = empty($settings['postmod_active']) || allowedTo(array('moderate_forum', 'moderate_board', 'approve_posts'));
$user_info['query_see_topic'] = '
(
t.id_member_started = ' . $uid . ' OR (' . ($user_info['can_skip_approval'] ? '' : (empty($user_info['mod_cache']['ap']) ? '
t.approved = 1' : '
(t.approved = 1 OR t.id_board IN (' . implode(', ', $user_info['mod_cache']['ap']) . '))') . '
AND') . '
(
t.privacy = \'default\'
OR (t.privacy = \'members\')
OR (
t.privacy = \'contacts\'
AND t.id_member_started != 0
AND FIND_IN_SET(' . $uid . ', (SELECT buddy_list FROM ' . $db_prefix . 'members WHERE id_member = t.id_member_started))
)
)
)
)';
I have a headache -- literally this time. So, hard to focus.Also, I thought you were still working on it at present?
After all, it's not like adding query_see_topic will break anything. It can be done gradually :)I think renaming it would help, but I think overhauling the UI will help even more in the long run - but the rename is good for now.
The implication is that a surprising amount of places simply do not use the underlying functions to confirm access at all. I discovered this last night - Display for example does not itself do {query_see_board} in the topic to validate access to the topic, it relies on loadBoard having already done it.
Point is... make *sure* loadBoard is working correctly as far as access to boards goes, and to topics if a topic is specified, and that has a lot of knock-on effects, not just for display, but anywhere ?topic is in the URL (e.g. attachments, remove topic, just to name two)
See, the notion of having a single group or contact list never actually occurred to me, I assumed right off the bat that multiple lists was the aim.
I just said that if we have to store an extra integer for the contact list ID, then we might as well enable Wedge to store a group ID in that same place...I'm not nearly familiar enough with what UP offers, perhaps I should learn!
Right now, it's 'relatively' cheap. It's still fixed-length rows, but the minute a text field is added to it to contend with the whole aspect of multiple groups attached to it, it's going to get expensive (and the alternatives, like SET columns, aren't really good alternatives).
I understand the nature of their complaint on it - being a binary state field, you can't actually optimise it that heavily in an index. An index works best if it's selective, or at least sufficiently distributed such that picking a value excludes a sizeable percentage of the rest of the table. approved being a two-state field is never going to exclude vast amounts of the tableset so you never get any advantage out of indexing it.
We'd actually get more benefit from keeping an accurate track of moderated posts in a topic and topics with moderated posts in a board - so if there's nothing that has pending moderation status in that board or topic, don't perform that test.
Because you don't have to expressly store it in the table then. For example, if you handle 'view replies' as a permission, you can simply exclude massive portions of the table, very cheaply.
The test is really (privacy = contact AND privacy-list IN (user contact lists)), isn't it? Since the user's contact list will be part of the user's own data and thus available for query insertion?
So it's faster, then? Indexable?
(Sorry, technical issue caused bump :P)
-
I still don't like it. I don't like that you're forced to wear a Moderator badge when you should have the Developer or Boss badge. Teaches people some respect
Are you sure really...? I tried looking for 'issue_warning' for instance, and neither Load.php or Subs.php have it. Only ManagePermissions.php which seems to do some voodoo about it, but what does it mean? That once you're set as a board moderator, these permissions are assigned to you automatically, hard-wired style, visible in the database and all? And that these permissions are removed when your status is removed?
So yes, it's a bug - they have the power to see the warn button but not to actually use it.Confusing, really... More confusing than the allowedTo().
Can you share your opinion on this? It's untested...
After all, it's not like adding query_see_topic will break anything. It can be done gradually
I wouldn't be able to determine what's best, between "mirror from profile..", "mirror profile", "alias of profile..." or whatever makes sense.
Hmm, well... As long as it works...
I'm afraid I still don't get it. But don't bother...
But, for some reason that escapes me, I ended up removing the feature (possibly because it was seen as too complex?), so here are are.
I just said that if we have to store an extra integer for the contact list ID, then we might as well enable Wedge to store a group ID in that same place...
Let's just say that UP has three good things
I see... I imagined that varchar fields only made the index bigger, not slower, than an integer field.
Here's the deal: if all the rows are fixed length (numeric, or specifically char fields), the table is faster - because it can use the index to identify a row and get there very quickly, as it doesn't have to go row by row to find it, it can just say 'row abc x bytes-per-row' and get to its position. (Since the position is not recorded in the index, or at least it never used to be)
But with variable length fields, it can't pre-calculate the position and has to go through the table to find them. While there are ways to speed that up, it's still not as fast as fixed-width rows.
Text/blob fields are special. As far as fixed/variable width rows are concerned, they're actually fixed, because they only contain a reference within a secondary data area where the content is located. That's how, for example, you transcend certain limits (a row in MyISAM at least can only be 64K in total, and a text field counts two bytes towards that, but only two bytes regardless of its actual length) - and that's not a performance issue in of itself when dealing with tables with text/blob fields.
The problem comes in when you query a table with such records, and actually retrieve the text/blob. In those situations, if you do any ordering, it triggers a filesort rather than an in-memory table.Hmm. So the same problem awaits us. A vast majority of topics and thoughts are gonna be public, so granularity or whatever will not be to our advantage...
Err, yes? I suppose so...
So it's faster, then? Indexable?
-
Before I get to this long post which I haven't had time to read yet -- I noticed that many m.approved calls in Wedge do not test for postmod_active before, unlike ALL of the t.approved calls. I'm thinking this *might* be a problem...?
-
Actually, it's only a problem since I messed with it. In the old code, disabling post moderation would actually silently force everything unapproved to become approved. Now it's a discrete option you have to use should you want to do so - but the odds of that happening are actually surprisingly slim (since it requires removing every moderation filter not just ones that trigger moderation)
-
Hmm... Are you implying that, if m.approved is always 0 (i.e. post mod never enabled), the performance penalty is actually 0, too? In which case, yes, I suppose it's fine to leave it in, but then maybe it should have been the case for t.approved as well...
-
Sadly, no. If it never checks for being enabled or not, but always checks anyway, there's a performance hit attached. It's just not one you can avoid by turning the feature off...
That's really more of an SMF issue; in their case, post moderation is a very specific feature, disabled by default, so you get penalised by default whether you use it or not.
In our case, I'm really expecting users to use it more than SMF users make use of post moderation, meaning that while you're still getting penalised for that query overhead, at least it's done mindfully of the fact that moderation is likely to be in place.
-
Sadly, no. If it never checks for being enabled or not, but always checks anyway, there's a performance hit attached. It's just not one you can avoid by turning the feature off...
That's really more of an SMF issue; in their case, post moderation is a very specific feature, disabled by default, so you get penalised by default whether you use it or not.
I'm not exactly sure I remember what changes you made to the approval system, if any...?In our case, I'm really expecting users to use it more than SMF users make use of post moderation,
-
Well, if it's turned off, m.approved should no longer be tested against, should it...?
...Unless you never enable it..?
Is it enabled by default in here?
However, I foresee people using it in all kinds of funky ways, and as such I think it's likely to be 'on' (in SMF terms) more often.
-
Well, if it's turned off, m.approved should no longer be tested against, should it...?
BTW, where the hell do I enable postmod?! Now that core features are gone... Well, I even made an admin search and didn't find any 'master' setting?!
(Okay, see below...)No, you're still penalised either way. If you never check for it at all, you're basically assuming it's enabled whatever - and with the performance issues attached.
No. But in any case, moderation filters is quite a different beast. The way postmod_active works now is that if there are any rules to be applied (of any kind), postmod_active is true. No rules to apply (of any kind) = no post moderation. So there's a performance gain potential if you *never* use that feature.
Is it working, at least? Last time you didn't have the Save feature for it, so I left it aside but now I need to have postmod enabled just to test my code... :P
BTW, in Display.php, if you are in a topic that has approved=0 but postmod is disabled, you get an error message saying max() needs at least one entry. That's on line 838... I suppose that means $messages is empty even if I'm an admin and I can see said topic...
Which brings me to... Heck, can anyone actually view an unapproved topic if they don't have permission to, but they know its ID? Because I can't see any code to stop them in Display.php! Actually noisen.com/wedge.org do have such code, that's where it gets funny. Well, they don't add approved flags to the mix but at least they support 'justme' privacy.
I suppose I'll add query_see_topic to the query at line 122... But it's quite scary all of a sudden. I thought I was finished and I only had the UI left to do, but I WILL definitely have to go through every single query_see_topic occurrence in the noisen patch to determine whether it's needed... :^^;:
-
Well, that's what I'm saying: it is... It's tested whatever the postmod_active state.
BTW, where the hell do I enable postmod?! Now that core features are gone... Well, I even made an admin search and didn't find any 'master' setting?!
...I don't get it. If testing for m.approved=0, m.approved=1 or nothing at all, same performance. How can it be, since in the last case we're sure not to ever use the approved index...?
Hmm... That does seem like something that might confuse people. It confuses even me...
Is it working, at least? Last time you didn't have the Save feature for it, so I left it aside but now I need to have postmod enabled just to test my code...
Heck, can anyone actually view an unapproved topic if they don't have permission to, but they know its ID? Because I can't see any code to stop them in Display.php! Actually noisen.com/wedge.org do have such code, that's where it gets funny. Well, they don't add approved flags to the mix but at least they support 'justme' privacy.
I know in SMF if you have a topic whose ID you know but it's moderated, you can only see it if you're the owner or a moderator with suitable permissions, but it's enforced by loadBoard as far as I can tell, not by Display. Display assumes to a point that if you can see the board, you can see the topic.
-
There isn't a master setting any more :P
See, post moderation is not just about filtering new posts... It's also about being able to unapprove posts (something we're still not doing, AFAIK...), which we won't be able to do if this happens. How about adding a master setting to the filter page...? Plus, it would allow admins to disable their filters without having to delete them, if that's not possible right now.
BTW I went ahead and looked at the page. I was a bit confused with the order. For instance, you can change the 'new post' or 'new topic' item after making new rules. I think it'd be more understandable if it was disabled, or hidden and repeated as a simple text string in the next fieldset. Also, the 'posting in these boards' thing may be confusing as well. I know that my direct French translation made me feel unsure about it. I changed it to 'posté dans ces sections', meaning 'postED in these boards', but I'm not sure it's the best.
Finally, it'd be nice to have slideDown animations, I think, instead of simple toggles. Perhaps a fade in for the "Save rule" button, because it's not always clear that the "Save filter" button is being hidden and replaced by another (even though the new one is inside the fieldset rather than outside). If it's going to be a wizard, it should be super-user-friendly, which right now I'm not sure it sure. I'd like to know your opinion on this....I don't get it. If testing for m.approved=0, m.approved=1 or nothing at all, same performance. How can it be, since in the last case we're sure not to ever use the approved index...?
See, this is what I don't get: I'm saying that SMF/Wedge has t.approved tests done *only* if postmod_active is enabled, while m.approved tests are done without testing for postmod_active first. It seems to me that m.approved is slower than t.approved (because of relative table sizes), so it should be the one with systematic postmod tests, not the other way around...Oh, it works. The only thing that doesn't work properly (out of what is shown) is the textual matches, when you're using any of the HTML special characters, they get encoded properly for sending, are stored properly but the regex doesn't apply them properly at this point.
Heck, can anyone actually view an unapproved topic if they don't have permission to, but they know its ID? Because I can't see any code to stop them in Display.php! Actually noisen.com/wedge.org do have such code, that's where it gets funny. Well, they don't add approved flags to the mix but at least they support 'justme' privacy.
I know in SMF if you have a topic whose ID you know but it's moderated, you can only see it if you're the owner or a moderator with suitable permissions, but it's enforced by loadBoard as far as I can tell, not by Display. Display assumes to a point that if you can see the board, you can see the topic.
-
But how can I enable it, at all?
See, post moderation is not just about filtering new posts... It's also about being able to unapprove posts (something we're still not doing, AFAIK...), which we won't be able to do if this happens. How about adding a master setting to the filter page...? Plus, it would allow admins to disable their filters without having to delete them, if that's not possible right now.
As for disabling a filter without deleting it - why would you? Either it's not working for you and needs editing, or it is working and should be left alone.BTW I went ahead and looked at the page. I was a bit confused with the order. For instance, you can change the 'new post' or 'new topic' item after making new rules. I think it'd be more understandable if it was disabled, or hidden and repeated as a simple text string in the next fieldset. Also, the 'posting in these boards' thing may be confusing as well. I know that my direct French translation made me feel unsure about it. I changed it to 'posté dans ces sections', meaning 'postED in these boards', but I'm not sure it's the best.
As for 'posting in these boards', I couldn't think of a better way to phrase it. I suppose past tense is more correct, but the idea is to convey the notion that if the post is made in one of the boards you're picking (or not, depending on situation), that the rule will apply.Finally, it'd be nice to have slideDown animations, I think, instead of simple toggles. Perhaps a fade in for the "Save rule" button, because it's not always clear that the "Save filter" button is being hidden and replaced by another (even though the new one is inside the fieldset rather than outside). If it's going to be a wizard, it should be super-user-friendly, which right now I'm not sure it sure. I'd like to know your opinion on this.
But we always need a WHERE clause, I never met a single case where a post-mod test was being done and the WHERE was added only for it.
SELECT ... WHERE stuff is always going to outperform SELECT ... WHERE stuff AND m.approved... unless (and only unless) the stuff in the first place excludes everything anyway. If there are any rows found by the stuff clauses, they will have to be evaluated by m.approved.See, this is what I don't get: I'm saying that SMF/Wedge has t.approved tests done *only* if postmod_active is enabled, while m.approved tests are done without testing for postmod_active first. It seems to me that m.approved is slower than t.approved (because of relative table sizes), so it should be the one with systematic postmod tests, not the other way around...
// Get each post and poster in this topic.
$request = wesql::query('
SELECT id_msg, id_member, modified_member, approved, poster_time
FROM {db_prefix}messages
WHERE id_topic = {int:current_topic}' . (!$settings['postmod_active'] || allowedTo('approve_posts') ? '' : (!empty($settings['db_mysql_group_by_fix']) ? '' : '
GROUP BY id_msg') . '
HAVING (approved = {int:is_approved}' . ($user_info['is_guest'] ? '' : ' OR id_member = {int:current_member}') . ')') . '
ORDER BY id_msg ' . ($ascending ? '' : 'DESC') . ($context['messages_per_page'] == -1 ? '' : '
LIMIT ' . $start . ', ' . $limit),
array(
'current_member' => $user_info['id'],
'current_topic' => $topic,
'is_approved' => 1,
'blank_id_member' => 0,
)
);
That's in Display.php. That's the query that identifies what posts are to be displayed, everything after that shouldn't be filtering it.Can do if you like... it doesn't help that the rules are non typical because of the whole entity 039 thing, and the fact that the regex raw type should expressly not perform that operation (because you'd end up screwing up <= operators in the regex for example) while the others should perform it.I'd be curious to know if this happens in stock SMF installs. I can't do it for my sites because they all use my custom code, but to anyone with a forum: please enable post moderation, post an unapproved topic, and use a low-level account to access it directly by ID... Does it show you the topic? What about the post itself?
-
Maybe that suggests a rethink of how the moderator badge is applied is in order. Perhaps, even, have it in addition to (rather than instead of) the standard badge?
What do you think?They won't, it's a standard permission. But specifically it is a general permission, not a board one, so board level moderators never get the chance to have it in the first place (because that's the quirk of group 3 - it never has general permissions)
So yes, it's a bug - they have the power to see the warn button but not to actually use it.
Sure it is, but as group 3 is special in other ways you can expressly do things with it that you can't with a bare allowedTo() check.
The only issue I have is with the FIND_IN_SET - but there's no way around it.
After all, it's not like adding query_see_topic will break anything. It can be done gradually
Too bad that the very first occurrence was in Display.php... Which spawned a discussion of its own ;)I wouldn't be able to determine what's best, between "mirror from profile..", "mirror profile", "alias of profile..." or whatever makes sense.
Point is: some queries that look like they need query_see_topic actually don't, because loadBoard should take care of them, or at least if loadBoard takes care of them, other places don't have to as much.
Sorry, hadn't read this post yet, it was in a tab waiting for a proper reply ;)
Well, I suppose we could add something in loadBoard... Check $topic, and if it's filled, add a query_see_topic?I just said that if we have to store an extra integer for the contact list ID, then we might as well enable Wedge to store a group ID in that same place...
Hey, how about we keep a table with proper relationships...?
{db_prefix}topic_privacy
id_privacy = unique ID
id_topic = topic ID
privacy = privacy type?
privacy_id = ID of group or contact or member, depending what's in the privacy type
{db_prefix}topics
privacy = privacy type?
The topic table could check for privacy type, and if != 'default', do a subselect on wedge_topic_privacy using its own topic ID, trying to determine whether we belong to the privacy_id list.
Because topic privacy is unlikely to be used on a majority of topics, it would keep the table quite small. Maybe we could even get rid of any privacy fields in the topic table, and just do a LEFT JOIN but I don't think it's very practical to do, just a silly idea.The problem comes in when you query a table with such records, and actually retrieve the text/blob. In those situations, if you do any ordering, it triggers a filesort rather than an in-memory table.
Posted: February 23rd, 2012, 05:29 PM
I really, REALLY want to know what you drink or eat to be able to answer things so quickly and so thoroughly.
I just can't keep up :PYup. But in the case of thoughts at least, depending on permissions checks (which are basically free in DB terms), you can exclude things on a more granular nature, rather than on a permissions flag. Consider, if you do 'permission to view thoughts' and 'permission to view replies', the former obviates any need to query at all,
Call it a dream or folly, but I want people to be able to think freely! :P :POh... I just realised I'm a doofus (based on your earlier post), because I mistakenly thought it was the current user's friend list which is already loaded and available, rather than the topic starter's (which can't be done without some kind of lookup)
-
I don't know. It screams "admin setting" to me, because some admins will want two badges, others will want to show moderator badges, and others like me will want to show regular badges if available...
Hmm yes, that's the way it should be. I think I meant moderate_forum per se, though. Does giving moderate_forum permission to someone, also give them implicitly (or explicitly through code I couldn't find), the issue_warning permission...?
I'm giving up trying to understand what that group is meant to do/be...
There's a way about it, it's using the contact table Which we'll do, as it's already in the DB...
Well, I suppose we could add something in loadBoard... Check $topic, and if it's filled, add a query_see_topic?
Hmm... A relationship, you mean a comma separated list?
Hey, how about we keep a table with proper relationships...?
A table would be better but you still have to get the query running nicely, though having the option of subqueries should help.Because topic privacy is unlikely to be used on a majority of topics, it would keep the table quite small. Maybe we could even get rid of any privacy fields in the topic table, and just do a LEFT JOIN but I don't think it's very practical to do, just a silly idea.
Hmm, so it's definitely a no-no...
I really, REALLY want to know what you drink or eat to be able to answer things so quickly and so thoroughly.
But I'm not sure it's any good. I mean, why would I would people to be able to view something or not? I want authors to be able to determine that themselves... They're the ones who should know whether they want a thought to be public or private.
-
:edit: Okay, that was a long post... Could you go through it and, while you're reading, compile a list of everything we're discussing in it, and what needs to be done, what needs to be further discussed, etc...? Because I'm feeling lost. And the more I talk about something, the less time I have to work on it ;)
The only way to do it is to create a new rule.
There's already a moderation entry in the main admin settings menu, so we could have the master moderation setting there, something like "Enable Post Approval and Moderation Rules", maybe something shorter, and depending on that setting, we show the moderation filters menu or not... (Although it's likely that you'll want to have it very much in everyone's sight for starters, if only because it's a powerful new Wedge feature...) (Well, that could be a good opportunity to set post moderation by default.)
One possibility I just thought of, to make post mod faster... We keep a list of unapproved posts, right? I remember seeing that setting in the code I dabbled with this morning. Well, how about we simply add, in query_see_topic, a PHP check to see if there are any unapproved posts... If anything's found, add the t.approved code as needed. Otherwise, skip it!Though, I guess we could add a tickbox, but I think it's going to be incongruous and out of place (which is why I deliberately didn't go down that road, and tied it into the rules logic as it currently is)
As for disabling a filter without deleting it - why would you? Either it's not working for you and needs editing, or it is working and should be left alone.
Or if a pre-moderated area is quite slow and you're going on vacation and want people to be able to post freely to it -- you disable it, until you come back and re-enable it without having to worry about re-entering a list of filters or whatever.Yup, because you might decide to change the scope of the rule, e.g. you set it to new topics by accident but want to switch it to all posts.
As for 'posting in these boards', I couldn't think of a better way to phrase it. I suppose past tense is more correct, but the idea is to convey the notion that if the post is made in one of the boards you're picking (or not, depending on situation), that the rule will apply.
slideDown/slideUp didn't occur to me at all when building it, but I agree (having seen it used in the paid subs area too)
One of the annoying things with these JS toggles though (it's valid for both slide and immediate toggles), is that if you go back to a page (Back button) where you'd changed an opened div, usually the default opened setting will be reestablished and the default selection will stay the same (or the other way around), meaning you have to click once or twice to reset it the way you wanted. Anyway... It's just a minor tidbit, and something I never found a solution for.SELECT ... WHERE stuff is always going to outperform SELECT ... WHERE stuff AND m.approved... unless (and only unless) the stuff in the first place excludes everything anyway. If there are any rows found by the stuff clauses, they will have to be evaluated by m.approved.
I noticed earlier today while making my commit, that I'd at some put put a query_see_topic at the beginning of a query instead of the end. Oops! I immediately moved it to the end, specifically because I figured it would be evaluated last and thus would be much less expensive. So I was right, right?And... hang on... it is testing for m.approved only when postmod is active.
Can do if you like... it doesn't help that the rules are non typical because of the whole entity 039 thing,
I discovered this on sm.org in their showcase board when I was a Friend. I'd see topics in the message index I wasn't supposed to but because they weren't approved I couldn't actually see the topics themselves. loadBoard should deal with that AFAIK.
Some will want to show multiple badges too - e.g. if you have multiple groups. It's not an easy thing to solve.
It may be just a detail, but if it can save us from people asking for a plugin for this... And I don't even know if it'd be doable as a plugin anyway ;)Hmm yes, that's the way it should be. I think I meant moderate_forum per se, though. Does giving moderate_forum permission to someone, also give them implicitly (or explicitly through code I couldn't find), the issue_warning permission...?
Then again, I'd tend to go to someone's membergroup and check permissions accordingly. But it's always unclear what moderate_forum does. Plus, SMF/Wedge tends to... be a bit lazy about it, and sometimes it adds allowedTo('moderate_forum') for something, where in other situations it won't...In the same way that YaBB was Yet Another Bulletin Board, perhaps Wedge 1.0 should be codenamed YaSBS - Yet another SMF bug squished :P
A subselect should be faster than a FIND_IN_SET, I believe.
Well, I suppose we could add something in loadBoard... Check $topic, and if it's filled, add a query_see_topic?
Okay, done.
Interestingly, it was already done this way... I just had to add the AND {query_see_topic} to the INNER JOIN. Problem solved. Hopefully!A table would be better but you still have to get the query running nicely, though having the option of subqueries should help.
Oh, and reminds me of something... Maybe, MAYBE we should allow for topic privacy to be respected even by admins... I mean, somehow, it's exactly like private messages, or PMs sent to multiple recipients (in the case of "contact-list-based topic privacy").Because topic privacy is unlikely to be used on a majority of topics, it would keep the table quite small. Maybe we could even get rid of any privacy fields in the topic table, and just do a LEFT JOIN but I don't think it's very practical to do, just a silly idea.
Well, technically it couldn't be a left (or inner) join I think -- if it doesn't find anything, does it mean it's a public topic, or it's private but you can't access it...?A healthy dose of computer addiction ;)
But I'm not sure it's any good. I mean, why would I would people to be able to view something or not? I want authors to be able to determine that themselves... They're the ones who should know whether they want a thought to be public or private.
My concept:
Admin: does nothing
Author: determines who can view their thoughts
User: views thoughts based on author's decision
Your concept:
Admin: determines who can view thoughts and replies
Author: determines who can view their thoughts... Well, APART from the people prevented from doing so by the admin
User: views thoughts based on author's and admin's decision
A penny for your thought. I believe in freedom of speech -- and thought. Admins could possibly disable the thought system (which I'm wary of -- considering I haven't even begun to implement this in the first place... not because of laziness but because of freedom), but I'm not sure there's a point in letting them determine that only a single membergroup can access thoughts.
Of course, I *suppose* you thought of that because of Wedge.org's current implementation -- you're thinking that it'd be simpler to just give 'View thought replies' permissions to Friends, and 'View thoughts' permissions to everyone else... Right?
-
Because I'm feeling lost. And the more I talk about something, the less time I have to work on it
And it's going to cause many people to ask about it... Then we'll make a FAQ rule, that people won't read... Then you'll give up and add a master setting
There's already a moderation entry in the main admin settings menu, so we could have the master moderation setting there, something like "Enable Post Approval and Moderation Rules", maybe something shorter, and depending on that setting, we show the moderation filters menu or not... (Although it's likely that you'll want to have it very much in everyone's sight for starters, if only because it's a powerful new Wedge feature...) (Well, that could be a good opportunity to set post moderation by default.)
One possibility I just thought of, to make post mod faster... We keep a list of unapproved posts, right? I remember seeing that setting in the code I dabbled with this morning. Well, how about we simply add, in query_see_topic, a PHP check to see if there are any unapproved posts... If anything's found, add the t.approved code as needed. Otherwise, skip it!
In any case, doing what amounts to SELECT ... FROM topics WHERE id_topic NOT IN (list) is actually very slow. The indexes are very good at finding what's there, but not so good at finding what isn't there - instead of using the index, they have to go through and attempt to match each one. It isn't pretty. (Incidentally, it's why the stock implementation of ignore topics is so painful.)Your logic itself is sound and very commendable. I'm more worried about our users' logic. And mine
I don't know -- just say you wanna test whether the filter is actually useful, so you disable it for a while, and if spam comes back or whatever, you re-enable it.
Or if a pre-moderated area is quite slow and you're going on vacation and want people to be able to post freely to it -- you disable it, until you come back and re-enable it without having to worry about re-entering a list of filters or whatever.
Then I think it should be 'acknowledged' at the very least, for instance in the rule settings, instead of 'Posting in...', have something like 'Creating new topic in...' or 'Sending new post in...'?
Your way of phrasing it is correct, it's just harder to understand on first use. It might be better with the suggestions I wrote above.
Yeah, I like them a lot eheh. I try and use them anywhere I can.
I noticed earlier today while making my commit, that I'd at some put put a query_see_topic at the beginning of a query instead of the end. Oops! I immediately moved it to the end, specifically because I figured it would be evaluated last and thus would be much less expensive. So I was right, right?
Yes, in many queries. But not all... Look at Recent.php, for instance. There are more like that. I left them as is because I figured we should discuss it first.
Oh, my... That again? Oh, I try not to touch these with a 10-foot pole...
So it's the opposite of what I thought, right...? I thought that people wouldn't be able to see the topics in the topic list, but would be able to access them by ID...
Then again, I'd tend to go to someone's membergroup and check permissions accordingly. But it's always unclear what moderate_forum does. Plus, SMF/Wedge tends to... be a bit lazy about it, and sometimes it adds allowedTo('moderate_forum') for something, where in other situations it won't...
* access to registration management
* access to the view/delete members screen
* extensive profile info, including track IP/user and (hidden) online status
* activate accounts
* get approval notifications and approve accounts
* immune to ignore PM
* several small thingsIf not, then there's no point in using extra tables and such... Imagine a whole world running on FIND_IN_SET... Lol, what a mess
Oh, and reminds me of something... Maybe, MAYBE we should allow for topic privacy to be respected even by admins... I mean, somehow, it's exactly like private messages, or PMs sent to multiple recipients (in the case of "contact-list-based topic privacy").
You think...? Because it would be run even if topic privacy is set to default... (Of course, it wouldn't return anything.)
Well, technically it couldn't be a left (or inner) join I think -- if it doesn't find anything, does it mean it's a public topic, or it's private but you can't access it...?
And I'm not even getting started on this forum I found through googling for 'wedge fork' a couple of days ago, 'anotheradminforum', where I was surprised to find you and a bunch of our members discussing Wedge and other things... And nobody even cared to tell me Of course, you *had* to post nearly 3000 posts over there in the last 4-5 months, which is more than your post output on wedge.org... And I thought you'd gone quieter over the end of the last year
Of course, I *suppose* you thought of that because of Wedge.org's current implementation -- you're thinking that it'd be simpler to just give 'View thought replies' permissions to Friends, and 'View thoughts' permissions to everyone else... Right?
The thing is, privacy controls are wonderful - but they're also complex, no matter how you do them, unless your interface is totally geared to it - which is why G+ got it quite right, because so much dev time and effort went on the interface. Meanwhile, for what is, proportionate to the actual act of posting in a forum, a minor feature - privacy on thoughts is a bit more complicated than perhaps it needs to be.
If I don't care whether people can see it, I'll post it as a thought. If I do care who can see it, I'll post it in the relevant board.
-
Looks like you didn't compile the list I asked you to do... I know, I know, it's annoying for you too :P
If it requires a FAQ rule, it's already broken :lol: OK, then, put it in and we can see if it's necessary or not (it's not like we can't remove it in a later version if needed)... making such a setting be taken care of is as simple as updating Load.php to check for !empty($settings['postmod_rules']) || !empty($settings['postmod_override']) or whatever you want to call it.
.......I don't really want to change your stuff :^^;:
Wouldn't feel comfortable doing so.
I can only suggest what I feel would be best.We could certainly have a link to the moderation filters area from there. I don't like the idea of just configuring an option directly from the menu though.
Oh, possible re-ordering of the menus: (I'm translating from French to English from memory)
Features & Settings > Settings, Pretty URLs, Anti-spam
Moderation > Settings (currently 'Moderation' in 'Security & Moderation'), Moderation filters, and maybe a link to add a filter directly?
That would save us an extra entry... No?There is, of sorts, an approval queue, yes. It's a table with a single column, id_msg. We don't track topics, only the messages themselves.
Topics have unapproved posts.
Boards have unapproved topics and unapproved posts.
I can't find any global setting with unapproved posts, contrary to what I thought... :-/
Only an unapprovedMembers variable.
It would seem to me that unapprovedTopics and unapprovedPosts variables would be good things to have, if only to make the queries faster.
Basically, I thought that we had unapproved_posts somewhere, so it was easy thinking: because a topic is a post, if there are no unapproved posts, then there are no unapproved topics, thus we can skip the test entirely.Hmm, there's actually nowhere in the system to be able to activate or deactivate a filter. I guess one can be added, though.
Then I think it should be 'acknowledged' at the very least, for instance in the rule settings, instead of 'Posting in...', have something like 'Creating new topic in...' or 'Sending new post in...'?
(And no, I don't particularly like to give you more work to do... I'm busy on my own things, too :P)They're pretty nice, just I'd never thought of them before. On the subject of toggles, note that slideToggle and toggle do behave differently as far as forcing a state goes (slideToggle(false) does not do the same overall thing toggle(false) does)
Yes. Query conditions are evaluated much as they are in PHP, from left to right with AND/OR working much the same way. One thing... the order that fields are used in conditions is normally supposed to be the same order that the components of a primary/other key are supposed to be, but it's probable they fixed that limitation some time ago.
Ugh. My understanding was that if postmod isn't active, the checks shouldn't be performed. Anything contrary to that is a bug, especially since disabling postmod forced everything to be approved anyway. (It's not like us where clearing the rules leaves things unapproved!)
Heck... Maybe we could have a {query_see_message} as well. Why not? It'd be simpler -- just the 'approved' test. Although sometimes the tables have a different alias from 'm'...Oh yes. And it's not directly solvable just by doing that either, e.g. cases of XSS injection attempts get foiled here.
I'm on the fence about that. Private messages are not private in reality, they are merely personal. Though the core doesn't allow the admin to access it, it also doesn't do anything to prevent it (e.g. obfuscation of the data)
I mean, admins can't access anyone else's drafts either, can they...?Sure it would still be run, but the table it's joining to would be similarly small, so you'd just get an empty column coming back. If the table's small enough to be in memory it's going to be fast.
To be fair, though, those posts take less mental effort to do. They're also, typically, a *lot* shorter than my posts here. Besides, we're at the stage now where I'm doing a little bit of positive marketing without having to deal with "Post here and... and I'll fucking spank you"
(Oh, and one thing you shouldn't mention to me is Kevin Smith. I stopped watching Comic Book Men #1 after 10 minutes, it was felt so... unnatural, scripted and just generally not funny and uninteresting. All of that in only 10 minutes, when you usually pack all of your good stuff into the first few minutes to catch the viewer's attention. I had high hopes. :-/Of course, I *suppose* you thought of that because of Wedge.org's current implementation -- you're thinking that it'd be simpler to just give 'View thought replies' permissions to Friends, and 'View thoughts' permissions to everyone else... Right?
Oh, my. I just realized a thought privacy setting that SHOULD be mandatory in Wedge. "Me and the recipient", or something. Just like a mini-PM... And replying to such a thought would automatically set it as private, too.
Hmm, maybe it could be set within the 'author' privacy setting. i.e. if you reply another thought but set it to 'author', what good is it...? If it's just to say "Die, die sucker!", it's not very smart...The thing is, privacy controls are wonderful - but they're also complex, no matter how you do them, unless your interface is totally geared to it - which is why G+ got it quite right,
I just know that Facebook's privacy settings are more extensive now than they used to be...If I don't care whether people can see it, I'll post it as a thought. If I do care who can see it, I'll post it in the relevant board.
-
I have this fear all of a sudden... If t.approved=0, then m.approved=0 where m.id_msg=t.id_first_msg...
Which means that if you do a m.approved=1 test on any query, it WILL correctly ignore unapproved topics...
As a result, I think that there's a big problem with query_see_topic, in that it can and will only be called on queries that have a topics table in them, and if there's no need to access the topics table to see if it's actually approved... It means we have to add inner joins on every single friggin' table.
Hopefully, I'm wrong.......?
Posted: February 26th, 2012, 03:16 AM
(Bump for this last post. And a friendly reminder for the one above, but it can wait a bit longer :P)
-
Looks like you didn't compile the list I asked you to do... I know, I know, it's annoying for you too
Wouldn't feel comfortable doing so.
I don't think I said that...?
Oh, possible re-ordering of the menus: (I'm translating from French to English from memory)
Features & Settings > Settings, Pretty URLs, Anti-spam
Moderation > Settings (currently 'Moderation' in 'Security & Moderation'), Moderation filters, and maybe a link to add a filter directly?
Configuration
Features and Settings
General
Pretty URLs
Anti-Spam
Moderation
Settings
Moderation Filters
Add a new filter rule
Hm. I'd rather bring anti-spam under the moderation menu, really. It's really closer to that than to general configuration. In which case we're really talking about:
Configuration
Features and Settings
General
Pretty URLs
Moderation
Anti-Spam
Warnings
Settings
Moderation Filters
Add a new filter rule
It doesn't save us anything and has a byproduct of pushing things to side entries - I deliberately made Mod Filters a top level item, much as I would do if Post Moderation were its own item (and not configured through mod filters), simply because it's a task of configuration that needs to be prominent - it's forum wide and not subtle.I have this fear all of a sudden... If t.approved=0, then m.approved=0 where m.id_msg=t.id_first_msg...
It means we have to add inner joins on every single friggin' table.
Oh. Wait a minute, now I do: it won't apply to messages after the first one.
Actually, though, that's still not a problem. When would you ever make a query against messages when you haven't already checked the topic in some fashion? Be that directly yourself or less directly by way of loadBoard()...It would seem to me that unapprovedTopics and unapprovedPosts variables would be good things to have, if only to make the queries faster.
Basically, I thought that we had unapproved_posts somewhere, so it was easy thinking: because a topic is a post, if there are no unapproved posts, then there are no unapproved topics, thus we can skip the test entirely.
On your time and your own terms, of course.
And no, I don't particularly like to give you more work to do... I'm busy on my own things, too
Sure, and this just reminds me that maybe I should have done something else instead of simplifying your JS code to use toggle(boolean)...
So... Shall we change these to use postmod_active all the time?
Heck... Maybe we could have a {query_see_message} as well. Why not? It'd be simpler -- just the 'approved' test. Although sometimes the tables have a different alias from 'm'...
Why XSS? If we're careful to replace 039 with a quoted single quote right before insertion, it shouldn't be a problem...?
Yes, and thoughts and topics, when set to a privacy of 'author', are not meant to be published. And thus, should be treated the same way as PMs, in my opinion. Unlike an unapproved post, which requires an admin to read them before they're available to everyone.
I mean, admins can't access anyone else's drafts either, can they...?
But how could it be faster than a subselect that, first, doesn't always get executed (yeah, we could apply the same stuff to the left join, but it'd be quite tiresome), and second, uses the same secondary table so should effectively be as fast as the join...?
I'm just surprised nobody mentioned that particular forum in here...
Hmm, maybe it could be set within the 'author' privacy setting. i.e. if you reply another thought but set it to 'author', what good is it...? If it's just to say "Die, die sucker!", it's not very smart...
I don't know about G+'s privacy features, I never actually used the thing (in spite of my account over there.)
I just know that Facebook's privacy settings are more extensive now than they used to be...
-
Okay, there's a lot to be said and I'll answer more thoroughly later. Right now I'm in the middle of Dae Jang Geum and the Highest Kitchen Court Lady got elected but the other High Court Ladies went ahead and decided to tell the King's Mother that they won't accept her orders because they're faithful to the rival family of corrupts and the previous Highest Kitchen Court Lady just got a heart attack and the preview showed that she'll be dead by the next episode so the new Highest Kitchen Court Lady won't have any support and I got spoiled and she'll be wrongly accused of treason against the king and she'll be executed so how's her protégée the heroine gonna cope with that, what with being sent in exile or what?
So, now you'll certainly understand that with such shitty soap opera crap, I'm totally hooked and won't look into this all for a while... :lol:
No, seriously, it's a great show. It's just that it's on an extremely cheap budget/production values that aren't worthy of the quality of its story. If you ever watched 'Lady Vengeance' (by the director of smash hit Oldboy), they both share the same main actress.
Anyway, jokes aside... What I'm thinking for now, is that I'll be creating extra tables and use these. In the end it's gonna make things easier for everyone, and we'll be able to go for a Google+ experience if need be, with multiple contact lists and such. (I'm thinking, "all contact" could be represented by an ID of '0' in the table. It would save having to manually enter an entry for each contact list. I'll still have to find a way to quickly find your contact lists before I start matching them in the topic_privacy table of course... Headaches ahead, but if the big guys do it, I'll do it too.)
-
Looks like you didn't compile the list I asked you to do... I know, I know, it's annoying for you too
And I guess no one else than us is reading this...That's how I feel about working with Aeva's code a lot of the time, actually.
I'm sure I could get to work on the plugin code too if I wanted, but I have to admit I was scared by the amount of commits related to it. It's just.. Easier to let people who use plugins debug it :P
Because one of my reasons for doing Wedge was that I could have in core whatever I had in my mods or custom code... :PSo... (I just find it easier to visualise it in a real hierarchy)
Hm. I'd rather bring anti-spam under the moderation menu, really.
It's really closer to that than to general configuration. In which case we're really talking about:
Configuration
Features and Settings
General
---------------
Pretty URLs
Moderation
Settings
---------------
Anti-Spam
Warnings
---------------
Filters
Add filter
(I'm also positive Pretty URLs configuration could still be moved somewhere else, so that Features and Settings could let go of the sub-menus.)It doesn't save us anything and has a byproduct of pushing things to side entries - I deliberately made Mod Filters a top level item, much as I would do if Post Moderation were its own item (and not configured through mod filters), simply because it's a task of configuration that needs to be prominent - it's forum wide and not subtle.
Technically, though, we could always do that restructuring later, to give people time to 'discover' the filter system, but then again we'll probably have forgotten about the restructuring by then... :PI have this fear all of a sudden... If t.approved=0, then m.approved=0 where m.id_msg=t.id_first_msg...
Oh, my...I'm not seeing the problem? If you're validating topic access, the topic's approved state is all that matters - when the topic is approved, the first post is also updated.
Look at the only query with a m.approved test. It doesn't test for topic privacy. Then, the subsequent query does a topic join but doesn't have query_see_topic, either. In the end -- it's broken.Actually, though, that's still not a problem. When would you ever make a query against messages when you haven't already checked the topic in some fashion? Be that directly yourself or less directly by way of loadBoard()...
It would seem to me that unapprovedTopics and unapprovedPosts variables would be good things to have, if only to make the queries faster.
Basically, I thought that we had unapproved_posts somewhere, so it was easy thinking: because a topic is a post, if there are no unapproved posts, then there are no unapproved topics, thus we can skip the test entirely.
Oh, I'm doing the privacy tables, and a bit of a bummer with indexes... I have no auto-incrementing fields, and just a id_topic, privacy_type (groups, members...) and privacy_id (group ID, member ID...). So basically an index on this would be based on all three items, not one more.
Would it help at all to add a primary key based on the three keys...? Or is it a waste of space? (Maybe if I add a field later it'll be useful, of course... But right now I'm not so sure and I'm not well versed in MySQL indexing.)So... Shall we change these to use postmod_active all the time?
Heck... Maybe we could have a {query_see_message} as well. Why not? It'd be simpler -- just the 'approved' test. Although sometimes the tables have a different alias from 'm'...
As far as tables go, there are plenty of times m isn't the table alias (message index is the one that really comes to mind) so it's not like we can directly make it easy that way, sadly.
Why XSS? If we're careful to replace 039 with a quoted single quote right before insertion, it shouldn't be a problem...?
To me, single quotes and double quotes can always be used in a field. It's just when *communicating* with that field that one has to be careful... i.e. you have to escape quotes at the last moment when inserting data. But that's really the ONLY moment I can think of where we absolutely NEED to escape stuff... Unlike a 039 that has no particular purpose in a table row.Yes, that's a valid POV (and no, drafts can't be accessed by anyone other than the author - but just as with PMs, admins can still read them in the DB... there are no other protections made in that respect, so the software doesn't enable but it doesn't proactively prevent it either)
So, basically, respecting topic privacy against admins means that query_see_topic will need to test against the privacy field, rather than just returning 1=1. I'm fine with that, BTW.I wasn't really debating subselect vs join, the real performance is whether the table's able to be kept in memory or not.
I guess it just never came up.
(And re Kevin Smith, his earlier stuff is far funnier than his latest stuff, though his later stuff is meant to be less comedic, I guess.)
Hmm, maybe it could be set within the 'author' privacy setting. i.e. if you reply another thought but set it to 'author', what good is it...? If it's just to say "Die, die sucker!", it's not very smart...
Essentially you create groups as you see fit and you can share items to none, one or more groups.
-
I haven't digested everything here, but wanted to catch up on the most important points (to me at least)
Look at the only query with a m.approved test. It doesn't test for topic privacy. Then, the subsequent query does a topic join but doesn't have query_see_topic, either. In the end -- it's broken.
a m.approved test is very close to a t.approved test because in SMF, an unapproved topic can't hold approved posts. I think
Would be fixable I think -- if anything, by renaming these custom aliases to 'm', and renaming the other 'm' to something else...
To me, single quotes and double quotes can always be used in a field. It's just when *communicating* with that field that one has to be careful... i.e. you have to escape quotes at the last moment when inserting data. But that's really the ONLY moment I can think of where we absolutely NEED to escape stuff... Unlike a 039 that has no particular purpose in a table row.
-
I haven't digested everything here, but wanted to catch up on the most important points (to me at least)
Look at the only query with a m.approved test. It doesn't test for topic privacy. Then, the subsequent query does a topic join but doesn't have query_see_topic, either. In the end -- it's broken.
Ah, but it can. This is one of the many foibles of the system. If you have a topic that is unapproved, the owner and moderators/approvers can see it. Should a moderator/approver reply to it (e.g. in response to the opening post, as per SMF Showcase board), that post will be approved, but the topic as a whole and its opening post will both be marked unapproved.
The point is, while I can't bring any examples immediately to mind, it's possible that someone can find a combination of ' and " that would allow them to escape out of the usual protective layers, so that malicious links or onclicks could be added.
So... About topic privacy, then. Back on this... And I'm scared. Really scared.
Considering that topic privacy can be given to either of these (one or more!):
- everyone
- only registered members
- only one or more membergroups
- only one or more contact lists
- only one or more members
The query_see_topic can quickly become quite a behemoth...
Pseudo-code.
privacy = everyone
OR (privacy = members)
OR (
privacy = contacts
AND t.id_member_started != 0
AND FIND_IN_SET(<user_id>, (SELECT buddy_list FROM members WHERE id_member = t.id_member_started))
)
This is what we have right now...
What customization will give:
privacy = everyone
OR (privacy = no_guests)
OR (
privacy = contacts
AND (SELECT id_member FROM contact_lists WHERE id_contacts = <privacy_id> AND id_member = <user_id>) = <user_id>)
)
OR (
privacy = groups
AND (<privacy_id> IN (<user_membergroup_list>))
)
OR (
privacy = members
AND (<user_id> = <privacy_id>)
)
Now, that looks like an acceptable price to pay... Only, this query is no longer done on the topic table itself, but as a JOIN on the privacy table. Meaning that, first of all, we need to add a row for every topic (saying 'privacy = everyone' by default), even if everyone can view it. Or we have to use a has_privacy field on the topic table and use the above code in a subselect instead of doing a join. Then, it's a BIT annoying to do a subselect on every contact list test, maybe we should also maintain a list of contacts in the member table -- maybe even two lists: the contact lists you own, and the contact lists you're in. (Heck, at that point, maybe we don't even need a contact list table... Then again we'd have to have a huge field for the contact lists you're in... If only because I suppose you could be in multiple contact lists *per member*...)
Also, what about if I'm found out to be a positive for topic privacy as soon as the first item in the privacy list? Will the JOIN keep going through the table? Would a LIMIT 1 be enough to stop it from doing that? But then we need a subselect, right...?
So much fun. And we get to do this for both topic and thought privacy... *And* write the UI later as well... Eh eh.
-
So, technically, anyone will be able to read their post in the Recent Posts area, but not click on it to see it in context...?
Also, what about if I'm found out to be a positive for topic privacy as soon as the first item in the privacy list? Will the JOIN keep going through the table? Would a LIMIT 1 be enough to stop it from doing that? But then we need a subselect, right...?
So much fun. And we get to do this for both topic and thought privacy... *And* write the UI later as well... Eh eh.
Or query_see_message when no topic join is available... I don't really see a better way of doing it than forcibly adding a subselect on the topic.
Ready to digest, or giving up?
-
Wasn't there a t.approved = 1 test there as well? I don't remember. But it wouldn't surprise me if it were buggy.
But we'll definitely need to audit most of the queries.
I think I'll do it like I did on Noisen.com back when I implemented the feature... I'll create a few topics, add a hidden word in them, set privacy to myself on them, and tell everyone that if they ever find the word, they should contact me immediately and I'll reward them for helping protecting the website's security settings.
It'll only be made more important by the fact that people are really gonna use these privacy settings...
Oh, while I'm at it.
I'm still working on database structure, getting close to the end thankfully. The major change these last few days, is that I decided that if topic privacy is going to be so specific ("member 34 and group 12..."), that it would only make sense to use the same feature on board access. We'll need to keep in touch to make sure it doesn't conflict with board access/view permissions, obviously... (I'm thinking board privacy should be tested first, and then these permissions. Does Wedge do a {query_see_board} BEFORE the permissions are tested? In which case it should be fine.)
Now, my main concern is with the has_privacy field. It's a tinyint(1) field (i.e. bool), so it won't take any significant space in the DB, but I'm not sure that the field will be taken into account by the key selection system in MySQL...
For performance reasons, I *suppose* it's best to do the test for that bit as late as possible in the process...? i.e. if its index isn't used, might as well execute the test last, so that it doesn't waste time doing a filesort or something on the table? (Yeah, after all these years I'm still THAT bad at MySQL optimization...)
In which case it'd mean we have to move all query_see_board tests to the end of all queries. I don't know.
Now, the boards table has a member_groups key, all alone by itself, so maybe it's really not a problem and our has_privacy field will not make it slower... (Assuming that the board table has thousands of entries, which is ridiculous normally, but not when you add a feature to allow people to create their own blogs. Will do... Later.)
Anyway...
So, if has_privacy = 0, there's nothing else to do. If has_privacy = 1, we join the privacy_boards table (I've switched the two words so that all privacy boards are next to each other, easier debugging), and we test for availability.
Also, Noisen's topic privacy feature has a 'default' which uses its parent board privacy settings. (IIRC it'll always use the parent board's privacy and then apply its own on top.) I'm assuming that all 'sensitive' topic queries in SMF/Wedge already do a query_see_board on them, so it shouldn't be a problem, but what if it is...? We don't want to show, in a list of topics, some topic that has no privacy settings but is in a board that shouldn't be viewed... But I suppose this privacy issue would have been spotted first, right..?
Obviously we'd have to remove the member_groups field from wedge_boards and have TE include some code to import these entries into the privacy_boards table. Shouldn't be a problem...
NB: board privacy is not yet included, only did the table in the sql file, I just want to discuss it and make sure it's the right direction... But I somehow suspect that if people are gonna be able to create their own boards (once we had a permission for them to do so), they'll be *thrilled* to know that they can fine-tune who gets to see them.
Noisen has such a feature already, but if I'm going to write a complex JavaScript UI for maintaining privacy lists, might as well use it everywhere...
NB2: Okay, I looked into board access... Oh my.
Wedge adds the ability to choose whether a group can KNOW (view) about a board, can ACCESS or board, and can be prevented from knowing or accessing a board regardless of other group memberships.
That is very fine, but if the board is created by a user who wants to prevent someone they dislike from viewing it, they won't be able to do it because they can't create membergroups... Unless we allow them to do just that (which would also make topic privacy and thought privacy simpler to manage, let's just say...)
Maybe I'm "in the wrong". Maybe our "contact lists" should be membergroups and so on. Maybe we need to silently create a membergroup that contains just users 5 and 47 if topic creator requests that only users 5 and 47 can read it. (And remove the membergroup if they change privacy settings for that topic and no other privacy area uses that group.)
I don't know...
Oh my. It's never going to end, is it...?Would need benchmarking and query plans to be sure, but AFAIK, if the first branch is matched, the rest aren't if a given row would be returned in the OR case (if x OR y causes a row to be returned, it will be returned as soon as x is matched, or if x isn't matched, then return if y is matched)
(Or they really need to buy huge equipment and not bother. I don't know how Facebook does it...)Ready to digest, or giving up?
I could not conceive of a more backwards-ass specification if I *tried*, and to a degree I've just been thinking about everything else rather than focusing on this >_> I keep getting feelings of having so much to do, you know?
I'm just a bit sad that I have so much desire for user freedom when writing privacy settings, and that it suddenly seems so unrealistic a goal to achieve with regards to database performance...
-
I have this horrible feeling that I won't get an answer anytime soon to this long post, so let's just make it simple...
Long ago I decided to add 'custom membergroups' to Noisen. I implemented it. Then I removed it and ended up using UltimateProfile's custom buddy lists.
When we made Wedge, we discussed adding custom membergroups. Then the idea was abandoned, too.
I just don't remember WHY I gave up on these the last two times, when it seems like the 'cleanest' way of implementing user relationships on a website... (Having multiple relationship types, like groups and contacts, is just inefficient IMHO when we could simply merge both table types.)
Does anyone remember reading a conversation around here about membergroups...?
-
That's the thing, it's about control, and who has what control.
The admin can set up who's in what groups, not the user. So it seems a bit odd to expect the user to use that setup, whereas if they're using buddy lists, they have control over who sees their topic.
-
Admin = can control ownerless groups
User = can only control own groups
No?
-
Yes, and that's the point.
If a user wants to restrict who can see a topic themselves, they want control over it, i.e. setting the list of people who can see it. If they don't have control over that list, why would they want to use it?
-
Well... Friend list?
Member groups would get extra fields. I'm sure it'll be all right.
My main concern is that if you have thousands of friends, your membergroup field will overflow in the member table...
-
Which is why it can't really be done that way :(
-
Except if we increase the text field size..?
-
I guess 1MB is a big enough container for most sites, but it's really not that useful when it's scaled up because you just can't index that field in any fashion forcing a table scan to query it.
-
Yep, but even 1MB sounds quite big when it comes to doing tests like id_group IN (...) and then you start listing the entire 1MB field in it...
My idea is just that contact lists and membergroups are one and the same thing. They're groups of members. One member can be in multiple groups, and a group can hold multiple members, so it's a many-to-many relationship, and I don't really see why SMF's membergroup memberships are never kept anywhere but in the members table.
Look at ManageMembergroups: to retrieve a list of members for a specific group, it does a query on the members table with a find_in_set... Not exactly optimized.
But even if we leave membergroups aside, and add contact lists in a similar fashion to how UltimateProfile did it (i.e. keep the buddy_list field in the members table for find_in_set searches, and add a relational table that holds lists of user IDs relative to a buddy list owner), the problem is exactly the same... Instead of holding membergroup IDs in the buddy_list field, you hold member IDs, and it's quite likely that there will be many more members than there are membergroups (eh), so if you have hundreds of 'buddies', the field will be overflowing before you know it... AFAIK at least.
So, basically:
With contact lists:
- a contact_lists table that holds a list of all existing contact lists (id_contact_list + id_owner)
- a contacts table that holds a list of all members associated with a specific contact list (id_contact_list + id_member + possibly id_owner)
- a buddy_list field in the members table that holds a list of all members who are in your own contact lists
- possibly replaced with 'simply' a comma-separated list of your contact lists. In which case, a JOIN will be needed to get the member list. Making the comma list a bit useless.
With membergroups:
- the existing membergroups table, with a new field (id_owner) indicating who created it, and maybe add a new type (contacts) or something.
- possibly a memberships table (similar to the contacts table), with id_member, and id_membergroup relationship, and maybe a 'hidden' flag (i.e. whether or not this membership should be disclosed), things like that, and an id_owner to avoid having to join the membergroups table if you want to retrieve a list of someone's friends.
- the existing members table, with a comma-separated list of membergroups that you BELONG to. If you need to obtain the list of your friends, you would do a JOIN on the memberships table, as mentioned above.
I think we already discussed the fact that find_in_set isn't very efficient and that maybe we should move membergroups to outside the members field... If we move it entirely out, then the field size is no longer an issue. Otherwise, it still is, but not for some time at least.
Meh...
-
My idea is just that contact lists and membergroups are one and the same thing. They're groups of members. One member can be in multiple groups, and a group can hold multiple members, so it's a many-to-many relationship, and I don't really see why SMF's membergroup memberships are never kept anywhere but in the members table.
As far as membergroups vs contact lists go, yes, they're fundamentally the same thing - lists of users - but membergroups have a lot more to them. There's multiple types of membergroups, they have badges, they have PM limits and permissions and *stuff* attached to them. They also get the distinction of having three separate elements of the members table (primary group, post count group, other groups)
I have no real objections to making contact lists and membergroups basically the same thing, provided that proper protections are taken to ensure that the two are not permitted to overlap (e.g. you can't edit or view a contact list anywhere you're not supposed to be able to and vice versa)
In fact, that reminds me of something I considered... I considered the possibility of ditching post count groups as they are currently implemented and reimplementing it separately so that they're not physical groups, but instead a separate entity (that has badges etc.) and rearranging permissions so that they're set up based on being granted at different post counts rather than by groups. Needs more thought to explain what I have in mind but it would simplify this setup and move it closer to what you're looking at doing.
-
OK, last point first, that's mostly historical. MySQL until later on didn't have subselects at all, and we even removed the block on subselects in the code. Not only that, but there's also a performance consideration - going back, the belief was fewer queries = better, and if you can't do subselects, the best method is to denormalise the data somewhat and do what was done.
So to me that's an extra reason to just drop these.As far as membergroups vs contact lists go, yes, they're fundamentally the same thing - lists of users - but membergroups have a lot more to them. There's multiple types of membergroups, they have badges, they have PM limits and permissions and *stuff* attached to them.
They also get the distinction of having three separate elements of the members table (primary group, post count group, other groups)
I have no real objections to making contact lists and membergroups basically the same thing, provided that proper protections are taken to ensure that the two are not permitted to overlap (e.g. you can't edit or view a contact list anywhere you're not supposed to be able to and vice versa)
I'm more interested in determining whether you think that performance-wise, having a separate membership table would be better for membergroups overall (not only for contact lists.)In fact, that reminds me of something I considered... I considered the possibility of ditching post count groups as they are currently implemented and reimplementing it separately so that they're not physical groups, but instead a separate entity (that has badges etc.) and rearranging permissions so that they're set up based on being granted at different post counts rather than by groups. Needs more thought to explain what I have in mind but it would simplify this setup and move it closer to what you're looking at doing.
-
But, because membergroups are not usually exclusive, if you don't apply any setting to a membergroup, Wedge won't even bother with it... Basically, if id_owner > 0, then Wedge should reject them from the admin area when it comes to applying special settings to them.
I'm more interested in determining whether you think that performance-wise, having a separate membership table would be better for membergroups overall (not only for contact lists.)
Namely, the way permissions are generally loaded would have to be rethought, as also whether we store the user's primary group in their user record too. What we'd end up doing is pushing the permissions check out of the loading the member's main record, and pushing it off to loadPermissions where we'd end up querying for user's groups LEFT JOIN permissions table for the permissions for those groups (since that's the only way to identify users in group 1, who don't have any entries in the permissions tables otherwise)
And of course you have to be careful not to join the contact lists for the purposes of permissions.
That part is mostly not a *huge* deal in performance, but where it will make a difference is topic privacy as you suspect. The problem though is that I don't know *how* it's going to make a difference. Honestly I don't think there's any better way than to benchmark it and see what happens.
What I do know is that it will have a beneficial impact on handling users who want to implement shared forums (multiple forums with a single members table) to not have the groups tied explicitly to users in the members table.Hmm... About that, I have a feeling it could be complicated to implement. And if it doesn't help with performance but only with UIs, I would recommend against it.
That might not sound like a problem but I've seen WAY too many cases where people delete all the post count groups. That wouldn't be so bad if it weren't for the problem that results where it fucks groups up leading to any user anywhere having full admin access because of the way the post count assignation query works.
I'd almost argue that the UI simplification it would bring for setting permissions would actually be enough.
-
Still. Your post scared the shit out of me!
And I don't care if it isn't proper English :P