Thought system

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #60, on November 9th, 2011, 09:50 PM »
It doesn't need much indexing, it's mostly accessed by primary key anyway...
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

Nao

  • Dadman with a boy
  • Posts: 16,079

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #62, on November 9th, 2011, 11:50 PM »
Media items are always referred to by their id only, which is a PK lookup, and then a lookup against the album's permissions.

Attachments are currently done as a request referring to the attachment id, plus the topic id. I'm not quite sure why that's done that way, because it causes two problems. Firstly, it means that if a message is moved to another topic, any pre-existing link in posts is broken. Secondly, it means there's a lookup of attachments table (by attach id) to messages table (by id_msg) which will then return the topic.

The only reason the topic id is even given in the first place is so that ;topic= is filled in the URL and the board permissions are implicitly loaded as a by-product. That means a BFQ[1] is issued, one that queries the topics table, the boards table, the categories table, plus fetching the board's owner, board moderators... yeah. That's not clever.

So pushing it solely to id and managing the access and permissions internally would almost certainly be at least as fast, if not faster, than the current approach.

The exact method to be used is up for debate, since it implies allowing for unexpected alterations and extensions later, but the principle seems clear enough. That's the 'main' problem with using attachments, from my perspective.

The other issue is relating to how the file path is determined. Right now, the path is determined based on the album's information. What I figure is that, invariably, a second query is going to have to be made. The first query identifies that the file exists, and would return the information necessary to locate and pass to the authorisation subsystem.

The authorisation subsystem would not only identify that the file can be passed on to the user but should also return the path to it.

As far as files being attached to posts, we could retain the current UI. Once the files are uploaded, we can insert the equivalent of the smg bbcode into the post itself, meaning that users are free to reposition the items within the page if they wish.

I'm sure there's more but that's all I have right now in my little brain for it. (Busy watching The 7th Voyage of Sinbad. Awesome film.)
 1. Big fucking query.
Re: Thought system
« Reply #63, on November 10th, 2011, 10:33 AM »
Getting back to the original topic, I find two problems and one thing I don't understand.

First up, there's a bug in it whereby nobbc is not handled properly. It's not a potential security issue as far as I can see, because htmlspecialchars is run still, so even though nobbc is not parsed the same way as other tags, it won't do anything bad. It just won't work as expected. (Since nobbc is available to the inline parser, we need to run preparsecode on it to actually carry out the inlining on it, I did wonder if that would be the case, though.)

Second, the JS is broken if Packer is used. I haven't figured out quite where that is yet, but it gets very upset with "Unexpected token (" which breaks all remaining JS on the page (which is problematic as it also breaks things like the menus, fortunately menu navigation isn't the only route to Server Settings now ;))

Also, is it intentional that links posted in thoughts should not be clickable? In fact... the URL tag doesn't work. It's parsed in the display view but it's raw BBC in both the thoughts view in profile and in the sidebar box.

The one thing I don't entirely get is how a user is supposed to reply to thoughts, there's no interface like in Noisen or here.

Otherwise though, it looks pretty good :)
Posted: November 10th, 2011, 10:29 AM

On a related note, the inline parser has 'color', 'black', 'blue', 'green', 'red', 'white' listed, but this should just be 'color' as preparsecode does the conversion. (Though it does it case-sensitively.)

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Thought system
« Reply #64, on November 10th, 2011, 11:55 AM »
- I'll test with Packer.
- I haven't been using preparsecode, as I said before. It's something I'll have to deal with, too.
- Sidebar box doesn't show a parsed thought because the Ajax stuff doesn't return anything. It was done this way on noisen, never cared to develop further... Obviously, I *should* return a parsed string to show. And I will.
- BTW, if you add a URL between nb tags, it won't be parsed into a url tag. Just a limitation I noticed yesterday in here... Dunno if it's in Wedge too, but I suppose so.
- I'm kind of on a break right now because, well... I thought I was done but I started logging a lot of items I should do, and I figured, oh crap this is going to take me another week... So right now, it's all you get :P

- Color stuff: ah, yes, thanks for the tip...
Posted: November 10th, 2011, 11:51 AM

Update: not getting any problems with Packer...?

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #65, on November 10th, 2011, 12:02 PM »
Quote
- I haven't been using preparsecode, as I said before. It's something I'll have to deal with, too.
I'd assume that newlines should be stripped before sending to preparsecode, but that's the only gotcha as far as I'm concerned.
Quote
- Sidebar box doesn't show a parsed thought because the Ajax stuff doesn't return anything. It was done this way on noisen, never cared to develop further... Obviously, I *should* return a parsed string to show. And I will.
I had a horrible discovery on that note the other day. I put a thought reply of 'I made him say <odd item> though... ' in and I was more than a shade bothered when I found it didn't protect the tag on the inline save (but of course it is when it's held on the backend, and reloading the page is of course fine)
Quote
- BTW, if you add a URL between nb tags, it won't be parsed into a url tag. Just a limitation I noticed yesterday in here... Dunno if it's in Wedge too, but I suppose so.
Really? Fairly sure I've put links in footnotes before and it worked as expected. In other news I've been quietly wondering if the footnotes couldn't be made even more awesome: if you have two footnotes with the same contents, I feel that only one entry should be put at the bottom and the same number used in both cases. The main reason this came into my head was because for one thing I want to do, I want to be able to do Wikipedia like citations by reusing footnotes instead of having to roll my own (because I have a project in mind that I would rather use a customised Wedge, or even a customised SMF,[1] for than MediaWiki)
Quote
I'm kind of on a break right now because, well... I thought I was done but I started logging a lot of items I should do, and I figured, oh crap this is going to take me another week... So right now, it's all you get
I know the feeling. I'm currently tackling moving the calendar into its own plugin. It's... interesting.
Quote
- Color stuff: ah, yes, thanks for the tip...
Side note: I seem to recall the bbcode parser will chew through uppercase bbcode e.g. [WHITE] but tags like that won't be converted, because the regular expression just replaces white|blue|black|red|green with color=$1 without any callback or inline eval to strtolower it. (Since this is at preparsecode time rather than display time there's no reason why it couldn't be rewritten to use a case-insensitive match and strtolower it once found)
Quote
Update: not getting any problems with Packer...?
Chrome doesn't like it for some reason. This was off a totally fresh install, too.
 1. As much as developing on Wedge is awesome, for a project that is a one-off codebase, I'd rather build it once and leave it without having to do regular patching and updates, if that makes sense.

AngelinaBelle

  • Still thinking...
  • Posts: 92
Re: Thought system
« Reply #66, on November 15th, 2011, 08:36 PM »
I second the whole moved-topic-attachment-link problem -- learned not to link to attachments. Only to gallery items.

Using a gallery for attachments is complicated because then the attachment can have a permission COMPLETELY different from the topic/board it is attached to, which could be tricky and confusing.

Using the gallery instead of attachments. But that's OT.
I'm an SMF doc writer.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #67, on November 15th, 2011, 09:44 PM »
That's precisely the point: while normally a gallery item is tied to the album it was in, I figure that if you attach an item to a post, it should use the permissions of that post, but there's no reason it has to advertise it to the user, it should "just work", and we can display a footnote somewhere on the item to indicate who can see it if needed.

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Thought system
« Reply #68, on November 16th, 2011, 01:07 AM »
But it's the eternal question: does the user WANT their item to be tied to the post's permissions...? Maybe they want it to be accessible from the gallery.
Which is (probably) why we need to enable this only for items in an 'attachments' album, i.e. if the user chooses to post in another album, the item will use the album's permissions, while if they post in the general attachment album, it'll use the post's permissions and a reprint won't be allowed in another post.

Which of course also begs the question... Are people going to understand how it all works :lol:

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #69, on November 16th, 2011, 01:17 AM »Last edited on December 12th, 2011, 05:54 PM by Nao
Quote
Which is (probably) why we need to enable this only for items in an 'attachments' album, i.e. if the user chooses to post in another album, the item will use the album's permissions, while if they post in the general attachment album, it'll use the post's permissions and a reprint won't be allowed in another post.
That wasn't really my intention, because it's not really want I would want to use. I'd personally prefer that it uses the permissions but that reposting should be permitted, so that if I attach a file to a post, that isn't in a regular album, I would want to be able to repost it elsewhere - for example, the logo madness thread here.

If it's tied to visibility behind the scenes so that users don't have to worry about it, they can just repost the image without any concern, e.g. through whatever equivalent of the smg tag we end up with. That way, it's no longer tied to the *topic* as it currently is (with little good reason, IMO)

The idea, primarily, is to make it as easy as possible to get files onto a post with as little effort as practical - it being in the gallery makes it easier to reuse later.

DoctorMalboro

  • I like rounded borders.
  • Posts: 316
Re: Thought system
« Reply #70, on November 16th, 2011, 02:39 AM »
I like the idea, but I don't know how many people will use it with other social networks added, maybe it could be added as a plugin the social integration...

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #71, on November 16th, 2011, 02:41 AM »
I somehow don't think it's going to become a plugin.

It was a requirement for migrating this site to Wedge (because it's actively used here) but it might end up being a path for reposting to, say, Twitter from the forum.

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Thought system
« Reply #72, on November 16th, 2011, 10:07 AM »
Quote from Arantor on November 16th, 2011, 01:17 AM
That wasn't really my intention, because it's not really want I would want to use. I'd personally prefer that it uses the permissions but that reposting should be permitted, so that if I attach a file to a post, that isn't in a regular album, I would want to be able to repost it elsewhere - for example, the logo madness thread here.
That seems to be in line with what I suggested..? What exactly is there to clear up between our visions of the thing?

Also, I'm starting to wonder if we shouldn't simply keep the attachments table and files and everything, and *just* move most of the front-end and back-end manipulation to the Aeva codebase...
Well, that would be something in-between no implementation and full implementation. I don't particularly like that either...
Quote
If it's tied to visibility behind the scenes so that users don't have to worry about it, they can just repost the image without any concern, e.g. through whatever equivalent of the smg tag we end up with.
(That's the media tag, now, BTW.)

Also, regarding rev 1165... I must admit I haven't followed any developments regarding this bug. I had a quick look this morning, and apparently it's a subject that's been discussed at the AeMe topic at sm.org these last few days, with a weird fix posted, and I'm not even sure what your fix does, whatever, over at sm.org it seems the problems are with Firefox 8 and here they're with IE9...?!

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Thought system
« Reply #73, on November 16th, 2011, 12:53 PM »
Quote
That seems to be in line with what I suggested..? What exactly is there to clear up between our visions of the thing?
You said that if it were tied to posts, users wouldn't be able to repost it... whereas specifically I'm saying they should be able to do so. They just don't need to worry about the other details.
Quote
Also, regarding rev 1165... I must admit I haven't followed any developments regarding this bug. I had a quick look this morning, and apparently it's a subject that's been discussed at the AeMe topic at sm.org these last few days, with a weird fix posted, and I'm not even sure what your fix does, whatever, over at sm.org it seems the problems are with Firefox 8 and here they're with IE9...?!
OK, here's the problem.

There's the Content-Disposition header, which is supposed to indicate what the browser is supposed to do with the content that follows, except the specification pretty much states it should be ASCII characters only. Which is fine, right up until you have high bit characters in the name, which in non English forums is more common than you'd think. So there is a quasi-extension that's now pretty much standardised which allows a content provider to state that the data is really UTF-8, except browser support is quirky for it.

The original code in SMF issues separate different headers for IE, Firefox and Opera, I changed that a year ago to IE and everyone else.

Now, the problem is that IE9 got its act together and doesn't like the header being used for IE because it's special, ugly and it wants to conform with the in-crowd. Trouble is, Wedge recognises IE9 as IE (and SMF does not), so it was given the IE specific codepath which spits out the wrong filename on IE9. So the first change was to make the IE-only code path into IE8 and down.

The second change is that FF8 stopped accepting the otherwise-standard header, it would ignore the filename entirely because it didn't like the quote in the name. So the quote has to be changed for another delimiter, which caused it to start choking on spaces and other things, so the entire thing has to be urlencoded and it must also include spaces in that to be %20, rather than +, hence the rawurlencode call.

As far as I can tell, this works properly on all the big browsers. I'm not so sure it would work so well for SMF even though I gave the few lines in question to be reposted; there was a rather curt "this is the same fix I posted" note, except that it isn't, for several reasons (my fix doesn't bother to differentiate ISO and UTF-8, and I don't have a special case for FF, merely IE8 and down - when they get round to listing IE9, they'll have to fix it again, but hey, what do I know?)

Nao

  • Dadman with a boy
  • Posts: 16,079
Re: Thought system
« Reply #74, on November 16th, 2011, 11:14 PM »
Thanks for the details.
Just for the records, what I did in rev 1167 is trying to follow the test cases described here:
http://greenbytes.de/tech/tc2231/
A real goldmine... And scary, too! Anyway -- hopefully I got the code right.