Wedge

In a situation like this, my original code would first fix the mismatched tags in the middle, then it would look for an opener to 's' and eventually add one at the very beginning... Which would break everything because there's a quote in between. So I added a 'last_safe' variable which pointed out what was the last *safe* place to insert something (i.e. anything BEFORE it is considered valid and thus shouldn't be messed with again.)
Problem is, in a situation like that, the variable would be, at best[1] set to... The s closer's position. So we'd end up with an opener, immediately followed by a closer.

So... I'd like to know which you think is best. Shall we:

1- silently remove closer tags if no openers were found?


2- add an opener right before them?


3- leave them be, whatever, except maybe for code tags?


:edit: Added a footnote. BTW, my favorite is (1), personally...

1.	I'm saying "at best" because there's still an open quote at this point, so I'd have to implement code that would check LATER tags to make sure the quote is actually closed itself and is thus safe...

Thankfully you know about them PantsMan. Imagine how dreadful it is to provide support for things like that on shared hostings for noobs. ;)

Oh... Not a good sign... I'm already lost in my recursive code... :-/

PHP has a PECL library for bbcode parsing, but it only returns html, not fixed bbc.
There's also a PEAR library written in PHP, it seems to be nicely written albeit a bit large (30KB for main source + more for specific tags like url...), and doesn't provide a fixer by default -- but it does seem to fix tags on its side.
I've also found "NBBC" on sourceforge, it has an extensive test suite and documentation. Unfortunately it doesn't provide a 'check' mode either, only converts directly to html but at least this one is BSD.
It doesn't generate opening tags automatically when finding an orphan closed tag, though... So apart from its alleged speed, it's not that interesting. Plus, it's huge.I think the whole point of the html5 tokenizer is to provide for a common ground to handle errors...?Yes, that's pretty much what fixNesting does...NBBC does that. (Wedge, too, obviously. But not fixNesting.)The reporting is already being done... If it were JUST about that, I wouldn't have worked on the alternative 'fixer' today.
Unfortunately I can't really feel satisfied with just the report code because it's not going to be used in quick edit etc.
(Well... Unless we add an errorbox for quick edit, of course. Which would not be such a bad idea...)The end of the line? :PWe could always draw our inspiration from the best elements in wiki code...

Anyway. Time for bed.
I would have posted my 'bad work in progress' from today, but the source code is pretty fucked up (commented out code, echos and print_rs everywhere...), and I don't want to ruin my reputation :P

Yeah... Didn't think about this all...

Well, it's all pretty fucked up really. A waste of my time... -_-
Basically, it works 100%, until you meet some of the more complicated stuff I have in my test case. And I don't see how to fix it without, tadaaam... Another full rewrite!
If I do this... I'll probably have to, hmm... Do it recursively... Oh, I don't like that... :-/

Y'know, like, "if a tag is opened and is not self-closed, re-run the function on the string AFTER that tag, asking for it to return after it meets the closer tag..." And if it meets a closer that's not the one we're expecting, we'll just add our closer there (manually), and return from the function.

What annoys me the most is that every time, I get this very simple idea that ends up being flawed in one aspect or another... I just don't want to spend another day on that.

PS: any special tags that aren't in the list of double tags, like x or * or whatever, are not important here because Wedge will simply ignore them anyway.

PPS: my test case is as such. It chokes on the first [/nb] (which doesn't have a matching opener at this point because we already rewrote the first opener to add a closer to it.)

Allow me to quote myself:So, yes, I totally agree... It's just that my original pseudo-code doesn't account for that, and I knew I was in for a complicated day if I didn't start by clearly writing a few broken posts and figuring out what would be the most reliable way to fix them.But *once* it is redone through ::fixNesting, with my super-solid code etc, won't all of the remaining code become unnecessary all of a sudden...? :PThe only fixing I've seen preparsecode do is add code tags at the beginning or end of a post which sucks a bit...
::parse_bbc does some fixing on its own, IIRC, but if it does, it's in the wrong place. It should be done at save time, obviously. (AND, we should remove any fixer code from ::parse_bbc to force modders to go through ::preparsecode. Believe me, I didn't even know this function existed before Shitiz used it in SMG, and I didn't have the *reflex* to use it systematically until, err.... Now?)But I'm not writing an AI... Otherwise we won't release before 2015...
Things like, "Okay, this is a closing tag BUT it's at the beginning of a line *and* is immediately followed by content, so MAYBE it's an opener, let's try to turn it into an opener and see if it suddenly validates"... These things are doable, but they take time to implement.

(Well, that particular solution would still be a very good one to the first test case I posted.)

Thanks, I didn't notice that one bit ;)

Okay, I'm in the process of moving the code to the wedit object (where it should have been from the beginning), and trying to fix the code automatically... So, considering the fact that the most important content (topic posts) is okay because we clearly specify the errors, fixing posts automatically shouldn't be too much of a hassle but obviously the code to actually fix them is going to be more complex...

So, let's say I have this:


I think it's safe to say that the poster added a / by mistake, and it should be removed, but I hardly see how Wedge is going to be able to spot it automatically without going into a great deal of large-scale testing. So I suppose we could do it this way...


i.e, the extra /quote knows there is no previous matching quote tag, so it simply looks for the *last* tag it found (or MAYBE the last closer tag of its kind, i.e quote?), and it adds an opener tag right after it. Then we mark the tag as fixed (i.e. we remove 'quote' from the stack of opened tags). So, we are in a situation where 'comment' is suddenly stuck into a quote tag. Continue as usual. Then we spot the other closer quote, and we do the same, i.e. add a closer tag after the last closer, so in this case the 'post2' bit is fixed.

The obvious problem with this solution is that our comment is now in a quote. But because we'll have three separate quotes in a row, it'll be *relatively* obvious (not captain-obvious obvious, but still), that the middle one is a reply to the previous quote. What do you think..?

I was thinking of other solutions, like checking whether a tag is something like '[/quote author=Nao]', which in this case would mean "it's an opening quote where the / was added by mistake" but I don't really think it's a realistic case.

Now for another test case...


Using the pseudo-code from above, this would be really messy -- an opening quote would be added just before the closer, and then the s and b tags would remain opened until the end of the post, where they would then be closed forcibly.
So in this case, I think it's best that when we look through the latest opened tag in the stack, and it's not our closer, we simply add the related closer automatically and then keep going through the stack in reverse order, closing tags as required, until we find ours (or not). This is actually pretty much what my code is doing right now, as opposed to the pseudo-code in the first example.

Now, if we mix our two examples...


The closer quote will trigger a search for the last closed tag, which in this case is another closer quote, BUT between them it will find two unclosed tags... Which gets confusing, so it's best to first close any opened tags, and THEN go through the search for a place to add a new quote opener.

It's all very 'amusing' because I have to maintain a parallel stack of tag positions and the code has currently jumped from 15 lines to 50, which caused me to write this post in the hope that it'll allow me to sort things out... :lol:

Anyway, opinions welcome...
Hey, perhaps someone has heard of some BSD/MIT code available online that precisely does just that -- a flawless fix of all BBC or HTML tags left opened or closed... :P

Okay, I'll look into doing that tomorrow...

Still, complex errors won't be magically fixed, I'm afraid... Unless, unless I do a stricter check.
So while typing this post, it came to me that I could use a stack of tags and stack/unstack data and... Well, have a look at this code:


And... Here's the error message I'm getting following my latest rewrite. Which is actually SHORTER than the last one ;) It's not perfect but I'm working on it eheh.

Pretty cool uh?

PS: and yes, it works with tag nesting too, since it's a stack... i.e. if I have properly nested 'b' tags inside the quote, everything's fine.

PPS: the main issue with fixing tags in the middle of a message is that I would then have to find the exact position of the tag... I guess it's doable, though, but I'll have to go through a series of strpos etc or something to fill in the list first, so it'll definitely make the code bigger.