[Security] BBCode in SQL Database

CerealGuy

  • Posts: 318
[Security] BBCode in SQL Database
«  »Last edited
Why are the default bbcodes in the sql table {prefix}_bbcode? Wouldn't it be better to have those in the php code?
Also it's maybe a security issue to eval the validate_func from the sql database.
[Security] Re: BBCode in SQL Database
« Reply #1,  »Last edited
Bumping this, because this one bugs me a bit. I think this a security problem, even if it's low priority.
Why? Because if someone hijacked a database where a wedge instance is running on, he doesn't need too much more
to also undertake the php execution level. Which basically means he has many more possibilities to harm you.
Injecting some bad code into a often used bbcode validation, maybe changing the type of it and accessing a post where this bbcode gets translated into html (and bad code) is all what needs to be done.

It wouldn't be hard to make this ways less evil. We just need to get rid of those php code in the database and load it from a file. So adding three fields: validation_plugin, validation_file, validation_func would be all we need besides some code moving and a bit of adjusting.
Just to mention it again, php code in a database is not even handy. If there's an exception you won't know on which bbcode (in the moment), to debug it you have to update your database, to fix it you need to roll out a database update. This just makes no real sense in my opinion. And if you want to hook in with a plugin, you have to do hacky stuff (see my hidemod post on this topic).


Nao

  • Dadman with a boy
  • Posts: 15,932
[Security] Re: BBCode in SQL Database
« Reply #2,  »
I'll be honest with you, this is one of those features that I *saw* Pete adding, and I thought "oh my God this is horrible..."
Because really, what happens is that it just makes everything harder if you want to add smileys & such, and modify code 'easily'. You now have to go through phpMyAdmin to do your thing. I tried it, I wasted hours over those things. The only 'good' thing is that you can quietly change your smiley code without having to care about not overwriting them by uploading a source file. Well, the thing is, I don't see anyone but the *developer* changing those. And I'm the dev. So it's just a bit odd overall.

If anything, I like the idea of reverting this. The reason Pete did that was for plugins to be able to easily add smileys. Err, really? I've never seen a plugin do that. At least not with Wedge's level of public awareness. And SMF does well without that. Considering that I added SMF-like mod support back into Wedge a couple years ago (and in a more elegant way!), it should be trivial to add smileys just using the regular old SMF way.

PS: I just have no time to look into this right now. I'm pretty busy with the new site. Plus I may be moving servers again in the future-- this one is so blazing fast, I love it, but there are so many configuration issues, it gives me nightmares... -_-

CerealGuy

  • Posts: 318
[Security] Re: BBCode in SQL Database
« Reply #3,  »Last edited
Yup, totally aggree with your points. On the other hand, adding bbcodes via plugins is quite a handy thing. So i won't say the idea is bad, just the way it's done in the moment is not the best. Also this validate thing i really don't like. The name suspects a very limited function, even if it's not. Have to think of something for this.
I will give it a try and see what i can do. I'm ill in the moment (EB Virus is quite a nasty one) so I have some time.

OT:
Quote from Nao on February 7th, 12:58 AM
PS: I just have no time to look into this right now. I'm pretty busy with the new site. Plus I may be moving servers again in the future-- this one is so blazing fast, I love it, but there are so many configuration issues, it gives me nightmares... -_-
What kind of a server are you using that the configuration issues give you nightmares :hmm:

PS: It feels good to see wedge back under active developement :D

Nao

  • Dadman with a boy
  • Posts: 15,932
[Security] Re: BBCode in SQL Database
« Reply #4,  »
Quote from CerealGuy on February 7th, 11:58 AM
Yup, totally aggree with your points. On the other hand, adding bbcodes via plugins is quite a handy thing. So i won't say the idea is bad, just the way it's done in the moment is not the best.
Maybe do... BOTH?!
- Have the main BBCode in a source file,
- Give plugins the ability to add or replace BBCode,
- Only, plugins can't add code in the database, rather a function name. IIRC plugin files are always loaded so it's easy enough to just put the code in a plugin file and it should get executed.

Or, if it's too much to handle, just do points 2 and 3, because it still means putting the code to be executed into a source file, goddammit.
Yeah, that's probably the most realistic solution..?!
Quote
What kind of a server are you using that the configuration issues give you nightmares :hmm:
Well, I've moved Wedge.org and Noisen.com at the same time. Wedge.org was flawless, but Noisen.com is running a heavily customized SMF 2.0 RC, with none of the code fixes to make it run on PHP 7. However, that server can ONLY run one version of PHP (because the admin doesn't want to bother, and it's okay), so I had to manually convert SMF code to PHP 7 *and* my custom code as well, of course... Took at least a day.
If I move my site again, I'll make sure it has PHP 5 as an option. I'd rather use PHP 5 for Noisen, even though it's now working correctly in PHP 7.
Sorry if it was confusing. Wedge was fine.
I'm having Wedge problems with my other new site though, ahah. But again-- server problem. Another server, other problems: this time it's file permissions. The difference is, the admin has been AWOL for 2 months now. Not great...
Quote
PS: It feels good to see wedge back under active developement :D
Yeah, me too. And thank you for being the only developer who still believes in it. :)

Although I never felt back about leaving it -- I just considered it finished. What I'm doing is maintenance work, plus extra minor features from time to time because, well, it's fun?
My game development plans were set aside due to the state of the market. Purely from a business point of view, there just isn't enough visibility for a new game, and for now I don't want to rely on my 'Kyodai' brand name to attain more recognition. So I just focus on selling that old game and building that game trading site. (Which for now is just a game trade matching site. Supposed to be unveiled today, except the site isn't working due to aforementioned permissions ahah.)

CerealGuy

  • Posts: 318
[Security] Re: BBCode in SQL Database
« Reply #5,  »Last edited
Quote from Nao on February 9th, 11:26 AM
Quote from CerealGuy on February 7th, 11:58 AM
Yup, totally aggree with your points. On the other hand, adding bbcodes via plugins is quite a handy thing. So i won't say the idea is bad, just the way it's done in the moment is not the best.
Maybe do... BOTH?!
- Have the main BBCode in a source file,
- Give plugins the ability to add or replace BBCode,
- Only, plugins can't add code in the database, rather a function name. IIRC plugin files are always loaded so it's easy enough to just put the code in a plugin file and it should get executed.

Or, if it's too much to handle, just do points 2 and 3, because it still means putting the code to be executed into a source file, goddammit.
Yeah, that's probably the most realistic solution..?!
- First thing I want to do is moving all "validate_funcs" to
  Subs-BBC (nearly done with that).
- Next thing is refactoring those functions, making more
  clear what they do (figured out that the do nothing which
  you should describe with the term "validate", most of them
  trim or beautify the content.) and  understanding why the
  hell there are often the same tag twice with nearly the same
  "validate_func". I'm sure it would be enough to have one of
  those function for each tag (which needs this stuff).
https://github.com/C3realGuy/wedge/commit/791bdfa2632190b0c23f570e4bc581de9a7bc2bd#diff-ea931b81508cfb000ca2c54e529ff570R1520
- If this is done I want to look if it's possible to "hardcode" all those
  default bbc tags in a nice function. Something like loadBBCodes(),
  which returns all default bbc tags plus loads all others from sql.
  Have to see how the disable tag stuff is done. Want to add a
  hook here too, so plugins really have all the power over bbcodes
  without the need of modifying the bbc parse code or doing some
  regex stuff on top of it.

It's a bit to do, won't get bored the next days. The Problem is not
the coding, it's understanding what goes on in Subs-BBC :lol:
Quote from Nao on February 9th, 11:26 AM
Well, I've moved Wedge.org and Noisen.com at the same time. Wedge.org was flawless, but Noisen.com is running a heavily customized SMF 2.0 RC, with none of the code fixes to make it run on PHP 7. However, that server can ONLY run one version of PHP (because the admin doesn't want to bother, and it's okay), so I had to manually convert SMF code to PHP 7 *and* my custom code as well, of course... Took at least a day.
If I move my site again, I'll make sure it has PHP 5 as an option. I'd rather use PHP 5 for Noisen, even though it's now working correctly in PHP 7.
Sorry if it was confusing. Wedge was fine.
I'm having Wedge problems with my other new site though, ahah. But again-- server problem. Another server, other problems: this time it's file permissions. The difference is, the admin has been AWOL for 2 months now. Not great...
Didn't play with a pure webspace in years, and i know why :D You just have more control, more power and of course more responsibility. But stuff like let's encrypt (free ssl certificates in case you don't know them), full control about software which is running and the configuration will always let me choose an vps/kvm/whatever over a webspace.
Quote from Nao on February 9th, 11:26 AM
Quote
PS: It feels good to see wedge back under active developement:D
Yeah, me too. And thank you for being the only developer who still believes in it.:)

Although I never felt back about leaving it -- I just considered it finished. What I'm doing is maintenance work, plus extra minor features from time to time because, well, it's fun?
My game development plans were set aside due to the state of the market. Purely from a business point of view, there just isn't enough visibility for a new game, and for now I don't want to rely on my 'Kyodai' brand name to attain more recognition. So I just focus on selling that old game and building that game trading site. (Which for now is just a game trade matching site. Supposed to be unveiled today, except the site isn't working due to aforementioned permissions ahah.)
Wedge is quite finished. It's totally usable, I only tweak stuff which I don't like how it behaves. For example the WYSIWG Editor. It's fully working, but awful :D Will be the next thing i want to look at. And some features I see around other forum softwares which i like. For example mark quotes. So you mark a part of post and you can only quote this part. Really like this idea, have to see how i can implement that. Maybe stuff for a plugin.

Besides that, i said it already in parts here and there, wedge is in my opinion the only good forum software around.
It's nearly perfect with balancing out pure html/css functions and extra js. It has many js stuff which makes it feel modern and dynamic without making it unusable on browsers without js. I mean solutions like nodebb they won't work in a browser without js. Those "in browser apps" are cool for sure, but not for a forum software (in my opinion). They are slow, make problems with search engines, are unusable with noscript. But forums should be accessable by as many people as possible and shouldn't make problems.
The next thing is mobile/responsive design. A lot of forum softwares aren't able to this. Xenforo just looks really really bad on a smartphone. They didn't understand this *less thing which wedge does. Less padding and style elements is more on a mobile screen. You wan't to use as much of the screen as possible to show content without confusing the user. Xenforo also didn't do this js thing well. It's just bloated with js even if it's more of traditional "non ajax" solution compared to nodebb. I hate it to surf on any xenforo forum. Only thing which is maybe worse is an old vbulletin forum. I just always fear to lose my account's email and password to some hackers :D
Wedge feels lightweight, it loads super fast on the browser. It looks modern, has a special touch to it which you will recognize. And it has a lot of really good ideas implemented. Besides that, it's super stable. I think we use wedge on our forum since 3 years. Never had a serious problem with the forum software. Not even a small one.

But still there are problems with wedge.
- PHP is just not modern anymore. In fact, it's a bit dying.
  I'm not sure about that, but for example websockets aren't
  supported yet in php. Which could be a nice thing to have,
  also for wedge. Especially for notifications. Wedge is maybe
  the last and best php Forum Software.
- Besides the PHP thing, the codebase is old and unmaintained.
  No unit tests, no coding style standards. You also don't want to
  touch that code, risks are too high to break a thing and don't even
  notice it. Not a big of a problem, there aren't too many who use wedge
  anyway :D 
- No documentation, no helper/"framework like" functions to make life
  easier for plugin developers. As a plugin developer you write so many
  functions again, which are maybe already implemented in the wedge
  core and you just could reuse. But you don't know of them and never will.
- In general you can make wedge do everything you want, but it's often not
  a nice way. It's more like forging it with a flamethrower.
- Some standard functions are just not as good as they should. For example
  the editor. Color chaning is bad, you can't asign a choosen color to another
  marked text without first changing it's color to something else and than to
  the color you want. Makes no fun.
  Or Private Messages. It's this old private messaging system you know from
  Bulletin Boards. It's just old and feels bad to use. It's a bit like emails and this
  is already bad. In fact i would always prefer to write you a message on any
  instant messanger instead of using wedge.

Wow this is more of an essay and totally off topic. But who cares.
[Security] Re: BBCode in SQL Database
« Reply #6,  »
Back to topic,
hardcoding the bbcodes array is around 550 lines at least :^^;:
Of course faster than loading it everytime over sql, but still not nice.
Any ideas? :hmm:
[Security] Re: BBCode in SQL Database
« Reply #7,  »Last edited
So loading bbcodes from file works, language parsing works, additional loading bbcodes from sql works too.
The bbcodes as an array take 602 lines.

Only problem i have is thinking of a good name for the sql fields and the array field.
Currently the functions are called 'validate' functions, which is in my opinion not describing it well, because you can also modify the content and do more advanced stuff as only validating. Any ideas for a better name? @Nao

EDIT: No need to do anything for ManagePosts, works already how it should.

Nao

  • Dadman with a boy
  • Posts: 15,932
[Security] Re: BBCode in SQL Database
« Reply #8,  »
Quote from CerealGuy on February 9th, 01:06 PM
- First thing I want to do is moving all "validate_funcs" to
  Subs-BBC (nearly done with that).
Yeah, as long as it's not in a Load.php or Subs.php file that gets loaded even if no BBCode parsing takes place.
(Gee, I just remembered about the Shift+Enter feature in Wedge to cut off quoted posts... How cool. Why am I not using it more?!)
Quote from CerealGuy on February 9th, 01:06 PM
- If this is done I want to look if it's possible to "hardcode" all those
  default bbc tags in a nice function. Something like loadBBCodes(),
  which returns all default bbc tags plus loads all others from sql.
  Have to see how the disable tag stuff is done. Want to add a
  hook here too, so plugins really have all the power over bbcodes
  without the need of modifying the bbc parse code or doing some
  regex stuff on top of it.
Oh yeah, hooks, good idea... Maybe with hooks, there's really no need to use the database at all..?
Quote from CerealGuy on February 9th, 01:06 PM
It's a bit to do, won't get bored the next days. The Problem is not
the coding, it's understanding what goes on in Subs-BBC :lol:
If I'd seen that post earlier, I could have offered to do it for you, I still have a wide knowledge of what everything does.
(And I'm saying that just after discovering/remembering that SMF/Wedge use a set_error_handler() function to show errors, when I was implementing my own for a debug session lol...)
Quote from CerealGuy on February 9th, 01:06 PM
Didn't play with a pure webspace in years, and i know why :D You just have more control, more power and of course more responsibility. But stuff like let's encrypt (free ssl certificates in case you don't know them), full control about software which is running and the configuration will always let me choose an vps/kvm/whatever over a webspace.
You don't need a VPS to use Let's Encrypt. In fact it's starting to become a generic feature in many shared hosting environments. IIRC you don't even need a unique IP to use a SSL certificate.
My hosting is a VPS, BTW, not shared hosting. My only shared hosting is for cynagames.com, and even then I'm considering moving my site somewhere else because it's getting really expensive for a SINGLE mostly static site with barely any visitors... Hmmm. But my current hosting being PHP7 only, it would take days to convert cynagames to use it, since it's running an old SMF 1.1!
Quote from CerealGuy on February 9th, 01:06 PM
Wedge is quite finished. It's totally usable, I only tweak stuff which I don't like how it behaves. For example the WYSIWG Editor. It's fully working, but awful :D
Well, if it were just up to me, I'd remove it entirely, I never use WYSIWYG. And I'm pretty sure 95% of all SMF/Wedge users don't, either.
Quote from CerealGuy on February 9th, 01:06 PM
Will be the next thing i want to look at. And some features I see around other forum softwares which i like. For example mark quotes. So you mark a part of post and you can only quote this part. Really like this idea, have to see how i can implement that. Maybe stuff for a plugin.
Interesting. I don't know if it's a feature people would use, though.
Quote from CerealGuy on February 9th, 01:06 PM
Besides that, i said it already in parts here and there, wedge is in my opinion the only good forum software around.
You probably haven't tried them all ;)
Maybe XenForo 2 is better, I don't know...
I know ElkArte has its fans, too. (Plus its authors are eminently respectable, so that helps!)
I'm not a big fan of their design though. I don't like Wedge's either, but Wilde is very tweakable and close to something both usable and good-looking.
Quote from CerealGuy on February 9th, 01:06 PM
It's nearly perfect with balancing out pure html/css functions and extra js. It has many js stuff which makes it feel modern and dynamic without making it unusable on browsers without js. I mean solutions like nodebb they won't work in a browser without js.
Yes, that's true. Anything that tries to reinvent forums is a failure in my opinion. I understand their point, not the execution.
I have a weakness for threaded forums, though. My main concern with that system is how new posts are presented to the user who already dead a previous version of the topic. At one point I wanted to have a hybrid solution, where updated topics are shown flat, and you can view posts "in context" if you'd like. Pretty much what I did for Thoughts...! Also, Wedge internally supports parent/children for posts, you could build a skin that shows posts in threaded mode.
Quote from CerealGuy on February 9th, 01:06 PM
The next thing is mobile/responsive design. A lot of forum softwares aren't able to this. Xenforo just looks really really bad on a smartphone. They didn't understand this *less thing which wedge does. Less padding and style elements is more on a mobile screen.
Well, I don't think Wedge is fantastic on mobile either, but yeah I'll agree with you on one point, it's the best mobile skin I've ever used on my phone. (IIRC it also had the shift-enter split by default for practical reasons... ^^)
Quote from CerealGuy on February 9th, 01:06 PM
Wedge feels lightweight, it loads super fast on the browser.
Because I spent so much time optimizing it... ;)
Then again, with modern browsers, modern servers and modern web software, it doesn't feel like much of a difference anymore.
Quote from CerealGuy on February 9th, 01:06 PM
It looks modern, has a special touch to it which you will recognize. And it has a lot of really good ideas implemented. Besides that, it's super stable. I think we use wedge on our forum since 3 years. Never had a serious problem with the forum software. Not even a small one.
I must be the only one, then... ;)
Or generally people who use weird setups like Nginx + Apache.
Quote from CerealGuy on February 9th, 01:06 PM
- PHP is just not modern anymore. In fact, it's a bit dying.
With PHP7 increasing performance ten-fold, I don't think it's dying anytime soon...
Quote from CerealGuy on February 9th, 01:06 PM
I'm not sure about that, but for example websockets aren't
  supported yet in php.
There are libraries like Ratchet. But websockets are more of a JS thing, aren't they..?
Quote from CerealGuy on February 9th, 01:06 PM
Which could be a nice thing to have,
  also for wedge. Especially for notifications. Wedge is maybe
  the last and best php Forum Software.
Well, for notifications I agree websockets are a good idea, but Wedge's implementation is already good enough-- it loads as little data as possible and pauses itself when needed. I could add some code to send the notifications through Chrome's own notification system, though.
Quote from CerealGuy on February 9th, 01:06 PM
- Besides the PHP thing, the codebase is old and unmaintained.
  No unit tests, no coding style standards. You also don't want to
  touch that code, risks are too high to break a thing and don't even
  notice it. Not a big of a problem, there aren't too many who use wedge
  anyway :D
Whenever I see code I don't understand, I'll just use git blame on the file, and find out what commit added it, and the related commit notes. That's often how I figure out something was improperly added.
Quote from CerealGuy on February 9th, 01:06 PM
- No documentation, no helper/"framework like" functions to make life
  easier for plugin developers.
Well, I never wanted to document the whole thing, I hate writing docs... Pete was supposed to do it, then someone else, been there done that... :-/
Quote from CerealGuy on February 9th, 01:06 PM
As a plugin developer you write so many
  functions again, which are maybe already implemented in the wedge
  core and you just could reuse. But you don't know of them and never will.
Agreed. Can't help much here.
Same can be said for SMF, though. Didn't stop it from being successful and having tons of mods. (Most of them being shit, though. Not all mods were of Aeva Media quality :P)
Quote from CerealGuy on February 9th, 01:06 PM
- Some standard functions are just not as good as they should. For example
  the editor. Color chaning is bad, you can't asign a choosen color to another
  marked text without first changing it's color to something else and than to
  the color you want. Makes no fun.
Quite honestly, I should remove that kind of 'feature'... Even from non-WYSIWYG... Leave the bbcode in, but just don't advertise it. Let's just say it was just a good excuse to make use of sbox.js's advanced features!
Quote from CerealGuy on February 9th, 01:06 PM
Or Private Messages. It's this old private messaging system you know from
  Bulletin Boards. It's just old and feels bad to use. It's a bit like emails and this
  is already bad. In fact i would always prefer to write you a message on any
  instant messanger instead of using wedge.
Yeah, you know what, I actually considered that for my new site, and was wondering if I should just replace it with a pseudo-chat system between 2 users... Something similar to Facebook, really.
Could be a path to explore.
Only problem is: how exactly do you 'import' the old PMs into that system... ;)
Also, it would probably encourage people to post more 'PMs', thus more entries in the PM base, thus slower to use... (?)
Quote from CerealGuy on February 9th, 01:06 PM
Wow this is more of an essay and totally off topic. But who cares.
Yeah, it's not like there's too many people around ;)
[Security] Re: BBCode in SQL Database
« Reply #9,  »
Quote from CerealGuy on February 11th, 06:43 PM
So loading bbcodes from file works, language parsing works, additional loading bbcodes from sql works too.
The bbcodes as an array take 602 lines.
Did you simply turn it back into something similar to SMF's version?
Because it'd be best, I suppose, for plugin authors to 'understand' the thing...
Quote from CerealGuy on February 11th, 06:43 PM
Only problem i have is thinking of a good name for the sql fields and the array field.
Currently the functions are called 'validate' functions, which is in my opinion not describing it well, because you can also modify the content and do more advanced stuff as only validating. Any ideas for a better name? @Nao
Sure.
preprocess for pre-validation
postprocess for validation
Or just 'process' if you want to keep it simple.

CerealGuy

  • Posts: 318
[Security] Re: BBCode in SQL Database
« Reply #10,  »
Quote from Nao on February 11th, 11:02 PM
Yeah, as long as it's not in a Load.php or Subs.php file that gets loaded even if no BBCode parsing takes place.
(Gee, I just remembered about the Shift+Enter feature in Wedge to cut off quoted posts... How cool. Why am I not using it more?!)
Nope, everything is in Subs-BBC.php. This Shift-Enter thing is nice, you should tell more people about that :D
Quote from Nao on February 11th, 11:02 PM
Oh yeah, hooks, good idea... Maybe with hooks, there's really no need to use the database at all..?
There wouldn't be a need for it, but I would keep the database stuff. I like it that you can add bbcodes over plugin-info.xml. Wouldn't be possible without loading bbcodes from database. Still I would add hooks, just to let people easily do more advanced stuff like modifiying existing bbcodes.
Quote from Nao on February 11th, 11:02 PM
If I'd seen that post earlier, I could have offered to do it for you, I still have a wide knowledge of what everything does.
(And I'm saying that just after discovering/remembering that SMF/Wedge use a set_error_handler() function to show errors, when I was implementing my own for a debug session lol...)
If you still know what's the difference between the 'unparsed_equals', 'unparsed_commas', 'unparsed_commas_content', 'unparsed_equals_content', 'parsed_equals' bbc types this would be very helpful :D
EDIT: The SMF Sources are helpful: https://github.com/SimpleMachines/SMF2.1/blob/release-2.1/Sources/Subs.php
Quote from Nao on February 11th, 11:02 PM
You don't need a VPS to use Let's Encrypt. In fact it's starting to become a generic feature in many shared hosting environments. IIRC you don't even need a unique IP to use a SSL certificate.
My hosting is a VPS, BTW, not shared hosting. My only shared hosting is for cynagames.com, and even then I'm considering moving my site somewhere else because it's getting really expensive for a SINGLE mostly static site with barely any visitors... Hmmm. But my current hosting being PHP7 only, it would take days to convert cynagames to use it, since it's running an old SMF 1.1!
When we're already talking about SSL, what about ssl for wedge.org?
Quote from Nao on February 11th, 11:02 PM
Well, if it were just up to me, I'd remove it entirely, I never use WYSIWYG. And I'm pretty sure 95% of all SMF/Wedge users don't, either.
Yup, remove it and only let the preview thing there. Maybe split the editor in two tabs, one for modifying and one for preview. Like the github editor.
Quote from Nao on February 11th, 11:02 PM
Interesting. I don't know if it's a feature people would use, though.
I will give it a try, people who quote a lot will use it. I just like quotes for off topic or threads with many questions/answers. Like this one :D
Quote from Nao on February 11th, 11:02 PM
You probably haven't tried them all ;)
Maybe XenForo 2 is better, I don't know...
I know ElkArte has its fans, too. (Plus its authors are eminently respectable, so that helps!)
I'm not a big fan of their design though. I don't like Wedge's either, but Wilde is very tweakable and close to something both usable and good-looking.
ElkArte looks like a good SMF fork. But I prefer the look and feel of wedge. They seem to have a nice codebase. Tests and stuff :D
Quote from Nao on February 11th, 11:02 PM
Because I spent so much time optimizing it... ;)
Then again, with modern browsers, modern servers and modern web software, it doesn't feel like much of a difference anymore.
Quite well optimized :cool:
Quote from Nao on February 11th, 11:02 PM
I must be the only one, then... ;)
Or generally people who use weird setups like Nginx + Apache.
nginx + latest php + mariadb. Works like a charm.
Quote from Nao on February 11th, 11:02 PM
With PHP7 increasing performance ten-fold, I don't think it's dying anytime soon...
It will exist for a long time, too many projects which rely on php are still heavily used.
Quote from Nao on February 11th, 11:02 PM
Quote from CerealGuy on February 9th, 01:06 PM
I'm not sure about that, but for example websockets aren't
  supported yet in php.
There are libraries like Ratchet. But websockets are more of a JS thing, aren't they..?
They are, but as soon as we use JS, we could use all the JS features, even the new fancy ones. For a chat system for example :D
Quote from Nao on February 11th, 11:02 PM
Quote from CerealGuy on February 9th, 01:06 PM
Which could be a nice thing to have,
  also for wedge. Especially for notifications. Wedge is maybe
  the last and best php Forum Software.
Well, for notifications I agree websockets are a good idea, but Wedge's implementation is already good enough-- it loads as little data as possible and pauses itself when needed. I could add some code to send the notifications through Chrome's own notification system, though.
Already thought about that notification system, maybe would be a nice thing, especially with a better PM System :P
Quote from Nao on February 11th, 11:02 PM
Whenever I see code I don't understand, I'll just use git blame on the file, and find out what commit added it, and the related commit notes. That's often how I figure out something was improperly added.
That's a good idea!
Quote from Nao on February 11th, 11:02 PM
Quote from CerealGuy on February 9th, 01:06 PM
- No documentation, no helper/"framework like" functions to make life
  easier for plugin developers.
Well, I never wanted to document the whole thing, I hate writing docs... Pete was supposed to do it, then someone else, been there done that... :-/
Maybe something like js-doc for php would already be enough with generated html docs. Many functions and classes are documented inside the code.
Quote from Nao on February 11th, 11:02 PM
Agreed. Can't help much here.
Same can be said for SMF, though. Didn't stop it from being successful and having tons of mods. (Most of them being shit, though. Not all mods were of Aeva Media quality :P)
I think this we can solve quite easily if we give wedge "framework like" functions. Every now and then someone needs a function like this in a plugin, we can simply add it to special core/app/file. Problem is of course that we have to document them somewhere, or at least in the code :D
Quote from Nao on February 11th, 11:02 PM
Quote from CerealGuy on February 9th, 01:06 PM
Leave the bbcode in, but just don't advertise it. Let's just say it was just a good excuse to make use of sbox.js's advanced features!
Maybe this would be an idea. For real WYSIWYG wedge would need a full bbc parser in js... Or a basic bbc parser which lets the server render the html if it's a more complex bbcode.
Quote from Nao on February 11th, 11:02 PM
Quote from CerealGuy on February 9th, 01:06 PM
Yeah, you know what, I actually considered that for my new site, and was wondering if I should just replace it with a pseudo-chat system between 2 users... Something similar to Facebook, really.
Could be a path to explore.
Only problem is: how exactly do you 'import' the old PMs into that system... ;)
Also, it would probably encourage people to post more 'PMs', thus more entries in the PM base, thus slower to use... (?)
I don't really have an idea, but definetly something more like a chat, even if it's pseudo and without ajax features.
But would be a neat thing. Maybe we can just reuse the database structure.
Quote from Nao on February 11th, 11:04 PM
Did you simply turn it back into something similar to SMF's version?
Because it'd be best, I suppose, for plugin authors to 'understand' the thing...
It's actually quite similiar to the SMF approach. I think the only difference was that wedge loaded it's bbcodes from db, smf has them hardcoded. We will now do both :D
Quote from Nao on February 11th, 11:04 PM
Sure.
preprocess for pre-validation
postprocess for validation
Or just 'process' if you want to keep it simple.
Thanks, process should hit the nail.
[Security] Re: BBCode in SQL Database
« Reply #11,  »Last edited
Umm... actually I'm more or less done with my changes. Problem is now, that the validate_func stuff never got called for some bbc types. And for some other it's not even implemented. Maybe this made sense at some time, but now it doesn't anymore. Have to see how SMF does this.

Changes:
https://github.com/C3realGuy/wedge/commits/dev_move_bbc_validate_to_disk
New syntax for bbcode in plugin-info.xml:
Code: [Select]
  <bbcodes>
    <bbcode tag="md" type="unparsed_commas" block-level="no">
      ....
      <process file="src/BBC-Markdown" function="bbc_process_markdown" />
    </bbcode>
  </bbcodes>

Oh and Elkarte rewrote the complete BBC parser. https://github.com/elkarte/Elkarte/wiki/New-BBC-Parser
I will see what they did, maybe we can reuse some of their work? Because the SMF thing is really ugly and unmaintable. You can fix it but you will never be happy with it  :lol:.
[Security] Re: BBCode in SQL Database
« Reply #12,  »Last edited
A little progess which is worth to mention in my opinion. We can now use the process function on all bbc types :cool:
and i wrote a small plugin to show you what this can now do.

https://github.com/C3realGuy/markdown (Don't use the plugin in production environments, markdown-extra is evil ;))

It's just hacked together, but still works quite well. Besides this autolink stuff wedge does (inserting random http://tags on everything which looks like a url to something) :D

Nao

  • Dadman with a boy
  • Posts: 15,932
[Security] Re: BBCode in SQL Database
« Reply #13,  »
Quote from CerealGuy on February 12th, 10:21 AM
Nope, everything is in Subs-BBC.php. This Shift-Enter thing is nice, you should tell more people about that :D
I know right... Well, when I introduced it, it was pretty popular in Wedge. I was even a bit scared that the feature would end up in SMF... Yeah, I guess I overestimated them. :P
One of the downsides with the auto-splitter, is that if there's a newline after the point where you're splitting, Wedge won't notice it, and will insert an opening quote tag, then that newline, then the original text. It looks ugly. I've gotten used to just editing those out, but honestly it'd be best to add some code at the beginning of splitQuote() to look for space/newline characters immediately before and after the split point, select them together, and remove them, then adjust the starting position. THEN it'd be the perfect splitter. Or maybe you have a simpler idea..?
Quote from CerealGuy on February 12th, 10:21 AM
There wouldn't be a need for it, but I would keep the database stuff. I like it that you can add bbcodes over plugin-info.xml.
Ah, yes indeed. It's just that I like simplifying the database as much as possible, like removing extra tables...
Quote from CerealGuy on February 12th, 10:21 AM
Wouldn't be possible without loading bbcodes from database.
Are you positive about that?
After all, an XML file can also contain function declarations...
Quote from CerealGuy on February 12th, 10:21 AM
If you still know what's the difference between the 'unparsed_equals', 'unparsed_commas', 'unparsed_commas_content', 'unparsed_equals_content', 'parsed_equals' bbc types this would be very helpful :D
Well, I could explain it, but I won't bother, as the SMF documentation (I think?) did it well... But Pete removed it in his commit to move BBCode to the database (December 3, 2010 -- quite soon after we started work on Wedge.)

(edit: removed block, since you've already seen it.)

Unparsed content just means that the contents won't be parsed by parse_bbc(), like in... Well, the code tag?
IIRC, one of the things I added to the system is the ability to have multiple (optional) parameters.Oh gee, looks like you found it by yourself. Sorry for wasting your time (and mine!) ;)
Quote from CerealGuy on February 12th, 10:21 AM
When we're already talking about SSL, what about ssl for wedge.org?
Well it's supported already, it's just not the 'default'.
The reason why it's not default is that Wedge uses http-based avatars, which means browsers showing an HTTPS page will consider it 'insecure' because there's an HTTP-based image embedded in the HTML.
The solution, of course, is to 'simply' fix all local avatar links to the correct protocol, but Wedge stores the URL in multiple places, so that's a bit annoying, unless we change it directly in the database, but that means if you switch back to HTTP (e.g. expired certificate), you're likely to get empty images (expired cert + image link = no image at all, because the browser won't trust it until it's approved manually, and since it won't show a popup for a simple image, you're screwed.)
Also, doesn't help with external avatars. There's no way to know if they're compatible with HTTPS.

And I'm pretty sure HTTPS fans would want that address bar icon to be green, not gray...
Quote from CerealGuy on February 12th, 10:21 AM
Yup, remove it and only let the preview thing there. Maybe split the editor in two tabs, one for modifying and one for preview. Like the github editor.
Yeah, I wouldn't know about removing it... But it's certainly worth posting a poll. Only, on this site, we wouldn't be getting many answers... Probably likelier to get proper answers at sm.org, of course, but I stopped going there years ago.
Quote from CerealGuy on February 12th, 10:21 AM
I will give it a try, people who quote a lot will use it. I just like quotes for off topic or threads with many questions/answers. Like this one :D
But you're talking about preventing people to quote parts of your message, no?
Maybe there's a misunderstanding.
Were you instead talking about a multi-quote feature? Like the one that's been in Invision Power Board literally since forever..?
That would also imply that topics ARE flat. My own 'implementation' of the thing is the soft-merging of posts, so that multiple answers don't take more space, and yet if you click Quote on a post, your reply is automatically threaded below that post, even if it doesn't show on the default flat skins.
Quote from CerealGuy on February 12th, 10:21 AM
ElkArte looks like a good SMF fork. But I prefer the look and feel of wedge. They seem to have a nice codebase. Tests and stuff :D
Yeah, I'd tend to say Elk is made by hardened professionals, and Wedge by enlightened amateurs.
The fact that they've been at work on it for the last 5 years is impressive, too. When they started, I doubted they'd 'last' for 5 years. In the end I worked fulltime on Wedge for 5 years, and they did too.
Personally though, I'd hate being restricted by test suites when it comes to adding new features. These aren't even a guarantee your feature will work in every situation. I prefer to rely on beta testers.
Quote from CerealGuy on February 12th, 10:21 AM
nginx + latest php + mariadb. Works like a charm.
Nginx is the php server too? Did you convert your htaccess to use Nginx too?
Quote from CerealGuy on February 12th, 10:21 AM
They are, but as soon as we use JS, we could use all the JS features, even the new fancy ones. For a chat system for example :D
Ahah.
I don't know. I'm not used to using a nuclear bomb to break a window.
Quote from CerealGuy on February 12th, 10:21 AM
Maybe something like js-doc for php would already be enough with generated html docs. Many functions and classes are documented inside the code.
Yeah. There's 'something' called phpdoc, I think.
Actually, that was the idea behind the comment refactoring that Pete did for a while. He wanted to use a tool to later automatize the extraction of function descriptions. I wasn't comfortable with that tool, so I just left him to his devices, unfortunately he never finished it. But he did a good job at what he did. (Basically, he commented most of what matters...)
Quote from CerealGuy on February 12th, 10:21 AM
Maybe this would be an idea. For real WYSIWYG wedge would need a full bbc parser in js... Or a basic bbc parser which lets the server render the html if it's a more complex bbcode.
A full bbc parser in JS..? But how is it different than using Ajax to parse said bbc?
Quote from CerealGuy on February 12th, 10:21 AM
I don't really have an idea, but definetly something more like a chat, even if it's pseudo and without ajax features.
But would be a neat thing. Maybe we can just reuse the database structure.
Yeah, I looked into it, and:
- there are a few columns that'd be useless, like the subject one. Not a big problem.
- PMs don't have a recipient ID assigned to them, instead it's done through an extra table that can hold multiple people as recipients. While it's a good idea to make it more flexible, it also makes it harder to sort PMs by conversation. How do we 'recognize' that a specific conversation should be treated separately? Maybe by having some sort of id_conversation toggle, I don't know. It's a possibility, just makes it harder. Then again, a multi-user chat message, aka a chat room, sounds good to me...

This is definitely the next refactoring work I'll be doing, as soon as I'm done with the new site (Lestrade's, if you're curious! Although, if you dont have a Steam account, it'll be quite useless to you ;))
[Security] Re: BBCode in SQL Database
« Reply #14,  »
Quote from CerealGuy on February 12th, 07:16 PM
A little progess which is worth to mention in my opinion. We can now use the process function on all bbc types :cool:
and i wrote a small plugin to show you what this can now do.
A markdown parser? Nice... :)
I'm sure it could be useful.
I don't think you have to use the CDATA block around such short code, though!
Although now I understand your point about leaving BBCode support in the database.
Quote from CerealGuy on February 12th, 07:16 PM
It's just hacked together, but still works quite well. Besides this autolink stuff wedge does (inserting random http://tags on everything which looks like a url to something) :D
'unparsed'? ;)