Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Arantor
166
Features / Re: Soft merging of posts
« on July 26th, 2013, 05:14 AM »
That's precisely my point. SO has a simple enough interface and yet it's screwed up, especially by theoretically technically minded users. Now multiply that by the lowest common denominator of forum posters and boom. Even FB gets badly messed up by it with its very limited notions of it.

It's not so much the whole thing of being free vs not free, it's a user (mod author) usability issue. The more things people can do, the more complex troubleshooting becomes - bear in mind we will not only be supporting the core code but also supporting plugin authors trying to do things too.
167
The Pub / Re: Posting from mobile, including Aeva
« on July 26th, 2013, 05:10 AM »
1. I'd never done it before, but it is surprisingly hard to do. I'd note that doing it on a regular basis is a niche thing, but it does sort of screw up doing a travel blog ;) Ideally I'd just like it all to be as slick as possible for users and accommodating such things if feasible.

2. Cool. I thought it probably was done but wanted to be sure.

3. Adding items. Albums is complicated in general and I wouldn't really want to do that on a mobile in any situation simply because there's a lot to it, just like I wouldn't really want to create a board on the fly for the same reason. But when posting items, I find myself not bothering with half the stuff, it's only really the title and description we change, when we bother to do that at all. I'm thinking kind of like the normal posting interface, really... half the options in there are hidden by default.

4. I remember our original plans and to be honest I think it's something that we'll just have to bite the bullet with and deal with it, sooner rather than later. There's a ton of changes I've been thinking about doing while I've been out here on the road, some for media, some for other stuff and a lot of it will factor into this stuff - and I think once I get back home (12 Aug), I will have accrued enough done on other stuff to be able to spend ages on Wedge, and more importantly the motivation to nail a lot of this stuff good and proper. I actually believe that it's doable and that while it won't be pretty to start with, it will work pretty well in the long run :)

5. A lot of the issue around the bbcode entry stuff ultimately stems from what I kind of come to consider as poor workflow. If we provide users the ability to browse media from the posting page, and more importantly the ability to bring in content from there (e.g. from a mass upload or similar), that stops being a problem for the most part and we can make that super easy for users to do.

6. That's the bulk of the problem. If there wasn't the zooming-on-focus, there wouldn't be a problem.

As far as disabling zooming on AJAX, I'm really not sure that's a huge deal. Most of the time we're not adding inputs or textareas via AJAX and the few times we are we can probably account for it just on those instances.
168
Other software / Re: SM.org compromised
« on July 25th, 2013, 10:00 AM »
Just to add, I posted this to the end of the debate where people are talking about making the hashes harder.
Quote
Much as I hate to get involved, someone who I owed a favour to asked me to comment.

Firstly, you're incorrect about some of your assertions. The salt you're referring to is not used to compute the password hash. The password in the database is stored as SHA1(strtolower(username) . password) and has been for a very long time in SMF. No extra table is required for this. Oh, and if you notice, it's not actually 'rolling their own'. It's following standard practice (take the password, add something that's account specific and hash that)... and in ANY case the salt would be useless if it were used because it's RIGHT THERE IN THE TABLE. It's only any use if it's otherwise an unknown, but since it's not an unknown element, because it's right there in the table, it doesn't really help any since you already have to construct a rainbow table for every account in the first place.

Secondly, not all hosts are currently using 5.3.7 and up. There are still some very major hosts running 5.2 and in fact some hosts still offering 4.4 hosting for the time being, even though 5.3 itself is officially maintenance only now, and most of SMF's typical demographic applies to those because they're the budget end of the market. So that's not really something SMF can implement in a realistic fashion, but I guess reality of lots of support requests (and there are already multiple requests per day related to one really poor free host and their limitations, but hey, let's add a crap ton more!) is irrelevant.

Thirdly, I don't believe you understand what it means by 'decrypting the password'. It is mathematically impossible to actually decrypt the password hashed by a digest hash like SHA1. What you do is find something that when combined with the username and then hashed will give you the same hash as what you're looking for... that means potentially multiple passwords will actually work because you're not looking to find the original, you're merely looking for a collision. You may want to search for multiple collisions for this very reason.

Fourthly, it is well researched and documented that most people suck at choosing passwords. A 2011 study (whose link I can't immediately find) found that 'password' was still by far the most common password, followed by '123456' and direct variations of that. Breaking into systems can be done with that fairly easily.

Incidentally, pushing everything to bcrypt may not be the smartest idea in the world. If everything is pushed to bcrypt and a *single* vulnerability is found, suddenly a lot more targets are actually located. Having everything in a slightly different form does at least have some benefits with respect to lowering the immediate attack vector options. There is a little comfort - but very certainly not a lot - from security through obscurity, but it is no substitute by any means for real security.

Thing to note: the system is only ever as strong as the people who hold the keys to it. It is often significantly easier to leverage a weakness in a person rather than in the technology under them, as was done here, to leverage the access to perform queries on the database. Once the database is compromised, best security practice is to assume it is *always completely compromised* even if the data is encrypted with the strongest possible methods, because you never know what resources the intruder has to break that.

Consider: this attack was to force an admin's account. If there was no way to run arbitrary code through some fashion with an admin's account or to pull a database backup of some kind, there would be less risk. In fact, if there is no method by which to run arbitrary code or pull a database backup without server access directly, you need to do more than brute force that account, and proceed to brute force a server account. If someone's already gotten into the server itself, assume everything is compromised anyway because by definition it is.

The whole thing about hashes is only an issue if someone gets into the server or otherwise can obtain raw access to the tables and right now there are ways by which an admin can do just that, either by splicing code into template files, or by uploading a package. Stronger controls need to be added to these to prevent arbitrary code being able to be run with just an admin's account, for example enforcing upload via FTP of plugins (as other software does), as well as limiting the ability to run arbitrary code from the admin panel by removing the ability to edit files from said admin panel and enforcing use of either hosting control panel or FTP download/local editing.

I see where you're going but honestly you're picking on one of the stronger links in the chain. There are far, far many more issues to consider... like how on a number of shared hosting setups it is actually possible for your setup to be hijacked as soon as you install a mod. Or how in a number of shared hosting setups it's possible for your entire database to be compromised and with you not being able to do *anything* about it whatsoever. But of course these are minor considerations when compared to worrying about what will happen when a hacker gets in... I'd personally prefer to keep them out in the first place where possible, rather than tightening up what happens when they already have. (Not that you shouldn't ALSO do that, but in terms of priority, it's really a secondary consideration. Don't worry about locking up the family silver if the front door is already unlocked.)
You'll note that at least some of those things I talk about are already done in Wedge ;) There is no backup facility and plugins have to be uploaded via FTP (even if Wedge does it for you, it's still doing it via FTP and inheriting permissions while doing so, there's no point where files are made writable)

On this matter, I can make the case for upgrading to bcrypt just as I can (and did) make the case against it. I'm not averse to upgrading the password security at all, and I do believe the benefits outweigh the cons here, but there are more changes to the architecture that I want to make to support this. I don't want to discuss my changes at this time because I don't believe it is relevant to this discussion and I'd really rather not divert this discussion too far.
169
Test board / Re: this is test message
« on July 25th, 2013, 09:57 AM »
Just to clarify, hard merge is still done on non-admin users who post multiple posts too quickly (IIRC within 5 minutes)
170
The Pub / Posting from mobile, including Aeva
« on July 25th, 2013, 09:30 AM »
OK, so as you guys know, I'm on this crazy trip across the US and we've been logging our stuff on our little website which is based on SMF + Aeva and a custom theme.

There are some interesting things we've found from doing this that I want to share, and since the Wedge version of Aeva is still basically Aeva from a user standpoint, I think a lot of that stuff stands. Note that this is SMF without a mobile theme and not using WAP because there are other things tied into the site's posting page that aren't in WAP and I didn't have time (or energy) to code them up multiple times. (Namely, the site has a topic tagging system and also a facility to geotag individual posts with a location. This requires various splices into the posting setup.)

1) Posting, generally, sucks from a mobile phone. It actually really sucks on an iPhone 4S, I'm actually surprised more people don't complain about how bad the experience is. Even with the phone in landscape, getting the subject and then the text in is painful. We don't really need formatting much in mobile, even the bold etc. is probably too exotic for the most part but perhaps a collapsed menu of some kind to bring up the tags would be OK. Ideally though I'd consider doing what Tapatalk does: the post interface for a new topic is little more than two text boxes. (Of course I need other stuff but that's another matter entirely and the specific requirements we have for Crossing Overland will actually necessitate a custom app to support background data being gathered and batch uploaded to the site later on.)

2) Does Wedge's version of Aeva support spitting out the non Flash version of embed code where we have it? I'm thinking YouTube in particular. (I can't exactly do a hotpatch on the road to splice the youtu.be code in, but there's at least one video on CO that uses the standard embed code.)

3) Media uploading can typically be much, much simpler than what we currently have. Sure, most of those options will be useful for some people but for the most part, especially when uploading from mobile, it's just not necessary. Perhaps a collapsible menu for the stuff like album covers and the extended information.

4) We really need a mass upload option of some kind (even just AJAX uploading a la Dragooon's plugin for regular posting) that works without Flash so that uploading from mobile is not a hideous chore. It's gotten to the point where we've broken out the one laptop we did actually bring to handle mass uploading, but that's a 2006 MacBook and doesn't exactly handle it *well*. (In unrelated news, the camera connector kit for iPad works really well)... actually, AJAX uploading for media would truly rule just to have it anyway, mobile or not.

5) The selector bit for media items where you can copy the bbcode is not really copyable on a mobile device and for reasons I can't entirely figure out, the item doesn't usually come up with anything useful in there other than the id number.

For example, we uploaded a picture, and the item in question just had:
Code: [Select]
[smg id=216]

as the text. Which would normally be fine but we wanted:
Code: [Select]
[smg id=216 type=preview align=center caption="First sight of the Pacific"]

or similar and this is a real pain to add on mobile if you want kind of nice formatting on blog posts. We find ourselves copy/pasting all the time to make this work... I don't know about on desktop yet but on mobile at least it would probably be neat to have the media button not just go straight to an upload button but instead to a media browser from which you can upload if you want, and then have the media browser cover the gallery contents. This would mean we could just upload everything, then filch it from the gallery easily, or upload one at a time without any real fuss either (since the popup could handle the upload AJAXively which would be slick even on an iPad). Fairly sure I've mentioned this before but IPB actually does it this way, and thinking about it, it would also make mass upload much nicer for posting images to the site and then pulling that into the post and that would greatly smooth off our workflow - and I'm fairly sure it would for others too.

Aeva works truly magnificently as a storage ground for media items but I gotta say, the workflow of getting items into posts isn't ideal and it gets worse with the more items you add and until recently I'd pretty much used it as a storage ground, rather than making it part of a cohesive blog structure.

6) Can we actually do something with the textbox sizing and positioning? What we've found with an iPhone (it's largely a non issue with iPad) is that there is some crazy resizing crap going on that means you tap to select the textbox and then it zooms in to the point where the textbox is larger than the screen. Now, I'm sure this is at least partly no longer an issue because it's using SMF 2 code rather than our code and I can't really test it on Wedge right now to be sure, but the way it does zooming on the fixed-width container is really frustrating since it means to type in a post you have to swipe left and right to actually see everything, and even composing the line-blank-line-blank-line posts I've been doing for the blog for the live video updates (yes that's me doing it on Louis' iPhone while he's driving) is frustrating. Again I'm thinking like Tapatalk here, where the box is sized for the screen and it all fits without any scrolling horizontally or any such nonsense. I would kind of presume @Dragooon's mobile theme for SMF works nicely in this regard but again right now I can't really test it. I should note this is the first time I've *really* tried to use an SMF setup for posting and even though I know we have a slightly weird theme, I can't be the only person who finds the experience of posting even 3 lines of text with 2 blank lines tiresome.


I really don't know how much of this stuff can be fixed by way of a mobile skin. Seems to me there is far more at stake than simple CSS resizing (e.g. pulling a different editor component for mobile that doesn't just push all the buttons as normal but has a totally different presentation and code to support it) and some of it will need much greater change if it is to be facilitated, especially the uploading workflow stuff. (Btw, I'd like to see the AJAX uploading of attachments as a core feature just in general. Would be easier to support it in Aeva too if the support files are already present.)
171
Other software / Re: SM.org compromised
« on July 25th, 2013, 07:25 AM »
Quote from Pandos on July 24th, 2013, 06:43 PM
For me (and perhaps others) this would be a chance to be compliant with the (local) law.
Even if you don't do it, nevertheless there's a way to do you evil and to bring you in trouble.
Just a thought:
How can you prove that you don't do it if someone else accused you to do it? ;)
You can't, of course. But however it is encrypted, it must be design be decryptable, and by consequence it must be accessible by the admin. All we're dancing around right now is how difficult it is for the admin to do that. Right now there's no block other than the knowledge simply to access the relevant part of the database. Enciphering or encoding elevates the bar but it just means someone will write a plugin to bypass it. Just because such a plugin is not permitted on sm.org does not mean it does not exist (in fact it was even written and published on sm.org for 1.0.x many moons ago), and a PM spy type mod will exist anyway. Ultimately this is still security theatre and if it ever comes up that someone has to defend it for Wedge, I will explain the issues with it from our point of view. There should not be an expectation of privacy beyond whatever the admin wants to give.
Quote from xrunner on July 25th, 2013, 12:18 AM
My bank uses some kind of extra check where if you log in from an "unknown" computer, it asks you verification questions. So even if your password is stolen it can't be used on another computer without knowing the answers to very specific personal security questions. It's too bad that technology isn't available to all, but it probably adds a lot of complexity.
That's sort of the point. How much security is too much security? There is also a world of difference between a typical forum and a typical bank. But it's possible to have a bank too secure to allow you to use it, which is not really that much use to you, and a feature that's too complicated to use is not really worth it.
Quote from Hristo on July 25th, 2013, 04:14 AM
@xrunner My bank used to require from us to download a certificate file and to install it in the browser. If you are using browser without certificate you can't login. But this method was deprecated, now they require digital sign. SMF's Login Security mod offers some additional protection - user defined IP range of which he/she can login.
It's interesting, sure. One thing I would note is that it would be possible to add these sorts of things to Wedge as a plugin. The Login Security mod is actually not as effective as it might sound, depending on how flexible your IP address is and it is entirely possible to lock yourself out of your admin panel and have to resort to DB edits to get back in... the problem then with us adding that is that we'll be the ones who have to help people who get into that situation and to be honest that's really not something I want to encourage.
172
Features / Re: Soft merging of posts
« on July 25th, 2013, 07:17 AM »
The thing about Stack Overflow and similar Q&A is that most people still manage to screw it up and post new messages rather than replying to things as they wanted to do, and the whole system gets screwed up as a result.

I'd be inclined not to have 'render just this layer' thing if possible... no need to get *too* complex with it. There is a level of functionality we need, and pretty much we have that. Anything else we'll add when we need it.

As for the alternation fixing, the solution to that would be to call the next page of results as not already-prerendered and then add it at post time, e.g. bring it from the server in JSON or a similar wrapper and have the wrapper be expanded to include classes in the client.
173
Features / Re: Soft merging of posts
« on July 24th, 2013, 05:47 PM »
When I first saw it in one of the other topics, I was like... is that a bug? It's neat, and yes, threaded does pretty much demand not physically merging. (Though I still don't see how we fix the various UX issues with threaded even if the technical implementation isn't a killer. Show me a site that actually does it well... Reddit doesn't count because it looks and acts like a mess, sorry.)

I like what you've done, once I realised it was supposed to be like that ;) Doing it at JS time is an interesting solution and does generally make it cleaner on all levels (knowing the skeleton stuff etc.) though I can argue there is a usability issue if you have a run of posts from one person, e.g. like in New Revs, and the user info is way way way back up... though that's largely true for all long posts anyway.

And yes, tampering with it at the skeleton level would be incredibly messy, though I daresay it would be possible. I would expect it to become very convoluted if so done, and it's already got a little bit of a barrier to entry for modders (though once they get round the idea of skeletons and miniskeletons, which they'll have to for pretty much any plugin, it's not so much a big dea), so making it more complex is not a good idea.

I do wonder what will happen if themes (which modify templates, rather than skins which just modify CSS) come along and change that up. Perhaps the variable is a good idea so that a theme can indicate it as such.
174
Other software / Re: SM.org compromised
« on July 24th, 2013, 05:40 PM »
If you do that per user, you have to maintain copies for each PM/each user. And if you then change the password you have to rebuild the message (decrypt it with the old password, reencrypt it with the new one)

Doing it per message does the job but if they already get the database and get the encrypted data, boom, they already get all the keys too.

Regarding the legal status of PMs, I'm not being funny but that really isn't our problem. We cannot police what users do or don't do with our software. It is the user's responsibility to comply with their local laws. We don't give them anywhere that they can actually do it so they have to access the database. Now, if they do, that's up to them. The chances of *accidentally* seeing it in the database are so negligible it's not even worth contemplating (the only way it is possible is if you accidentally see it in the database backup SQL file... and we don't have a database backup feature)

The method described in that mod for vBulletin is one that has been discussed before and if you bothered to even read it properly it would tell you what it does and why it simply isn't suitable. All it does is encipher (not even encrypt) the content and since it's decodable trivially, it really is no better than leaving it unencoded in the first place. And if we add it to Wedge, the method by which this is handled will be entirely visible anyway... so all it actually prevents is admins who aren't entirely above board in the first place (who wouldn't actually install it)... in other words, for every use for which the plugin is designed, it's actually useless from a practical standpoint and constitutes security theatre.[1]

I realise this is important to you but I feel you're just not listening to the points I'm making.
 1. Something which by its existence and use leaves people believing they are more secure than they actually are. It is one thing to be insecure, it is something else to be insecure whilst believing you are secure. This is mostly the latter.
175
Other software / Re: SM.org compromised
« on July 24th, 2013, 09:28 AM »
It is borderline to impossible to do it sanely, yes. You need to be able to keep the original content in an accessible fashion and whatever you do, the decryption key must by definition still in the database too... a hacker by definition can obtain all the content in the database to deal with it. Now, that doesn't rule out things like private/public key pairs but that does significantly ramp up the user usability issues, and that reduces their usability to only a few people; it would actually be easier just to remove the feature than worry about that because a feature that is so obtuse for users probably shouldn't exist; most people don't understand what private/public keys are, or why they would have to use them.

I don't really see how a site owner can be sued when he is legally responsible for all content on the site including private messages but the easiest way to solve that is to have a clause in the registration agreement that says the site owner has the ability to do so should it be necessary... users agree to it when they register. That said, investigating PMs without due cause is always tricky ground.
176
Other software / Re: SM.org compromised
« on July 24th, 2013, 08:58 AM »
-sigh- You do realise that all you do is simply slow down an attacker rather than actually preventing anything? And the slowdown is trivial to beat, not lengthy...

The PMs must be in a form that can be decrypted by each user. Unless you create one duplicate for every recipient (as opposed to the current situation of one copy per PM and one entry for each recipient) which is separately encrypted which opens a massive glut of issues in terms of storage (it would render the announcements system basically useless), you have to keep them decryptable without complex systems. Which means it's basically a waste of effort because it can always be decrypted by anyone who gets the database.

:edit: Did you actually read the thread you linked? It would explain pretty much every reason why this is a bad idea.
177
Other software / Re: SM.org compromised
« on July 24th, 2013, 07:03 AM »
Also, we really need to add a method by which user passwords can be force-reset by the admin on a bulk level like this.
178
Other software / Re: SM.org compromised
« on July 24th, 2013, 06:39 AM »
Just to elaborate, the entire plugin architecture in Wedge is different to SMF, there is no code remaining from the old packman. Live is right about the plugin upload aspect - but I'll elaborate as to what we do and why.

When you upload a plugin to the admin panel as a zip file, some basic stuff is done to validate the package (looks for plugin-info.xml, quickly validates it) and if successful, it is moved out of the temporary folder into one of the Wedge folders where it can be dealt with.

What then happens is the admin is prompted for FTP details and then Wedge performs the upload of the package itself. More accurately, it opens the connection, creates all the folders via FTP then proceeds to upload the files via FTP too. The very important detail here is that at no point does the www-data/nobody/apache user ever actually own the files, because they are uploaded with the user's FTP details so all the files are then also owned by that FTP user, making all those files write-protected against by other users even on shared systems. This is the really neat part about this: since plugins never have to edit files (not that, generally, they could anyway), permissions never need to be escalated.

I should add, I actually tested this on a real shared server and was able to upload plugins without ever having to escalate permissions in any way whatsoever.

Yes, it would be possible to add a plugin which allowed downloads of the DB but it would have to be done by the admin with actual FTP credentials.

Just one last thing: this was not, at least not directly, a vulnerability of SMF. Information has been provided to me about a method by which the password forcing could have been expedited, a method which I was already aware of and has even been discussed on this forum, but it is not a vulnerability in the software as such (though it probably should be modified to mitigate this sort of circumstance). All in all this was a user account being forced, and as ever the users are the weakest part of the security chain.
Posted: July 24th, 2013, 06:37 AM

Also, as much as I didn't want to, I logged into sm.org to reset my password.
179
Features / Re: More sidebar complications...
« on July 19th, 2013, 08:05 AM »
iPad 3 still doesn't show the sidebar in either portrait mode or landscape until I press the button. It's also smaller than it was yesterday, but it's still big enough to be usable.

Regarding the legality of OS X... I know some people are bothered by that, and I know that the way OS X is built, it just doesn't work properly on non stock hardware (which can have issues with virtualised machines) because they just don't have the drivers. I will be able to help debugging when I'm not on the road (and right now using an iPad connected to an iPhone for its 3G connection!)

There is an iPad emulator of sorts with Xcode but I don't really know how it's geared to browsing.
180
Features / Re: More sidebar complications...
« on July 18th, 2013, 04:52 PM »
Yay for doing it illegally. (The licence actually states it can only be used on Apple equipment. It is also unlikely it will work properly for users who have AMD processors.)