UTF8 only!


  • Dadman with a boy
  • Posts: 16,061
UTF8 only!
« on May 6th, 2011, 05:10 PM »
Feature: UTF8 only!
Developer: Arantor & Nao
Target: modders, translators, admins
Status: 99% (believed to be complete.)

SMF was celebrated for supporting all possible charsets in its codebase. This was back in 2003. Work on SMF2 was started in 2005. Six years later, UTF8 is supported everywhere, and text string size is not really a problem anymore.
We decided to drop support for all charsets and force the use of UTF8 everywhere. Ultimately, this will be a blessing to modders who always had to make sure their string manipulation routines was using the correct charset path.
Also, $smcFunc functions were slow, we worked on their performance when moving them over to the westr object. Generally speaking, everything should be made easier with this move. It's time to say goodbye to ISO-8859-1.

Re: UTF8 only!
« Reply #16, on May 9th, 2011, 12:47 AM »
I think it's even part of 8859-1 actually, which does define the very most common accented characters.

But in Wedge it can be 'just used' without any problems.

Re: UTF8 only!
« Reply #17, on May 9th, 2011, 12:59 AM »
-15 doesn't have a lot of differences with -1, it just adds a few special chars including ¤, yeah I think it's in 8859-1 then.

Re: UTF8 only!
« Reply #18, on May 9th, 2011, 01:15 AM »
I hated the fact that many forums didn't support it...

Re: UTF8 only!
« Reply #19, on May 9th, 2011, 01:19 AM »
But SMF always did, AFAIK? (It's just a matter of adding accept-charset="ISO-8859-1" into your textareas... Browsers do the rest of the job!)

Re: UTF8 only!
« Reply #20, on May 9th, 2011, 01:22 AM »
Yeah, hopefully... Anyways, the bastards in the Royal Academy are so fu*** up that the Ñ will die in a few years...

* DoctorMalboro facepalm against the new "language management"...

Re: UTF8 only!
« Reply #21, on June 22nd, 2011, 07:33 PM »
Wedge is more Simple ...... forum

I like that.


Re: UTF8 only!
« Reply #22, on July 19th, 2011, 07:20 AM »
Is the backup problem with UTF fixed? (Where if you used the backup feature in SMF, and restored the DB, the charset was fubar.)

Re: UTF8 only!
« Reply #23, on July 19th, 2011, 09:47 AM »
I never tried doing that.
Don't believe it's been tackled either way.

Re: UTF8 only!
« Reply #24, on July 19th, 2011, 11:09 AM »
There is a part of me that just wants to ditch it because it's not suited to large dumps and has all kinds of odd failure conditions, and part of me wants to find a better way.

Re: UTF8 only!
« Reply #25, on July 19th, 2011, 12:30 PM »
Perhaps integrating the backup feature with something like bigdump?

Re: UTF8 only!
« Reply #26, on July 19th, 2011, 12:57 PM »
Here's where it gets complicated. Hosts that use cPanel etc, already have a backup facility, that isn't tied to PHP memory limits or Apache timeouts etc. meanwhile folks on unmanaged setups (typically VPS or meatier) will be running their own backup scripts anyway (or *should* be)

It's only then for the hosts that are that bad who don't provide backups AND don't provide access to anything else, in which case you're still screwed anyway since the majority of those hosts don't allow access to SMF's backup service either.

Re: UTF8 only!
« Reply #27, on July 19th, 2011, 01:57 PM »
I would rather you deleted the whole backup function then. I was almost burned on this one, lucky for me my previous experiences had taught me to have two backups from different sources... (one from SMF and one directly from MySQL.)

I restored the database and bam, thousands of "foreign" letters, all gone. (Well, not gone, just mangled.) I promptly changed to the direct from MySQL backup, and stuff was fine.

So yeah. It sort of needs to be addressed. I'd say it's a pretty big bug.

Re: UTF8 only!
« Reply #28, on July 19th, 2011, 08:38 PM »
Awesome :)