This topic was marked solved by Arantor, on February 10th, 2013, 05:48 PM
« Tabs in code tags
Long cache keys make the cache fail.
Input boxes not working in Chrome »

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Long cache keys make the cache fail.
« Reply #15, on May 2nd, 2012, 01:51 PM »
We salt the hash by using the username as part of the hash itself meaning that the rainbow table will have to be regenerated for every username. Though the multi-iteration hashing will also slow down generation of the rainbow tables too. Salting is a given regardless of how the hash was done ;)


Nao, what you've got there seems workable; all the embed URLs are embedded as [url=blah]blah[/url] IIRC, so you should be fine with that.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

nend

  • When is a theme, no longer what it was when installed?
  • Posts: 165
Re: Long cache keys make the cache fail.
« Reply #16, on May 8th, 2012, 02:12 AM »
I found a option, that works. You guys are probably going to call me crazy for doing this, because this is one of the things you planned to strip out of SMF. I ended up making a cache with another MySql database, that was slow though so I went with another option. This option allows keys to be longer than the file cache and is typically safe, so no key conversion required. I ended up using SQLite as the cache and it seems comparable to file caching, with slightly better write performance.

This is the first time I used a SQLite DB, so I probably have it all set up wrong, hopefully not.

I do have a problem with the set up, I am sure I set it up right according to the manual, but I can't get SQLite's auto vacuum going. :whistle:

So the file grows and grows. When data is removed it is deleted but with null, so you have padding. The DB system uses any empty space, but who wants a file that may be grow big without any way to shrink it. Auto vacuum is suppose to return this space to the file system but I can't get it to work.

Code: [Select]
function sicache_setup() {

global $sicacheDB;

if ($sicacheDB = new SQLiteDatabase('./cache/cache')) {
$query = @$sicacheDB->query('SELECT * FROM cache');
if ($query === false) {
@$sicacheDB->queryExec('PRAGMA auto_vacuum = 2');
$sicacheDB->queryExec('CREATE TABLE cache (key text primary key, value text, ttl int);');
}
}
}

function sicache_get($key) {

// sicache_setup();

global $sicache, $sicacheDB;

if ($sicacheDB = new SQLiteDatabase('./cache/cache')) {
$query = @$sicacheDB->query('SELECT * FROM cache WHERE key = \''.sqlite_escape_string($key).'\' LIMIT 1');
if ($query != false) {
$data = $query->fetch();
    if (!isset($sicacheData['sicache_purge']) && $data['ttl'] < time()) {
// Might as well purge the whole thing, only once though and only if needed.
$sicacheDB->queryExec('DELETE FROM cache WHERE ttl < '.time());
$sicacheData['sicache_purge'] = true;
}
return $data['value'];
}
}
}

function sicache_put($key, $value, $ttl) {

global $sicacheDB;

if ($value === null) {
@$sicacheDB->queryExec('DELETE FROM cache WHERE key = \''.sqlite_escape_string($key).'\'');
} elseif ($sicacheDB = new SQLiteDatabase('./cache/cache')) {
$sicacheDB->queryExec('INSERT INTO cache VALUES (\''.sqlite_escape_string($key).'\', \''.sqlite_escape_string($value).'\', '.sqlite_escape_string(time() + $ttl).')');
}
}
Re: Long cache keys make the cache fail.
« Reply #17, on May 16th, 2012, 06:07 PM »
I guess I can update on my alternative solutions, SQLite3 Cache.

After a while of tinkering and optimization, I am sort of bummed. I guess I was expecting to make a system that would surplus the file cache system, not the case though, but a alternative it is. While not surpassing the file cache it is almost equal to it, sometimes slower, sometimes faster, so equal.

I got this idea from a Word Press cache that used either the popular memory cache systems, SQLite or file cache.

I don't know what else I can do, totally bummed about the outcome of it. A simple script though so didn't waste too much time into it, but still... :(
SI Community

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Long cache keys make the cache fail.
« Reply #18, on May 16th, 2012, 06:11 PM »
The biggest delay in the file cache, really, is the physical I/O on it, everything else is going to be a push depending on everything else going on all the time; SQLite typically has more going on with respect to CPU overhead but that's invariably worn down by the I/O overhead.

It's certainly been an interesting journey and I'm sorry to hear that you weren't able to get somewhere really interesting, but this strikes me as something that isn't quite in the scope of SQLite, though I'm not sure what scope SQLite has on the server anyway. (For embedded databases or uses like the history in Chrome, I can understand it, but not on the server as a poor man's MySQL.)

nend

  • When is a theme, no longer what it was when installed?
  • Posts: 165
Re: Long cache keys make the cache fail.
« Reply #19, on May 16th, 2012, 06:24 PM »
I am keeping it though because it solves the problem with the cache keys, but no performance gain. Just disappointing that is all I can get from it.

Here is the thing, the first cache hits start out slower than the file cache system. As the cache hits keep going SQLite gets faster and faster until at the end it has surpassed the file cache system in speed.

It seems the initial connection is the hardest. My theory though is SQLite on first connection continues to load the database, sort of a read ahead and loads the database into memory while PHP is doing its work. If it hasn't got into memory yet then a file request is needed, this can explain the slow queries in the beginning. Theory though, I can't find any documentation on how SQLite loads.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Re: Long cache keys make the cache fail.
« Reply #20, on May 17th, 2012, 04:45 PM »
The initial connection being the hardest is not surprising, because it's doing a bit more than just loading; it's also rejigging the file as well to a point. SQLite is pretty complicated.

I also didn't think it was async behaviour, that would certainly run counter to how PHP works (i.e. it's a blocking behaviour rather than a non blocking one)
« Tabs in code tags
Input boxes not working in Chrome »