Actually, that's exactly what the installer does... It deletes the install.php and install.sql files, and only those.
I didn't test whether I could 'simply' try to unlink the install folder itself, but it didn't matter much... Especially since the install process is now streamlined in a way that makes sure (1) install.php is never called independently ("hacking attempt"), (2) it's never called again after something is installed.
I did find a bug in the test for obgz though, it links to install.php instead of index.php so it'll probably fail all attempts... I could link to install/install.php instead and add the obgz test before the hacking attempt test, but I don't think it's worth it... I'll commit an updated version, hopefully it'll work.
As for the rest, I don't know if it's worth changing...