I'm trying to figure out a 'simple' way to prevent nesting 'li' tags inside others...
That's the beauty of the nesting logic as it stands - if the li > li isn't inside a list (i.e. it's not li > list > li), the inner li should not be parsed by the bbc parser. That's why preparsecode munges it first, so that you always get li being contained inside a list and not anything else.
You don't need to explicitly disallow li as a child, as IIRC disallowing as a child may even disallow it entirely in the nested chain which isn't necessarily desirable (because IIRC it would disallow the otherwise valid permutation of li > list > li)