OK, the issue as was originally pointed out is that calling add_plugin_css_file will create a new file in the cache, rather than integrate it into the main single CSS file.
There are both good and bad points to this, and it sort of bothers me that it would be left to plugin authors. If you add it on every page without fail, e.g. there is a bandwidth benefit yo getting it into every single CSS file that's cached, with the downside is that cache evaluation goes up every page as extra paths are evaluated.
On the other hand, if you indicate an extra file to be loaded, that goes through the minifier and is then essentially cached with a long life time on it, but it is a second HTTP request. Now, that might not be best, but I'd argue it might be preferable to the hordes of cached CSS files.
Right now, for example, I as a logged in Chrome user get chrome-member built. It's also possible that I might have triggered the guest version for Chrome also to have been cached. Then I add a style sheet that appears to both guests and members, but not every page.
Now I have chrome-guest, chrome-member, plugin-chrome-member, plugin-chrome-guest. Now multiply that by permutations based on browser sheets, and plugin combinations that might emerge. 3 plugins with their own stylesheets that may or may not be loaded at the same time gives you 8 permutations of CSS, multiplied by guest/member, multiplied by browser branches.
I figured it wasn't worth the effort and that it was better not to go down that road at all.