4
What about caching? Both of the modules and the blocks. This could help your site tremendously, but I didn't see it mentioned.
The number of groups really shouldn't be an issue as the amount of program effort to go through one group vs a dozen to check permissions should be relatively negligible. Interestingly, too, XOOPS caches this in the session somehow because if your groups change, for the system to recognize this, you have to logout and then log back in. However, I've been know to be wrong.