5
At some point, that behavior was changed.
In XOOPS 2.0.14, that setting causes recompilation of all templates, including module templates.
In XOOPS 2.2.4, module templates do not get recompiled.
I've been planning to figure out why, but haven't got around to it.