I've not made a concerted effort to try and do what you're doing, but some observations that might help...
Your idea to look at it in reverse is a good one, but also highlights the fundamental challenge.
XOOPS handles all rendering of information to the screen using its core libraries. Every module, and everything in the core of XOOPS, is all setup so that once the logic has finished doing its thing, all the details are sent to a page rendering function of one kind or other that spits out a formatted HTML page for the web browser.
What you want to do is decouple the rendering of that information from the rendering engine itself. If the information were rendered separately from the XOOPS core and then handed off to XOOPS, that would make it easier. But that's not the way it works.
However, if you look at the main template file for the default theme, you'll see that by the time -- very late in the rendering process -- that the master page gets drawn, the center block contents are contained inside one single smarty variable. You could rewrite the entire template for the theme your XOOPS site is using and simply display that single smarty variable in a different way.
I think, though, that you are more interested in displaying the output of a module on another site, rather than simply displaying the output of a module without the XOOPS theme surrounding it? Unless the module author has created a way to export the information in the module for use on a non-XOOPS page, you'll have a very hard time doing the former. But simply stripping away the XOOPS theme, that's fairly simple if you modify the template.
But if you completely cut back the template, then you'll have no navigation controls that will let you get from one module to another.
If you setup your site in frames, and had navigation links in one frame that changed the contents of another frame, then you could make all those links call up a different stripped-XOOPS page in the other frame, that might work.
If you strip back the theme though, you'd have a hell of a time working with your XOOPS site to get information into it, since there would be no navigation elements, login boxes, etc, in your XOOPS site any longer.
Perhaps there's a way to pass a variable in the URL that will specify a template to use, so you could strip down the default template, and then for purposes of actually logging in and working with the XOOPS site, you could use a slightly different URL, ie:
http://www.myxoopssite.com/?template=admintempYou'd have to ask someone else if there is such a template hack out there. If there is, it would probably make more sense to use the URL passed template as the stripped down one, so your XOOPS site works like normal by default, but inside your frame site, the XOOPS content is displayed without the basic template.
Confused yet?
--Julian