It's not where you would think it would be. The forum (assuming newBB) uses the system 'Comments Display Mode' configuration option for 'Flat' or 'Threaded'.

In the admin section 'System' -> 'Edit User', you can change it for each user individually. Look for the 'Comments Display Mode' option.

You can also change your own user setting (as can each individual user) quicker, in the 'Edit Account' on the home page (that is if you've given yourself access to it).

One more spot. If you want to change the default for new users. In 'System' -> 'Preferences' -> 'General Settings', you can change the 'Default Comment Display Mode' option to whatever you want new users to default to when viewing the forum.

Note, newbb does not have a 'Nested' view, if it sees 'Nested' it will assume 'Threaded'.

I had the same problem with SmartFAQ when I installed it on a PHP 5. It's not a PHP 5 problem. It's a problem in the admin class code. Details are below.

Xoops Version: 2.0.9.2
SmartFAQ Version: 1.03 (Final)

Problem:
After initial install of SmartFAQ, message that
the category/question/answer save had an error, with
no further detail.

Resolution:
In the category.php, answer.php, and faq.php files
in the smartfaq/class folder, there is a line
checking the class type of the object passed in.
It's checked against a lowercase literal string,
whereas the result of get_class returns a mixed case
string. Solved this problem by running the get_class
result through the strtolower php function. Also made
this modification in any area that did the same
check. Below are the lines changed in each file.

File modifications: (All files in smartfaq/class)
category.php ----
Lines 273 and 319 now read:
if (strtolower(get_class($category)) ! = 'sfcategory') {

faq.php ---------
Lines 579 and 631 now read:
if (strtolower(get_class($faq)) != 'sffaq') {

answer.php ------
Lines 212 and 260 now read:
if (strtolower(get_class($answerObj)) ! = 'sfanswer') {

Line 287 now reads:
if (strtolower(get_class($faqObj)) != 'sffaq') {
----------------------------------------------------------

Just change those lines (copy and past if you want). Will fix the problem right then. Be sure to do this on each file. Because that same problem will show up when you try to add an queastion and answer as well.