I had the same problem. Check to see that you have GD2 libraries enabled in your php.ini file. You must have a graphics library enable to make the captcha image show up.

No captcha image: I have the same problem with one of my host company. I checked with them, they have the GD2 installed:
Yep, we do have GD support current version is 2.0.34 compatiable.

GD Support enabled
GD Version bundled (2.0.34 compatible)
FreeType Support enabled
FreeType Linkage with freetype
FreeType Version 2.1.7
GIF Read Support enabled
GIF Create Support enabled
JPG Support enabled
PNG Support enabled
WBMP Support enabled
XBM Support enabled

I seems that the captcha could get into the fonts and backgrounds to create the image.

The root path of my server is:
XOOPS_ROOT_PATH = /home/.chaxyz/wonyiu1018/xp.locusland.net

Will using a ". subdirectory" setting be a problem?

My another host server does not have the problem with captcha and it's root path does not use ". subdirectory"?..