3
You're right, it isn't intuitive. What you need to do is the following:
1. Create the custom block.
2. Go to Groups and then the select the "Anonymous users" group
3. Give the "Anonymous users" group block access rights to your new custom block.
That should do it... By the way, if you don't want registered users to see your block after they've signed in just remove (uncheck) block access for your "Registered users" group.