In general, recording "style" can be split into two broad camps:

1. The simple "done once you press Stop" style; and
2. Recording for further post processing

Style: Done after Pressing Stop

Method one is pretty simple. You setup iShowU to give you video that's usable as soon as you've finished recording it.  This is useful when you're doing a quick recording and you don't need to achieve a super-professional finish.  We find ourselves making these types of recordings when describing something and sending via email. The goals are a) speed - you don't want to spend 3 hours making a perfect recording, you want to get your point across without typing 3000 words. b) minimal size - you don't want to spend the next 2hrs sending a 4GB email. You want the video to be about 4MB.  

Method one recommendation: Use H264.  Grab at 5-10fps.

Style: Recording for further processing

Method two is what we've receive the most questions about so lets spend a bit more time talking about it. 

All you need to do is spend a couple of minutes thinking about the following questions. Doing this up front will save you time later (you won't be unnecessarily recompressing video, or having to record it again!)

Who will view it? and on what?

This is one of the most important questions to answer - it defines how you'll setup iShowU.  I'll give two examples to demonstrate:

1. YouTube -You can't really do full screen recording of a 1920x1600 display because it'll never be visible on the YouTube site. So you need to either change my monitor resolution, or choose a smaller area to capture. Probably 1280x720 as an absolute maximum.  We recommend Apple Intermediate because you might want to import into iMovie and zoom in some areas. Zooming is really important. YouTube lowers the visual quality so much that if you want your video to be clear you're going to need to zoom into various areas to make sure that the important info is clear.
2. HD recording, onto a BluRay disc.  OK, Recording the 1920x1200 display won't be a problem. Since you know that your viewers will likely be using 1920x1080p you can use less zooming (not required, because the footage won't be munged by a web compressor like YouTube).  Still record in Apple Intermediate, because this will import nicely into Final Cut Pro.

You can see from this that choices regarding possible recording resolution need to be made up front, based on your audience and the method they will use to view your media.

Don't compress until the end

Maximum clarity is attained by not recompressing the video every time you move it to the next step.  In general, the less you mess with it (changing codecs) the better it'll be.  So our advice is this:

Compress to H264 / FLV / WMV only when you're done editing

For recording using Apple Intermediate.  This format is understood by all Apple tools (iMovie, Final Cut). It's fast enough, it's retains good quality and uses about 10 times less disk space than RAW data.

Do a test run

Make a 30s video and put this through your intended workflow. Make some edits to it, and a do a couple of transitions and cuts so that you can get the confidence that it'll all work as you expect for a larger recording.

This will also let you preview the final output quality.  Doing this can save you a significant amount of time when doing your real recording.  Manipulating 30s of video, compressing 30s of video (in general, doing anything with just 30s of video) is far far quicker than trying to import / edit / recompress 45m of video!