Cyberciti.biz has a tutorial on how to save your settings and data to a USB device on the Ubuntu live CD.

The tutorial involves use of the terminal. If you want to save your settings to a thumb drive without having to set things up with the terminal, there are other distros that have this feature built in. Examples of distros that include this feature are PCLinuxOS, Slax, Knoppix, and Puppy Linux.

After I try this technique with the Ubuntu live CD, I'll post my results here and see if I can explain it in simpler terms.