Porteus is a portable Linux operating system which is able to be installed on a portable USB device.
"Porteus manages to offer a full-blown distribution by installing in a compressed state (to keep its footprint small). During the boot process, it uncompresses and boots very quickly. Porteus also offers a package manager so you can easily install more applications. The package manager has a dedicated Porteus repository to select from, but you can install applications from any repository. All installed applications are treated as separate modules and can be activated and deactivated any time (to retain the speedy boot time)."

If you would like to know more and how to install it, check this article in Tech Republic.