Get out the shredder! The NY Times had a good piece on how long to keep stuff.
Also, if you have a lot of stuff to shred and no shredder (or a wimpy one), places like Office Max offer shredding by the pound, and banks and other organizations often have free community shred events, where you can bring your stuff. Just Google “community document shred [your area]” and it’ll tell you if anything’s coming up.