Utilities
MSI Afterburner - This allows for total control over your videocard. Change every setting from fans to voltage.
Prime 95 - This stress tests your CPU for stability.
HWMonitor - This is a system info utility. Allows you to monitor voltage, temperatures, and fan speeds.
GPU-Z - This displays information about your videocard.
CPU-Z - This displays information about your CPU, RAM, and chipset.
Guides
P67/Z68/Sandybridge Overclocking Guide
X58/Nehalem Overclocking Guide
P55/Lynnfield Overclocking Guide
Phenom II Overclocking Guide
Bulldozer Overclocking Guide #1,
and #2.
Because of the nature of GPUs, it's hard to aggregate good overclocking guides. There can't be one guide for all 5xx NVIDIA cards, as they all have different characteristics. Even looking at something as specific as a GTX 560Ti, some manufacturers bin the cards, and there's a wide range of non-reference coolers that have an effect on the card's limits. A good way to get started in the right direction is to search for "(card name) review". Websites such as Guru3D, OC3D, HardOCP, and Overclockers Club will have a lengthy report on the overclocking abilities of the cards they review.