Water is such an extremely versatile liquid and one the reasons for this is that water can dissolve many different substances. And that includes the many minerals underground that water come into contact with before it is pumped from the ground. The most common minerals found in hard water are calcium and magnesium.
Rainwater has no minerals in it, and so is naturally a very soft water. Water drawn from a well is likely to have hardness problems because the water is drawn directly from the earth with no processing. Many residents of rural areas have water softening systems. People who use regular 'city water' have had their water already processed (which typically includes softening, as well as the addition of chlorine and fluoride).
Unless it's really hard, you can't really notice any difference in taste and it won't cause your body any harm. Though the water itself may not have any adverse flavour, the minerals can effect your coffee or your tea. Coffee can develop a bitter undertone when brewed with hard water, and delicate green teas can end up with a completely flavour altogether.
There are several other negative effects that come from hard water, one being that soap and detergent won't foam up which can make washing difficult. Another related problem is that the insides of your coffee maker, espresso machine or tea kettle will get a crusty buildup on minerals on the insides. These deposits are called 'mineral scale' and can be a pain to clean.
If you have hard water and find that it's causing your coffee or tea to be distasteful, you may want to switch to bottled water for your brewing. Or get a water softening unit.