Sounds and bays are both indentations in the shoreline along an ocean which take the form of naturally protected harbors. They are formed by glaciers, erosion, and sometimes the hand of man. The terms are often used interchangeably, and there are no governing rules about naming such places in English, other than convention. The habit of using the terms interchangeably, however, can be confusing to people who are trying to get a mental image of the geographic feature under discussion. There are a few small differences between the two which can help distinguish a sound from a bay.
A sound is an inlet of the ocean substantially larger than a bay, and it may be less protected. Sounds are often characterized by large open spaces of water. A sound can be deeper than a bay, and is certainly deeper than a bight, a name for a shallow ocean inlet. It is also substantially wider than a fjord, an inlet formed along a shoreline by retreating glaciers.
On some maps, a channel or strait between two bodies of land is identified as a sound; some definitions require a sound to have at least two entrances. This is the case with Long Island Sound, the body of water separating Long Island from neighboring Connecticut. It is also probably an example of the original usage for the word "sound" in regards to geographical features, as the word derives from an Old English word meaning "to swim," suggesting that a person could potentially swim across a sound.
A bay, on the other hand, is a inlet of water enclosed on three sides by the land. The mouth of a bay may be narrower than the bay itself, as in the San Francisco Bay, or it can be much wider, gaping out into the open ocean. The Bay of Biscay, for example, has a very wide mouth, but it is still recognizably surrounded on three sides by land. Bays also tend to be more shallow than sounds, and some of them have to be dredged to admit ship traffic.
Both bodies of water are important geographical features, especially for sailors. Many fishing communities base their fleets in sounds and bays, so that their ships will be protected, and sailors looking for anchor take advantage of the sheltered environment to take a rest from the open ocean. Major port cities are almost universally located along the shoreline of either a sound or a bay, to facilitate a large area for docking so that the city's trade can thrive.