Breathing is something that beginners need to actively control when they are practicing - correct breathing will come naturally to advanced grades who have practised properly during training sessions.
How to breathe
You should breathe in through your nose and out through your mouth. Ensure that you are not simply breathing with your 'upper lungs' - you need to breathe as deeply as you can to get as much air as possible into your lungs. Ensure that your stomach is moving in and out when you are breathing, and that your ribs move up with you inhale and back down then you exhale. Pay attention to these movements and ensure you are doing this with every breath.
When to breathe
In order to maintain rhythm and ensure that your entire body is united when expressing a technique, you need to exhale when striking. Vocalisations help with this, as you need to exhale sharply to make a noise when striking.
You should apply this concept to all aspects of your training; exhale when performing the harder part of each exercise (i.e. the upwards motion on a push up or squat). This will get your body used to inhaling and exhaling at the right times, and will ensure healthy blood pressure, heart rate and oxygen transfer when exercising or fighting.
Effects of not breathing correctly
Taking a punch to the stomach can wind you if you are not breathing correctly. However, you can seriously injure yourself even if you have not been hit. If people have weak cardiovascular or respiratory systems, and frequently hold their breathe while contracting their abs and diaphragm or exhale sharply while constricting the glottis, they can induce black outs or even strokes. In order to perform techiques safely and effectively, it's essential that you learn to control your breathing.
Thanks to Chi Liu for the image.