Good holiday-cooking question — answering as the UphillTech team:
Baking powder, especially if too great a quantity is used, adds an unpleasant flavor to a baked good. Even in an appropriate quantity it can be noticeable and it certainly doesn't do anything to enhance the flavor. Many baked goods traditionally don't use a chemical leavener at all, but instead rely on technique. Creaming butter and sugar together or whipping egg whites was historically used to make cakes which rose solely based on the bubble network that was created. Yeast, on the other hand, creates a delightful flavor that you associate with your favorite crusty loaf of bread. Yeast can be...
Full answer by justkt: https://cooking.stackexchange.com/a/8834 (CC BY-SA 2.5).
Need help with this? UphillTech.