Preheat your oven to 325 F. Grease and flour two 9 x 4 inch loaf pans.
In a large mixing bowl add the butter and sugar. On medium speed using a paddle attachment or hand mixer, cream the butter and sugar for about 5 minutes until light and fluffy.
Add eggs, one at a time into the mixer set to medium speed. Beat for 45 seconds after each egg.
Add the vanilla to the mixture and beat an additional 30 seconds.
Add the lemon juice and lemon zest and beat an additional 30 seconds.
In a medium bowl, measure out the flour, baking powder, salt and nutmeg.
At low speed, add 1/2 of the dry ingredients to the butter and egg mixture, and beat until just blended. Then add half the milk and with the mixer still on low, beat the milk in until just blended. Scrape the sides of the bowl with a spatula between mixing in the dry ingredients and the milk. Repeat the process with the remaining half of the dry ingredients and the milk.
Pour the batter into the prepared baking pans and fill no more than 2/3 full. Smooth the tops with a spoon or spatula.
Place the pans into a 325 F preheated oven and bake for 85 - 90 minutes or until toothpick inserted in the middle of the loaf comes out clean and the cake is just starting to pull away from the sides.
Remove the loaves from the oven and allow to cool about 15 minutes before inverting and removing the from the pans. Allow to cool before icing.
Make the icing: In a small mixing bowl, add all of the ingredients. Stir rapidly with a fork until all of sugar and lemon juice are combined. Use immediately.