Preheat oven to 180 degrees C. Grease and flour three 20cm cake tins. Combine the buttermilk, bicarb soda and salt. Set aside.
In a large bowl, cream together the margarine and sugar until light and fluffy. Beat in the egg yolks one at a time, then stir in 1 teaspoon vanilla and almond essence. Beat in the buttermilk mixture alternately with the flour, mixing just until incorporated. In a separate bowl, beat the egg whites until they form stiff peaks. Gently fold the egg whites into the batter. Stir in pineapple, coconut and 1 cup pecans.
Pour batter evenly into prepared tins. Bake in the preheated oven for 35 minutes, or until a skewer inserted into the centre of the cake comes out clean. Allow cake layers to cool 10 minutes in pans, then turn out onto wire racks to cool completely.
To make the icing, cream together the butter, cream cheese and icing sugar until blended. Stir in 2 teaspoons vanilla and 1 cup pecans. If icing is too stiff to spread, stir in milk a teaspoon at a time until it reaches desired consistency. Ice the cake with icing between the layers.