Spread the nectarine and plum pieces into the bottom of a 20x30cm baking dish. Sprinkle the raspberries over the fruit. Whisk together the flour, white sugar, brown sugar, ginger and cinnamon in a bowl; cut the butter into the flour mixture with a pastry cutter until it resembles coarse crumbs. Spread the crumb mixture evenly over the fruit and raspberries.
Bake in the preheated oven for 15 minutes and turn the dish around in the oven for even cooking. Bake another 10 to 15 minutes; if crumb topping is pale, place dish under broiler for 1 to 3 minutes to brown.