Finely grate the zest, then juice 4 of the oranges. Arrange the duck in a single layer in a dish, sprinkle with the zest and pour over the juice, then turn several times. Cover with plastic wrap and marinate in the refrigerator for at least 1 hour.
Bring the duck to room temperature. Preheat the oven to 180 degrees C. Peel the remaining oranges and cut each into eight segments.
Heat the oil in a shallow flameproof casserole dish over a medium heat. Add the onions and celery and stir for 3 minutes. Add the garlic and fry for 1 minute until the onions are soft but not brown. Using a slotted spoon, remove from the pan and set aside.
Lift the duck breasts out of the marinade and transfer to the casserole dish. Spoon the onions and celery over them, then pour over the marinade. Add the potatoes and half the orange segments, scattering them around the duck breasts. Pour over the stock, add the chilli, if using, and bring to the boil.
Cover the dish and bake for 40 minutes until the duck and potatoes are tender. Season to taste.
Remove the duck from the casserole and slice them. Spoon the onions and celery into soup bowls and scatter the potatoes around the side. Top each portion with sliced duck, then spoon the juices over. Garnish with reserved orange segments and sprinkle with parsley.