Stir together the chicken livers and milk in a bowl. Set aside to soak for 2 hours then drain and discard the milk.
Melt the butter in a large frypan over medium heat. Stir in the onion and garlic; cook and stir until the onions have softened and turned translucent, about 10 minutes. Stir in the rosemary, thyme, bay leaves and drained chicken livers. Cook and stir until the livers have firmed on the outside but are still pink on the inside, about 5 minutes.
Transfer the liver mixture to a bowl and pour the wine into the frypan. Simmer the wine until it has reduced to 1/4 of its original volume, then return the livers to the frypan and continue cooking until no longer pink in the center, about 5 minutes more.
Remove and discard the bay leaves. Scrape the livers into a blender or food processor and add the capers, salt and pepper. Puree until smooth then scrape the mixture into a serving dish. Cover with plastic wrap and refrigerate at least 4 hours until cold.