Place the bacon into a large pot over medium-high heat. Cook until crisp, turning as needed. Crumble and return to the pot.
Reduce the heat to medium and add the onion, green capsicum, red capsicum, celery and garlic. Cook and stir until the onion is transparent and the peppers are soft. Add the yabbies and cook until the liquid evaporates and the yabbies begin to brown. Remove the contents of the pot and set aside.
Pour the chicken stock into the pot and add the potatoes. If the chicken stock does not cover the potatoes add enough water to compensate. Bring to a boil and cook for 8 to 10 minutes or until the potatoes are soft. Add the carrots and cook for about 8 more minutes.
Reduce the heat to low and return the vegetables and yabbies to the pot. Stir in the milk and heat through. Do not boil. Season with salt and pepper to taste. Ladle into bowls and garnish with Cheddar cheese to serve.