Prep chicken discarding skin and bones. Cut chicken into nice sized chunks.
Chop the celery and onion; saute in the olive oil until they start to soften, then add in the seasonings, frozen carrots, garlic and chicken broth. Bring to a boil.
Add the frozen dumplings and cook, covered, about 30 minutes.
When they are tender, whisk the cornstarch into the water, adding that with the milk and the chicken.
Return to a simmer then cook another minute or so until the slurry thickens the broth some.