Preheat oven to 350°F (180°C). Heat oil in a large skillet over medium heat. Add mushrooms; cook until very tender and no moisture remains. Add garlic and rosemary; cook for 1 min. Add cream; cook until slightly thickened. Add salt and pepper; remove from heat.
Remove tarts from foil liners and set on a baking sheet lined with parchment paper. Fill tarts with mushroom mixture; top with goat cheese, dividing evenly between the 18 tarts. Bake until well heated through, about 10 min. Garnish with small rosemary sprigs.
Tarts and filling can be prepared separately up to 1 day in advance. Keep prepared tart shells at room temperature, covered, until ready to fill and bake. Let mushroom mixture cool; cover and refrigerate.