So I ask Mr. Google specifically about using the leaves and yes, you can.
This company is one I have seen recommended on the forum:
A member of the mucilage containing Malvaceae family, Althaea officinalis has been consumed for millennia as an edible food and used for its gentle, soothing properties. Our marshmallow leaf is organically cultivated in the United States. Marshmallow leaves can be used topically in skin care recipes, blended into herbal syrups, or infused as marshmallow tea.
https://mountainroseherbs.com/marshmallow-leaf
Enjoy and let us know how the tea tastes!