I wouldn't use perlite in the garden, but I've used it growing seedlings and it's fabulous in seedling mixes; I've used around 1/4 coconut coir, 1/4 perlite and 1/2 screened
compost.
In NZ perlite's widely used by organic growers, but I'm pretty iffy about organic=good. Over here it's mined, then 'expanded', so very much not sustainable.