I would not like to do adobe or even stabillized adobe in Poland. Someone said that fired brick has properties of adobe without having its drawbacks - which is sensitivity to
water.
I'm a little bit cautious with perlite - my intuition is telling me that in 20-30 years it may become new asbestos :), but nonetheless I have used it for insulation of my chimney ducts set within stabilized adobe chimney.
Also for the wall that has contact with ground, you need a material that can handle moisture well, otherwise you will have to waterproof a lot and eventually it may fail. If I had to do anything underground I would choose: stone or solid clay brick (but rated for moist environment) or
concrete. These three materials proved to work. Quarries in southwestern Poland have wonderful selection of granite blocks (I wish I could get them here).
If you want it fast (is it temporary?) then I would go with popular Suporex or beton komórkowy instead of perlite blocks.