Margo, it be interesting to see your jar test of the site soil if you know how to post a pic. As far as soil type run this report below, it will also identify flood planes or look @ FEMAs latest:
http://websoilsurvey.sc.egov.usda.gov/App/HomePage.htm Click "Start WSS" follow the prompts.
Whether it's earth bag, rammed earth, adobe brick, etc, the goal is to resist thermal conductivity (r-value) in part, not thermally bridge from the two surfaces (inner and outer). In addition, in wet/hot climate zones humidity/heat buffering that is a function of clay content. The aggregates provide more of the r-value, the clay is the "glue"or binder.
In your hot dry annual climate zone overheating from too much mass that slowly reacts to temp change of the south/west facing walls will be the biggest challenge. You will want to use low E/SHGC min. window areas and/or natural shading and low levels of heavy hygroscopic mass (clay), more insulation less dense aggregates would be better.
If you consider the bags as the insulation "core" of the wall assembly you do not need alot of clay especially in your silt/sand soils. I'd be more concerned about putting too much foundation bearing loads since it can only take ~ 1500 PSF (see attached). With that said, less dense light weight walls are better. So, any soil or light high r-value aggregate will do.
When it comes to the "skins" plaster/stucco now you need a good binding clay as a first "brown' coat base that sticks to the bags. Since your soil has low clay content it may need to be stabilized with portland cement or lime that is a stronger clue. High calcium lime would be a good start, or try Type S mortar that has magnesium in it, if not go to Portland cement. Would not hurt to bond the bags too.
I would hope the quarry has dug deeper and has a higher clay content but do go get a sample to jar test to verify that. If you want to see if your walls are exceeding 1500 PSF and can take the roof loads hire a PE. Settling and foundation issues are not pretty.
I know that may not agree with your book so I tried to provide technical reasoning. Any questions feel free to ask.