Adobe is probably a better solution.
If your soil isn't right for making adobe, you can make cement stabilized soil if you aren't 100% set on it being "natural" or possibly lime stabilized (more natural)
I've made cement stabilized blocks using my
local soil (>95% sand and rock) with cement at a 9 to 1 ratio. It dries hard and has lasted 12 years so far in the rain, etc. without any noticeable degradation. I'm working on building a driveway using similar mix.
Another possibility that I've seen frequently in the middle east, is to create blocks, etc. out of mud, dry the blocks thoroughly, build a wall out of them, then coat the wall with a cement/lime soil mix. This uses even less cement and the wall is almost as durable. However, the coating can be a little fragile and can break off if you bang it with something (wheelbarrow,
tractor, etc.)