My 54 acres in Deming NM is clay soil and is rich in the right minerals. Just add compost. The people in the surrounding homes have successfully gardened. It's basically the same as what you're looking at. Go through your local ag extension to have your soil tested. Be prepared to spend a lot for a well. Worth it if you love the desert like we do!
