We are mortgage-free and off-grid in a rural southern county of Colorado. We can report that, in Colorado generally, and this county of Las Animas specifically, you'll need the following to get to a "fully-permitted" house:
1. septic system permit (implies a successful perk test, and no special system designs) from the
local health department. inspectors are good about coming out, but document with pics anyway.
2. building permit (with an "engineered" foundation plan, from your local PE) from the county building department. Inspector doesn't always come when you call, so *always* document all stages with pics; you can refer back to these if anyone questions things later. Saves having to rip things open if inspector is having a bad day. Supposedly, inspector will come out at various stages of construction ... what really happens is he'll want to see the engineered foundation stamp at the permit application stage in his office, where you pay the permit fee, and that is it, until the final inspection at the end of construction.
3. state plumbing and electrical permits (state inspectors for these). Inspectors are scheduled through state website, and are good about coming out. With these folks, the best advice I've ever heard, and it may apply to all others as well, is to "expect some amount of code violations or things they want done
differently ... and just do it".
No other permits required that I'm aware of for a built "residential structure". Agriculture (AG) buildings (outbuildings) are less restricted, to the point where they don't really care what you do, if you call it an AG outbuilding. We have seen
COB homes, yurts, and others in the area, some at our homestead friends locations. Most likely, all of them had to have a septic system permit, possibly an engineered foundation stamp, and doubtful on anything else. Most are self-built, and probably self-financed, as ours was (to bypass loan-madness). Further into the pines, and nobody knows or cares.
As others have pointed out, "insurance" is an issue, as would a bank-financed construction loan ... these would make it nearly impossible to do what you want to do with alternative home types. But, insurance is a joke anyway, and adding in our wildfire-prone area ... no self-respecting insurance agency (that is profit-based) would show up around here ... they stick to the cities and their code-heavy environs. We self-insure (to bypass insurance-madness), we are our own fire department, and so on.
Hope this helps ...